/t/ - Technology

Discussion of Technology

Index Catalog Archive Bottom Refresh
Options
Subject
Message

Max message length: 0/12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

Board Locked? Request Reopening

PNG and GIF uploads are temporarily disabled while we deal with a spammer problem.


8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

You may also be interested in: AI

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #11 Anonymous Board volunteer 02/05/2025 (Wed) 22:34:45 No. 17183
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Users can choose to download and share tags through a Public Tag Repository that now has more than 2 billion tag mappings, and advanced users may set up their own repositories just for themselves and friends. Everything is free and privacy is the first concern. Releases are available for Windows, Linux, and macOS, and it is now easy to run the program straight from source. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . Hydrus is a powerful and complicated program, and it is not for everyone. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
big n00b here, starting my selfhosting journey. Got a DS1515+ for free and I'd like to run hydrus on it so I have a 24/7 personal local only booru running. I tried to install it via docker but I'm too stoopid to figure it out. I got the suika/hydrus image from the registry, created a container and the log make it look like it's working but I cannot access it from the browser with http://yourdockerhost:5800/vnc.html or http://yourdockerhost:5900 using tiger VNC so obviously I'm missing something stupid and obvious.
>>17437 >Smart Screen I think that should only check exe on launch. I had hydrus running, downloaded some stuff, refreshed the inbox and got force-closed and client exe deleted with no recovery. Seems more like behaviour pattern recognition rather than simple hash popularity check. Though after reinstall all works fine with the same database same images etc.
Am I tripping and using the wrong blacklist or it straight up doesn't work when I download stuff? Had blacklist from tags -> manage tags in the all known tags & downloader tags and copy pasted my e621 blacklist and I still see vore/scat/etc being downloaded. The only thing the blacklist seems to do is remove the tags from the images. The only thing I can understand not working is the artist names since I imagine I'd need to add "creator:" at the start of every artist I want blacklisted because lol syntax but I don't get why the rest is still popping up.
>>17449 I always found hyrdrus to be a lot better when you take things more slowly and sort of ease into it. Don't just dump your whole collection and expect things to word out of the box. That's why I always recommend new users to only try a few images at a time just to get the hang of it. Keep doing what you've been doing but keep hydrus installed and always on your mind while still playing around with it until you're fully ready to commit. Definitely not brainlet friendly but goddamn amazing when you set it up the way you want it.
>>17472 >I always recommend new users to only try a few images at a time just to get the hang of it. This.
I had an excellent week. The first version of duplicates auto-resolution is ready for advanced users to try out. There's not much else in the changelog! The release should be as normal tomorrow.
>>17469 >Had blacklist from tags -> manage tags in the all known tags & downloader tags and copy pasted my e621 blacklist and I still see vore/scat/etc being downloaded. That section only controls display. What you want to do is go under "network" > "downloaders" > "manage default import options". Then click the "import options" button next to "default for file posts". Put your blacklist in "set file blacklist". This blacklist is will apply for all websites. You can set blacklists on a per-website basis in the section below.
>>17478 Thanks a lot. Some of these settings are put in some really dumb locations, legit spent an hour looking up on all the documentation and discussion to find where the correct blacklist is and it's always either the wrong thing or outdated images.
https://www.youtube.com/watch?v=aqzzowQcPZ4
[Embed]
windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Linux.-.Executable.tar.zst I had an excellent week. The duplicates auto-resolution system is ready for advanced users to try out. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html duplicates auto-resolution Any user can run this test, but I'll ask that you only go for it if you have a bit of experience with processing duplicates. Make a backup before you update, then go here: https://hydrusnetwork.github.io/hydrus/advanced_duplicates_auto_resolution.html Make sure your default duplicate merge options for 'better than' do not say 'archive both files' and then try out the jpeg/png rule. I want to know: if it all makes sense to you; where it runs smooth; where it runs garbage; do you get any errors. I'm also generally interested in your numbers. Out of ~800k potential pairs, I had ~6,000 jpeg/png pixel dupes, resulting in ~4,700 actual pairs (numbers shrink because multiple good files can share the same bad file). The preview panel ran pretty bad, about 30 seconds to load numbers, but the actual work was about 20k files/s for the search stage and 10 files/s for resolution. I cleared about 7.5GB of Clipboard.png in seven minutes! Thank you for trying it, and thank you for your patience as I figured all this out. next week Back to normal work, and I'll plan out future extensions to auto-resolution. Assuming the test goes well, I'll open up all the UI and invite everyone to try it out in v616.
Did the e621 tag importer broke again? I can no longer download any tags other than rating.
>>17484 https://files.catbox.moe/s4nzrh.png have to upload via catbox because "file format not allowed" on upload actually fixed, e621 changed tag format and added some shitty fucking "uploaded by artist" twitter badge in svg in the creator span that has to be filtered out note: this filters "contributor" to its own namespace, removes "creator:sound warning" because it's a trash tag and changes "creator:third-party edit" to "meta:third-party edit" via regex substitution.
When running version 615 on Debian 12, the program gives an error symbol lookup error: /home/%my user name%/Downloads/Hydrus Network/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags what can be done? Try the Flathub version?
>>17482 so, I did this, went there add suggested, apply, and missed out on some of the cool stuff that was also there because I was half paying attention. now, my thoughts on this process. 1) I like that it automatically does everything and that's about it, what I dislike 1) that it automatically does everything let me try to explain, I had a bit over 19k pixel perfect matches. it went down to about to about 26,000 images before it decided to not do anything, and I had to hit work harder... not sure why, what I would like to see happen is instead of auto deleting them (well moving them to the trash) I would like the program to effectively earmark these, and let me decide on how to resolve them or when to resolve them, more or less, the program knows they are duplicates but lets me review them if I want to, I personally see this as a massive thing when its not as obvious as jpeg/png pixel perfects, given how complex this seems, double checking a few of these to make sure it was set up right before I press forward would be nice. now, because I was a dumbass and skipped the interesting things, I have to refer to the images on the help page, again well aware i'm a dumbass, but the image with the preview is interesting to me, the one that shows the images side by side in previews would it be possible to do something like that will the current duplicate filter? maybe as a companion to the large full image view? my use case is when I have an image set that may be 10+ images deep and all of them are minor variants, or when the differences are fuzzy enough that even larger differences still pop up as duplicates, hell, I have several thousand pngs that are just shitposts someone made that downgrade image quality while keeping a large file size and are seen as duplicates in the filter, easy for a human, even from thumbnails to figure it out, but would take quite a bit of time loading up one image over the other. overall I like this, but instead of fully auto resolve, can we have the program parse it, and then let us open it in some way to make damn sure its doing what we want before we ok the next action?
>>17489 Nevermind. "Fix" this by using Ubuntu
mpv added HDR support for Wayland. https://www.phoronix.com/news/MPV-0.40-Released
Hi! I'm having this rare but still intermittent issue where the api just stops responding. The issue persists until restart. Windows platform. Usually it happens after I try to access videos via the api in quick succession. How would I even go about finding the reason for this? The "api test" option in the debug menu has no effect until the restart as well.
>>17484 Ah, I thought I had broken something myself, glad I checked the thread.
For duplicate auto-resolution rules, I hope to see fuzzy filesize comparison (A's filesize is within x% of B's filesize), comparison of image resolution and comparison of aspect ratios. For JPEGs, also quality/compression profile. I think having these types of rules available would make a lot of work with variant sets automatable, and even be able to reliably detect obvious better/worse pairs.
>>17488 >removes "creator:sound warning" because it's a trash tag and changes "creator:third-party edit" to "meta:third-party edit" via regex substitution. Why would you do that? If I wanted those changes I could do them non-destructively with display filters and siblings.
>>17501 Because I cannot imagine a single situation where having "creator:sound warning" as a tag would be useful. In fact, the tag is far more likely to be wrong than the built-in "has audio" file property.
>>17502 I don't care about your lack of imagination.
>>17502 it's useful if you don't want your ears blasted by a sudden loud noise. what's hard to get about that?
(30.33 KB 500x430 group of nuns.jpg)

>>17503 Then shut the fuck up and remove it from the parser filter for yourself. Here, I'll even give the exact path: Network -> downloader components -> manage parsers... -> "e621 file page parser" -> Edit -> content parsers -> creator tags -> Edit formula -> Big string processor button -> select "TAG FILTER: ..." -> X - > Yes -> Apply everything. >>17504 Then use the "system:has audio" predicate? Which does a better job? Out of 394 files with "creator:sound warning" in tags only downloaded from e621, 4 of them don't actually have any audio at all, and out of the remaining 527 files with audio, there's many that could be considered having "obnoxiously loud sound". I'd take it as a good, valuable tag if it were mechanically applied, like if the e621 website detected any sound past 30dB (arbitrary) on upload and added the tag (which shouldn't be under 'artist' anyway) but it's clearly not. To be fair, on the same logic I should also filter out the other various file tags like "meta:sound" or "meta:webm" because they're all useless, I actually forgot they even existed.
>>17506 If you're posting something on the internet for people to use, try to be user-friendly instead of enforcing your own arbitrary biases. Imagine if I posted a e621 parser that got rid of the species namespace and just moved them to unnamespaced, and then said "hurr durr i cannot imagine a situation where you would want the species namespace". You can see how some people might be annoyed by that, right? Especially in the hydrus thread were people are autistic about tags. I just don't see any reason to arbitrarily change the results to be different from what's actually on the page.
>>17504 If that's what sound warning was actually used for I'd agree.
>>17488 I'm dumb, explain this. To fix e621 tags not working I need to remove creator sound warning? I checked the catbox as well but it's just an image...
(33.23 KB 888x532 1.jpg)

>>17504 >it's useful Sure, but only for you. Imagine if much of these moot feature requests are fulfilled, the software would be so bloated and slow that anons would be demanding for a Hydrus Light version. What about if you try the Mute icon before you load the videos instead?
>>17511 >Imagine if much of these moot feature requests are fulfilled what feature request? the reason they have the tag on e621 is to let people know that the video has a sudden audio spike. nothing about hydrus here so I don't know what you're responding to >What about if you try the Mute icon before you load the videos instead? because that just... mutes the audio?
>>17511 >there shouldn't be a jumpscare tag in any context >just like, don't play the video
>>17510 The image has the downloader data inside of it. Download the picture and put it somewhere you can find. Go to hydrus>network>downloaders>import downloaders. Click on Lain's face and find the image, it will tell you what you are importing, click yes and now the parser's been added. Now go to network>downloader components>url class links, find e621 file page, edit and set it to the new one, which should be e621 file page parser (1), matching parsers should be at the top.
In case anyone used the e621 with notes variant I updated it to work with the new parsing thing. I didn't change the sound warning stuff because I didn't feel like it. No image uploads so here it is: https://files.catbox.moe/t6jw3w.png
>>17512 >because that just... mutes the audio? Exactly, then you adjust the volume located right on top of the Mute icon just before to click on a video. Not a big deal, I think.
any ways to make some images children/parents of others? i'm seeing the children/parent tag thing but it doesn't seem to correlate
>>17526 No, not yet, you can set them as alternates*, which just indicates they are related, it does not have any sort of hierarchical info. I think parent/child images might be planned but I might be making that up. *select 2 or more images>manage>file relationships>set as alternates.
>>17447 Great idea, thanks. I'd still like to completely rewrite how export filenames are created. We want something richer than the current template string, and we need better fallback logic for when some data is missing. I'll write this down and think about it. >>17448 That's an interesting idea, thanks. I'll write it down. >>17458 We do it for cbz now, although my cbz detection algorithm is a bit shaky, so if your cbzs are read as zip, let me know. I think it would be neat for normal archives to pick a decent thumb, although I know some users don't want it because many results will be unhelpful (imagine if you were storing lots of technical brush zips and the thumb every time was just a 20x20 white square or other weirdness), so I'd introduce it as optional for arbitrary archives, I think. We have zip reading right now (python comes with it), but I don't think we have any tech to read rars or 7z yet. I think it is fairly easy to add, but I haven't looked into it properly yet. I'd like thumbs for cbr and is it called 'cb7'? I generally want more archive-parsing tech in future. You mention a .thumb file--is that something you see often? Is that a standard in a particular .zip format you use? Are there any others I could look for? My general approach here has been to list all the .jpgs and .pngs and just pick the one with the earliest filename, since that works for cbz. As I (slowly) roll this stuff out, let me know how it works for you! >>17459 Thanks, I am glad you like it. If you are a new user, then, in a few weeks, once you are comfortable with things, let me know what has been difficult and easy to learn. Keeping the help guides updated and overall UI easy for new users to pick up is a constant battle. >>17462 Thanks. I am afraid 'rating' file sort does not mix with system:limit yet. I added a tooltip to the file sort menu button a couple weeks ago so it actually tells you now which do and don't work. That said, rating shouldn't be too difficult to add, so I'll look into it.
Edited last time by hydrus_dev on 03/29/2025 (Sat) 19:57:52.


Forms
Delete
Report
Quick Reply
Drag files here to upload or
click here to select them