https://www.youtube.com/watch?v=Xqg2KSdBLRU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v608/Hydrus.Network.608.-.Linux.-.Executable.tar.zst I had a good simple week. Just a bunch of small fixes and improvements. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights The new mpv error hook that detects the 'too many events' error is overhauled. It shouldn't fire unless there is a serious problem now. If you were hit with false positives, sorry for the inconvenience the past few weeks! After the recent successful 'group namespaces (user)' update in tag sort, and after talking with some users, I have decided to make that namespace grouping always sort the same way, no matter which way you sort the underlying groups of tags. This means, e.g., that 'creator' tags will stay at the top, then 'series' and 'character' etc.., whether you say 'most first' or 'fewest first'. See how you like it--I think it is more intuitive now. I improved how the multi-column lists across the program set their minimum and initial height, and in most cases and styles they appear to be pixel-perfect to the number of rows desired. Particularly, in the gallery and watcher downloader pages, where the multi-column lists dynamically size themselves based on how many items they are holding, it should fit exactly. I find the perfection a little eerie, so I may add just a few pixels of padding so the back of your brain knows there isn't anything to scroll down to. next week I finished the duplicates auto-resolution database work this week, so all that's left is a daemon to run everything and a preview panel. I'll see if I can get the daemon done.

just wanted to say: keep up the great work!

Just wanted to say: hi

Would it be possible to add an undo to pages? I sometimes think I have my browser selected and press F5, only to wipe a page on hydrus because the search is empty and I've been manually dragging files to the page. A searchless page could also work.

>>17188 This used to happen to me all the time, so I just deleted the shortcut.

>>17189 Not a bad idea.

>>17186 >>17187 Thanks lads, keep on pushing. >>17188 Yeah, I've done this myself. I have a barebones undo system, but I never wrote the original 'do a thing to a page' pipeline to plug into it correctly. Ideally, you want this chain recorded of all changes, and all changes tell the program what they did and how to undo them, but stuff like 'remove files' command just gets swallowed right now. I've slowly been converting my internal objects to better support this, but it'll be a while yet before I can support anything I'm happy with. On the other hand, I'll be more immediately working on some better tools to 'sync' a page to its current results. If you are a long-time-enough user, you'll remember that 'open these files' pages used to have no search. They wouldn't respond to F5 either. I replaced them with a page that spawns with a system:hashes search, for KISS reasons, and while you can flip the 'searching immediately' button to reduce problems, we are now broadly missing the ability to have a 'read only' page that just holds some files mate. I'll be figuring out a checkbox or something that freezes or pins a page as it is so guys like you don't lose their work on a scratchpad page.

In the duplicate filter, when you compare videos, it seems to always give the longer video a much higher score. it seems to give it around 50 points, with no way to change it. this happens even if the "longer" video is a single frame longer. this is probably a bug

Is there an option somewhere, or can we get it by default, that you get a notification for when cookies that are necessary for certain downloaders expire? >>17192 >keep on pushing. I will.

I had a good week. I mostly did boring cleanup work, but by serendipity I figured out a set of fun new file sorts that work using a file's average colour, so you can now sort from lightest to darkest, grey to saturated, and by hue. The release should be as normal tomorrow. >>17194 Thanks, I fixed it for tomorrow to do 1-50 depending on the relative lengths, capping at twice the length. That's probably still a bit wrong, so let me know how it feels and we'll keep adjusting. >>17198 The login system can kind of do this, except when a cookies is expired/missing, it then wants to start a login process. I'm not sure you can set up a login script that recognises a missing cookie but then won't cause domain errors when its empty or whatever login process inevitably fails. Maybe it is ok if it fails, since jobs on that domain will probably just cycle on 'login problem' delays until the cookie is filled in or you re-validate the login script for another attempt. Bit bleh all round though. Maybe I should consider starting an overhaul of that system, which I haven't touched in ages and is really starting to creak. If you use Hydrus Companion, it can regularly sync your cookies and you avoid these issues, but if you have to do it manually, in your situation, I think you might have to monitor it manually too. Maybe you can resync every two weeks or something. Sorry!

>>17200 >Maybe I should consider starting an overhaul of that system, which I haven't touched in ages and is really starting to creak. It's not a huge issue for me. Most of my cookies last for years, and the nearest expiring one is 4 months away.

>>17200 >grey to saturated, and by hue. This is immensely useful for tagging things as "black and white/monochrome/greyscale" or whatever tag one might use for that. It may also be able to sniff out things in other color scales for easy tagging. An image that's entirely in shades of blue is almost as lacking in color as an image in black and white.

Can I do a search of files that are only in multiple domains. Under 'edit multiple locations' it seems that I can do 'domain A' OR 'domain B', but not 'domain A' AND 'domain B'.

>>17203 Not sure what you really want, but it seems you want either 1. Search files that for example must be in domain A AND also in domain B. For that you should change your domain to one with a broad scope, for example 'all my files' or better 'all local files' (Advanced Mode), which would also contain trash. Then do several searches with 'system:file service: system:is currently in my files system:is currently in test location That way it searches for all files that are in 'my files' AND 'test location' at the same time. 2. You want to activate the checkboxes in 'edit multiple locations' to take a look into those domains at the same time and think you can't. Hydrus let's you activate the boxes that make sense. If you are not in Advanced Mode, you might have 'my files', 'all my files', 'trash' and maybe you added a new one, say 'test location'. If you check 'all my files', you can't check 'my files' or 'test location', because 'all my files' already contains them. You can check 'all my files' + 'trash' though at the same time, because 'trash' is not included in 'all my files'. In Advanced Mode there are much more. If you check 'all local files', you can't check all mentioned above, because it contains them all. You can for example check 'my files' + 'test location' + 'trash' at the same time, because you didn't check a parent that contains them already like 'all my files' or 'all local files'. Also this might help: https://hydrusnetwork.github.io/hydrus/advanced_multiple_local_file_services.html

https://www.youtube.com/watch?v=A1QUUmShhMM windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v609/Hydrus.Network.609.-.Linux.-.Executable.tar.zst I had a good week. I mostly did boring cleanup work, but there's some neat new colour-based file sorting to try out. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html macOS zip First off, the macOS release is now in a zip. We've had technical problems with DMG creation recently, and zip is much simpler and faster and more reliable. You'll now be double-clicking the zip to extract the App and then dragging it to your Applications directory yourself. I am recommending all macOS users consider running from source, since it skips a bunch of App hassle, and particularly so if you are on Silicon (the App is intel atm, which causes a very slow load time on Silicon). https://hydrusnetwork.github.io/hydrus/running_from_source.html colour sorting A user mentioned that the 'blurhash' we recently added stores a file's average colour. I played around with the idea and figured out some fast 'sort files by colour' tech. It works for all files with thumbnails. They are all under a new 'average colour' menu, and there are five types: 'lightness', which obviously does dark to light; 'hue', which orders files according to the rainbow colour wheel you will have seen in any HSL colour palette panel; 'chromatic magnitude', which is basically saturation and does grey to colourful; 'balance - blue-yellow', which uses the idea of 'chromaticity' and scales from the bluest files to the most yellow; and 'balance - green-red', which similarly does greens to reds. This isn't perfect, since it works on average colour, so a file with a very bright red blob on a dark background might be counted the same as a brown mush, but it is cool and fun to play with, and it is a good base for future, cleverer colour histogram tech. I'm really happy with how it turned out. Give it a go and let me know how it works for you. next week I did a bit of duplicates auto-resolution work this week, but I wasn't super successful. I'm going to do a bit more UI so I can get a better idea of how I want the daemon to work.

I made a new pixiv subscription and it failed to get any 18+ files, so I sent the logged in cookies via Hydrus Companion, tested that I could grab 18+ files by sending a tab via Hydrus Companion, and that worked, but deleting and remaking the subscription just gave me the same single SFW file, no 18+. What am I doing wrong?

>>17206 Figured it out. Had this issue before. Cookies don't stick right when sent from an incognito tab.

Some minor issues with the manage notes window that make it hard to use: >Whitespace at the start or end of the note is removed. Whitespace is also removed from the start or end of each individual line. Also, if note has multiple newlines in a row, even in the middle of the note, they will be turned into just one upon pressing apply. I want to use spaces to indent and I want to be able to have multiple newlines in order to organize information nicely... >If a file has multiple notes and you have the option "Start editing notes with the text cursor at the end of the document" unchecked, it only works for the initial note that was opened. Switching to another note tab will put you at the end of the document instead of the start. >It would be nice if when you pressed escape or cancel to leave the notes popup and you've made edits it would warn you like "unsaved changes will be lost. continue?" sort of thing. Please take a look. Thanks.

>>17205 >colour sorting Thanks, this is useful indeed.

Speaking of notes, hydev, could we get a mass edit option for notes? Or a least a mass delete?

>>17204 >1. Search files that for example must be in domain A AND also in domain B. > >For that you should change your domain to one with a broad scope, for example 'all my files' or better 'all local files' (Advanced Mode), which would also contain trash. Then do several searches with 'system:file service: > >system:is currently in my files >system:is currently in test location > >That way it searches for all files that are in 'my files' AND 'test location' at the same time. This is what I wanted, thank you. Trying to find files that are in BOTH file domains, so I can remove them from the domain they do not belong in.

>>17208 >Whitespace This is a tricky one. The notes import workflow relies heavily on a bunch of merge tech, and it is really useful for me to trim and collapse excess whitespace so notes from different sources can be compared well. That said, all this merge tech worked out pretty shit tbh and I hate it, so it wouldn't kill me just to make the text trimming smarter and add some options so users can do what they want. I'll see what I can do! >bad option Thanks, I will fix it! >warn on cancel Great idea! >>17213 This is doubly tricky, since the UI gets suddenly complicated and I need to do (even more!) merge tech to show how the files share notes and stuff, but I'll see what if I can make it work like the multifile URLs editing dialog, where it has some red text or something saying 'hey, careful, you are editing for a bunch of files here, all will be affected.' Some way to say 'add this note to the files that don't have it yet' would also be nice.

Are there any plans to make it so you can click and drag to select multiple files?

I just installed Hydrus in MX Linux and the MPV viewer won't load. The library libmpv2 is installed and even I upgraded it with the version in the Test Repository but I had not luck. Anyway, I'm just posting to inform devanon what and where MPV is failing, as I switched to hydrus from source and it is working great, see pic 4.

Just checking in to see how much of a crack dream reverse PTR + IPFS is With booru enshittification and the repeated attempts at making porn illegal I feel something like it may become necessary

>>17224 I'm no Linux expert, but I've seen that 'g_module_open_full' thing before--hit the 'Linux' tab here and scroll down a bit: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#installing Looks like you will be adding some .so files to your Linux install. I guess it is something to do with Ubuntu not agreeing with your Linux flavour about where some things should be. Failing that, running from source can help with these weird dll load problems since you'll then be completely native. >>17222 Yes, I would like this. I do it myself by accident sometimes, forgetting I still haven't added it. My thumbnail grid is waiting an important overhaul, after which I expect it will be much easier to insert this sort of behaviour. >>17225 IPFS use a slightly tricky hash/identifier for files--iirc it is a bit like a torrent magnet, in which all the blocks of the content have separate hashes, and the hash of the master file that lists those blocks is the main 'id'--and, as I understand, you just cannot predict an IPFS hash with only an SHA256 hash. And I'm not up to date, but I understand you cannot query the IPFS network using a hash like an md5 or sha256. So, while it would be possible to do this, you'd also need a way to propagate (SHA256-IPFS multihash) pairs so our hydrus db would be able to cross-reference tags and stuff to an IPFS multihash that would get you the file (so you could, say, do a search for files you don't have but which have the tag 'samus aran' on the PTR and have a known IPFS multihash, and that list could spawn a new downloader that would work pretty much like a booru gallery search). I've thought about this, and I generally call it a PUR for Public URL Repo, or basically a 'hey this repo stores known locations/ids for SHA256 hashes', for which IPFS multihash could be one mapping type we could add. Writing a whole new Repo would be a ton of work, and I am still of the mind that managing files remains the trickier problem to solve than sharing them, but I agree we've seen some more restrictions recently. I'm mostly concerned with the big CDNs putting anti-downloader captchas on many sites. And I'm worried AI is going to break captcha and make Real ID the standard for anything with a web domain. We'll see how it goes, but yes, I think there could be a way forward here if we wanted to. The Client API, also, is much more mature these days and is open to someone who had an interesting idea. I'm toying with the idea of client-to-client comms in the back of my mind, and I wouldn't be surprised if I slowly pivot over the coming years to a cleverer client, likely with AI tech to solve common problems like tagging character names and simple nouns, and retiring or minimising the server. We'll see how everything shakes out.

I had an ok week. I ended up mostly working on some boring cleanup and refactoring, so there's not much exciting to talk about. There's new file viewing statistics stuff and some misc quality of life. The release should be as normal tomorrow.

Hello! I'm having an issue with a custom parser. In cases when it tries setting two notes with different names but the same content, only one of them gets associated to the entry. Test parse always shows them correctly and separately, and they both always get associated properly when they differ in content. Could it be a bug?

>>17228 I was talking with another guy, I think by email, about this exact same issue this week, but we resolved it, so I presume you aren't him. My note merging logic isn't something I'm totally happy with, but it has a bunch of cases where it says stuff like 'if we are adding a name+text, and the name already exists and the note that comes with it is a prefix of the new text, then replace the existing text' in order to stop it adding name(1)+text duplicates. A similar thing basically says 'if the text we want to add exists anywhere, then don't add it, since it'll just be a dupe'. This stops you getting the same gumpf for the 'danbooru artist comments', 'gelbooru artist comments', 'e621 artist comments' etc... It isn't super beautiful, but it saves some spam. So, it isn't a bug. That said, I don't like it! And when the two duplicate note texts come in the same parse job, it pseudorandomly selects which will be the first. I've got several ideas and heard several user opinions on how notes should change going forward, some radical, but I am not sure what I want. How would you, in an ideal world, like to handle the overall problem of duplicate note names and/or texts?

https://www.youtube.com/watch?v=lUVCJGSTcO8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v610/Hydrus.Network.610.-.Linux.-.Executable.tar.zst I had an ok week mostly doing boring cleanup. There's some quality of life improvements. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights The new 'sort by hue' file sort puts anything that is a very grey average colour at the bottom of the sort. The 'edit notes' dialog has some nicer focus logic, and it will warn you on cancel if you have made changes. The menu settings under options->file viewing statistics are simplified. What was previously five different ways to show viewing stats in the file right-click menu is collapsed to two, and there's now a checkbox list of which view types you are interested in showing. The new 'client api' view type is integrated into the UI, and some views/viewtime system predicate parsing is easier to work with. I overhauled the parsing pipeline this week, and I fixed some weird parsing bugs and bad test UI presentation and so on. If you are an advanced downloader maker, let me know how things go. next week I got mired in cleanup this week and did not get to several things I wanted. I'll keep working my immediate small job todo and try to clear things out a bit.

hydrus that allows hardlinks/symlinks instead of copying files when? i don't care if the database breaks sometimes damnit

I think there is a bug when i 'sort by tags: number of tags'. So when i match the domain i am looking into in the search pane (the one under the OR button) to the 'tags' button domain (next to collect/leave unmatched button), let's say both are changed to 'downloader tags' and 'ascending', then the sorting is correct. I suppose you have to match them, because that way the tags you see are the tags you sort exactly. The first files have no tags, then when i hold the right arrow key, i can see how the tag list gets more and more populated with tags and the scroll bar in the tag list gets smaller and smaller. That is how it should be. But when i match the domain of the said buttons to everything else except 'downloader tags' (-> my tags, PTR, all known tags), the sorting doesn't work anymore and is all over the place. In average it might get more and more, but from file to file the number goes up and down. Now i realized how to fix it though. If i change 'sort by tags: number of tags' to a namespace sorting, a second 'tags' button appears right above the first 'tags' button. This is actually the one that changes the behaviour in the 'sort by tags: number of tags' sorting (which is not visible there), that means 'downloader tags' was activated here, so it coincidentally worked in 'sort by tags:number of tags', which i described in the first paragraph. So we would need THIS 'tags' button there instead of the one which is there, which seems it doesnt do anything in terms of sorting by number of tags. I think this 'tags' sort button is linked to the collection buttons left of it, so this changes the sorting of 'collecty by' collections. In short: how to sort by number of tags for 'all known tags'? If it works for you like for me in paragraph 2, then try another domain. Does it still?

>>17183 (OP) >weekly releases How do you (or others) keep the flame of motivation alive?

I just realized I had closed my page with many watchers 3 days ago. - restoring a db backup would lose my changes - exporting all files downloaded or modified is very complex and would not necessarily save everything - "last session" backups happened every 5 minutes, so there were not enough of them - apparently sessions cannot be exported - I couldn't find where they are stored in the databases. Not sure I would have been able to use it anyway. - the nearest "exit session" was 6 days old. I chose this option.

>>17229 Thanks for the reply! >How would you, in an ideal world, like to handle the overall problem of duplicate note names and/or texts? Don't know about an ideal world but the way I expected it to work was to simply add a note with the specified content even if it's duplicate content. Since it's a custom parser I personally don't have to worry about spammy notes. I just want to have notes created with the content that they get from the data if it exists. It's not the end of the world since the data is technically preserved but it's a bit annoying that I don't know which of the fields that I'm scraping was actually populated. Maybe adding an option to note content parser settings to have that specific one not get deduplicated?

Oh, and another question! Are the databases between hydrus platforms interchangeable? Can I go from using the windows client, move the data to the docker image and then if need be return with the updated data back to the windows client without data loss?

>>17238 I switched from running the windows binary in Wine, to running the Linux binary, then running on Linux from source, all with the same database, so it should be fine

>>17231 Not any time soon, sorry! Maybe in the future when the file storage system is more flexible and tolerant to error and can deal with multiple potential locations for files (i.e. handles less certain locations for files). Also, I don't have a Linux machine to test with atm, and Windows linking isn't nearly so capable, so I'm sure I would fuck it up. For now, I have to mandate KISS here. >>17232 Thank you, well spotted and great report. Yeah, the basic problem is that the 'thumbnail grid' object isn't kept up to date with the current tag domain, so when I tell it to sort by tags in some way, we either have to specify a specific domain, like I added for namespaces, or it will default to 'all known tags'. I simply need to do some cleanup so it understands the current tag context, like the 'selection tags' list does, and make this work more naturally in normal situations. I can add the tag service widget to the 'number of tags' sort, but I'd rather not do that first since it is a mild pain in the ass and it is probably pretty obscure to specifically want to search tags in one context but sort in another--most of the time I imagine people want to sort in the same domain as the search. >But when i match the domain of the said buttons to everything else except 'downloader tags' (-> my tags, PTR, all known tags), the sorting doesn't work anymore My bet here, btw, is simply that your 'downloader tags' service tends to have more tags than anything else so it lines up with 'all known tags' more often. In reference to your broader tests, if I set the file sort to 'num tags, descending', but then change the tag domain, as in vid related, the sort stays the same. I'm pretty confident it is just sorting by 'all known tags' the whole time. I've reminded myself to do this cleanup/overhaul.

>>17234 I went through a handful of crash-out drama bombs in my youth, and now I have a better handle on what causes burnout for me. Mostly it is working in teams; I simply cannot do it without sperging out and working too hard trying to keep everyone happy and then resenting the whole thing and quitting in a big to-do. So, I now work strictly alone. There's a handful of guys I trust to do pull requests on simple extensions or whatever, but I never 'collaborate' on code in any meaningful way. As for keeping on doing it every week, I am someone who always did his homework the night before. If I let myself schedule something for two months from now, that means six weeks of doing nothing. By having a tight loop and a hard regular deadline, I'm able to keep myself productive and I can also fix my mistakes quickly. I'm not professional and I throw out garbage code regularly. I don't know how to turn myself into a conscientious person, so instead I just sling a ton of mud at the wall and fix it when I screw up. For the actual motivation, I don't know. I enjoy helping others (I am beholden to this, in fact), and the anonymous imageboard community is my home, so that part all seems to line up. I'm very stubborn, and I avoided many opportunities to quit simply by keeping my head down and pushing on. I can't stomach becoming a normalfriend, and I am also supremely lucky to be in a personal situation where I am able to do this, so I'm free to dump an indefensible amount of time into this weird project, hahaha. If you are 22 and you find it difficult to stick to a difficult project, don't worry. All the emotional stuff just gets easier as you get older. >>17235 Well done. I have done this myself, same deal just by accident, and then noticing days later. I should add a yes/no confirmation any time you close a downloader with stuff in it. >>17237 Thanks. I think you are right--more options in 'note import options' is going to be better than half-hidden logic. >>17238 >>17239 Just to confirm, yeah, 100% portable. The db directory is the same no matter your platform. Only thing that can fudge an 'installation migration' is if there are hard-saved absolute paths for your import/export folders, or custom file storage locations, that point to ~/blah or G:\blah, which the other OS may have trouble with. But it won't break anything permanently--hydrus will generally just moan about it and let you fix it to the corrected path.

>>17241 >For the actual motivation, I don't know. I do know.

is there a way to let things files slip through the tag blacklist on a case-by-case method? Kinda like how you can click 'try again' in the file log of the url downloader page and redownload previously deleted files? Basically there's an image I want that has a tag I've blacklisted but I don't want to mess with the list, just want the one particular image.

>>17241 Perhaps I should try and set myself some deadlines. A couple of years ago I was pretty regularly making progress on my projects, but I had to abandon them when the job search was getting tough. Now I'm hoping to pick back up a project I had probably put about 4-6 months into before. Part of my problem is that I'm usually feeling quite tired during the week, another part is that it's been kind of hard to come home from 8 hours of work on a computer and start more computer work. But I've been thinking about the project more lately. Been hoping to be struck with "motivation" or "inspiration" to get back in the habit of working towards my goal. The big problem is getting started again I suppose. Anyway, thanks for the response. Always good to hear from those who work on cool projects.

>>17240 >I imagine people want to sort in the same domain as the search Yeah, basically i THINK it would be better when i change the domain in the main tag domain button (let's call it like that) and then want to sort for number of tags, the thumbnails sorting should change according to the tag domain chosen in the main tag domain button, but right now it changes according to the upper 'tags' button that appears when changing the sort to namespaces. Without that button, the num tags sorting is broken (since the thumbnails dont change like in your video), but as you said, i would also rather not have that button because it makes things more complicated imho. I guess that's exactly what you said :D >>17240 >In reference to your broader tests, if I set the file sort to 'num tags, descending', but then change the tag domain, as in vid related, the sort stays the same. I'm pretty confident it is just sorting by 'all known tags' the whole time. Not sure if i get what you are trying to say since it is complicated and hard to wrap my head around all the sorting systems of hydrus (there are basically 3 tag domain buttons when sorting by namespaces!), maybe you are correct, but i dont think it is always sorted by 'all known tags', since changing the domain in the upper 'tags' button that appears when sorting by namespaces, AND THEN going back to num tags sorting, the thumbnails changed, but from there the main tag domain button doesn't change the thumbnails anymore like in your video. So when i change the upper namespace 'tags' button to 'my tags', then going back to num tags sort and change the main tag domain button also to 'my tags', the tag list shows the correct 'from 0 tags to most tags' behaviour. That is also the case for every other domain but the upper 'tags' button and the main tag domain button have to match. Thats why i think it is not always 'all known tags' but most probably i just dont get it :D Sorry for the wall of text, you probably figured it out already anyway, just want help as much as i can so you might find a fix faster.

>>17245 Thanks--you are completely correct, and I didn't read+test properly. I did this cleanup work yesterday evening and it looks like I wasn't doing 'all known tags', but some internal value that happened to default to that but was overwritten in the case of the namespace stuff. I've got the thumb grid updating to the current tag domain now and the 'number of tags' file sort obeys that. Let me know if it gives you any more trouble! >>17243 I'm pulling back from 'exception' workflows since I'm on a big KISS kick, so I don't think I'll go in this direction. I've got many longer term plans to make selecting your 'import options' much easier, with favourites/templates you can load up real quick, which I think is the better answer here. If you could select a tag import options that didn't have the blacklist and then try again in like three clicks, that's nice and natural and doesn't shake up the workflow with complications. Unfortunately that will take much longer to get to so it doesn't fix your problem. Sorry! If it is just the one file, I think make a new urls downloader and set some specific tag import options, clearing the blacklist. Now I think of it, I can and should have an easy way to reset a blacklist you are looking at, so I'll see about that. >>17244 Good luck! I read sometimes about people who coded up some clever new game engine 'in their spare time' after coming home from work, and I don't know how they do it. If I had to do a nine-to-five every day, I simply wouldn't have the energy for hydrus. I am very very thankful that I don't have to. Maybe a deadline will help your situation, but if you force it I could see it leading to its own sort of burnout. Maybe you can reserve Sunday afternoons or something, and then at least something is being done, but you aren't slaughtered. I'm crazy about todo lists, and if I have a project summary like 'add video to the hydrus duplicates system', it is never getting done. I always have to break it down into eight sub-jobs and then those into scratchpads with twenty thoughts. I feel like the back of your brain knows the rough size of problems, and if it is too much you just discard it for something more immediately satisfying. But 'convert this vague project idea into smaller chunks' is itself doable in half an hour, and then you got something done and next time you work you have a small list that is again easy to get into and have some measurable progress on. In the inverse to what I said before, I feel like when I was twenty, I could code until 3am every night hahaha. Just endless energy and enthusiasm, but no emotional control. Although, all my code back then was total shit, so I guess it is not an honest memory.

I have a bit of a problem, and a feature request that should solve it. When you have a subscription or a gallery downloader that has post pages, and those post pages can contain files, there an issue where, if a file exists in multiple posts, the downloader will only check the file for the first post it sees, and basically ignore it for every post afterwards in the downloader. I get why this is done, but I big problem with it is that if that post had metadata (like tags and urls) that needed to be added to the file, the downloader won't do it because it didn't check the file again, due to it already being in the file log. If it's not too much trouble, could you add an option for downloaders to add the file as a child of a post, even if that file already appeared in the downloader under another post, so that it will grab the metadata from every post that the file is in, regardless of which order the downloader sees them in? It'd make managing my files a lot less confusing, because currently there's a lot of holes in the title tags and post urls of my files from certain artists, due to this issue. I hope this makes sense, and thanks!

>>17248 It sort of depends--can you tell me a little more about these duplicate/subsequent URLs? Is your situation: A) The same file is appearing multiple times at the same URL? B) The same file is appearing multiple times in different URLs? If A, like you have a sub that is failing to re-queue the same URL twice, or a downloader you are running twice and it refuses to queue up the same thing it saw once again, then check out the file right-click menu->urls->force metadata refetch->blah urls. This is a common problem, wanting to catch up to new metadata that has been added to URLs that were first downloaded from a while ago. However, that doesn't sound like what you are describing. So, if B, can you tell me broadly about the 'shape' of the downloader you have here? Are we walking through a gallery search that is producing a list of Post URLs, or is this some complicated nested thing? Is this something like Kemono, where you are parsing all sorts of files from a richly formatted post, so it can turn up over and over? It isn't Pixiv is it--I mean, Pixiv can have multiple files per post, but you don't get the same file in different places, do you? If the same file appears in multiple posts, hydrus would normally grab the metadata from the different URLs, even though it recognises the file has been seen before. It typically only recognises that it knows a file once it has hit the (new) Post URL, by which point it has parsed tags and stuff from the rest of the document, and it'll add it anyway even though we are 'already in db'. The only circumstance I can think where it won't is if you are somehow parsing hashes or maybe source URLs at a preliminary stage, probably the gallery parsing stage with a subsidiary page parser, which is causing hydrus to skip hitting the Post URLs to get additional metadata. Maybe you can tell me a bit about what you see in the 'notes' column for these 'already in db' results in the 'file log' for the downloader. Does it say 'url', 'hash', or 'file' recognised? OR am I misunderstanding, and this is a problem within the parsing step itself? Are you parsing separate posts within a single document, and somehow I'm deduplicating URLs somewhere? Or do you have Post URLs inheriting metadata from their parents, hence why you want child creation?

I had a good week. I fixed several annoying bugs and added some quality of life. The release should be as normal tomorrow.

>>17257 >So, if B, can you tell me broadly about the 'shape' of the downloader you have here? >Is this something like Kemono Yes! It's kemono.su that I'm having this issue with for downloaders and subscriptions. So it looks like this. There's a gallery that has posts, and the posts contain multiple files. If the same file appears in multiple posts, it will have the same basic url (from from what I can see, they sometimes or maybe always have a different "f" querry parameter, but I think Hydrus strips that anyway, and I'm not sure if it's always there). >If the same file appears in multiple posts, hydrus would normally grab the metadata from the different URLs, even though it recognises the file has been seen before. It doesn't seem to do this if it already has that exactly file in the current log. it'll only grab the urls that aren't already in the log. I just check with a post right now, and it said "Found 1 new URLs" but there's actually 2. it's just that the second url is already in the downloader's file log, so it seemingly just skips adding it as a child of that post, and thus skips adding the tags and urls and such from that post to the file. >It typically only recognises that it knows a file once it has hit the (new) Post URL, by which point it has parsed tags and stuff from the rest of the document, and it'll add it anyway even though we are 'already in db'. If I'm understanding you properly, the issue is that it's not adding the file url again and saying "already in db". It's instead just not adding the file url again AT ALL, because (I assume this is why) it's already if the file log, so it just silently skips it. That's the issue that's causing the metadata from only one post the file's in to actually be added, I think. >Maybe you can tell me a bit about what you see in the 'notes' column for these 'already in db' results in the 'file log' for the downloader. Does it say 'url', 'hash', or 'file' recognised? There is no notes column for the file url because it's not getting added the second time at all. It only gets added from the first post that the downloader sees that file url, then afterwords it just silently doesn't add it again as a child of any future post urls for that downloader. >OR am I misunderstanding, and this is a problem within the parsing step itself? Are you parsing separate posts within a single document, and somehow I'm deduplicating URLs somewhere? Yes, I think this is exactly the problem! Hydrus is silently deduplicating file urls in the log, which is causing it to not properly get the metadata of the file at that url for each post (and post url) that contains that file. Instead, it's only getting the metadata for the first post url that it sees that has that file, and then afterwards, it just pretends that any future posts that have that file url... don't have it. Like it's not there, so nothing gets added, and the metadata for those files are incomplete. >Or do you have Post URLs inheriting metadata from their parents, hence why you want child creation? I have file-urls inheriting metadata from their post-url parents. But I need the file url to be added to the file log by the downloader for each post-url that it appears in, so that hydrus can see and add the metadata from each post that the file appears in, to that file. Even if that exact url has already appear in the log, it's going to have different metadata attached to it, depending on which post url it appears under. I know I was kinda repetitive here but I just wanna make sure you understand what's happening properly. Thanks a lot for helping! This is a really annoying issue for me and I'd love to see it fixed!

https://www.youtube.com/watch?v=QYWR-jF7Mek windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v611/Hydrus.Network.611.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v611/Hydrus.Network.611.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v611/Hydrus.Network.611.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v611/Hydrus.Network.611.-.Linux.-.Executable.tar.zst I had a good week. There's a bunch of fixes and quality of life improvements. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html fixes I think I fixed the long-time issue where a search page that was scrolled some way down would often reset to the start position while you were on another page! Another long-time bug, a perplexing one, where the media viewer could on rare files suddenly not update on content changes--for instance, you'd click the rating, but nothing appeared to happen--is fixed. The update was being fired off correctly (you could watch the original thumbnail get the rating or archive or whatever), but the media viewer was just briefly confused on how to update itself. It basically happened when a file maintenance job ran on a file in a media viewer list after the media viewer had booted. Just an unlucky thing. Should be all fixed now, but let me know if you run into any more trouble. I fixed a couple of Drag and Drops--from thumbnails and page tabs--where if you released your click at precisely the wrong time, the DnD would stay attached to your mouse despite not having any mouse buttons down. The program now recognises this and cancels the drag. The 'apply image ICC Profile colour adjustments' option under the media viewer's 'eye' menu is more reliable. It wasn't clearing the caches properly, before, and so the old image was sometimes lingering unless you zoomed etc.. It seems to immediately update now, every time. new stuff If you have a search with a system:limit, changing the file sort will now refresh the query. When you have system:limit, most sorts will be able to fetch the nth 'tallest'/'smallest'/'reddest' files, so changing the sort will now resample from the larger pool of files at the source. You can turn this off under options->file search. Also, the file sort widget now says in a tooltip if it is able to do this database-level system:limit clipping (the complex ones can't). The thumbnail grid now sorts files by 'number of tags' according to the current tag domain--set it to 'my tags', it'll be different from 'all known tags', etc... If you try to close any non-empty downloader page, the program now throws up a yes/no confirmation dialog. You can turn this off under options->gui pages, where you can now also, if you prefer, set to confirm all page closes. In an experiment, you can now map the modifier keys on their own. They'll show as 'ctrl+nothing' or 'alt+shift+nothing'. They seem to work, but I can't promise it'll be fine everywhere. For parser makers, I added a STATIC formula, which simply spits out the same text every time, regardless of the actual parsing text. Might be useful for fixed, parser-based additional tags and so on. next week Back to duplicates auto-resolution.

>>17264 >fixed the long-time issue where a search page that was scrolled some way down would often reset to the start position while you were on another page! Funny, I was actually using that to quickly see if my (very full) subscriptions downloaded new files; https://xkcd.com/1172/ finally happened to me. Thanks for the work as always!

Hey hydev I've been having some issues with my Linux machine and I think I found what is probably a bug in Hydrus. When my issues crop up Xorg dies, and hydrus doesn't recognize the Xsession has gone away and keeps running, I have to sigkill hydrus if I want to reopen it. I haven't been reopening hydrus much recently since this issue is pervasive and I'm paranoid about my db getting messed up so I don't have much in the way of tests. So I'm sure if I'm just impatient or hydrus can't properly shutdown, but a nice shutdown with sigterm doesn't seem to work. Hopefully it is me being impatient but hydrus itself doesn't shut down on it's own, that I know for sure. No clue how it works but if possible could you please add a "nicely shut down if the X server dies" feature?

Do you recommend autoupdating hydrus with something like scoop? Or just keep using one version as long as it works?

>>17264 >I think I fixed the long-time issue where a search page that was scrolled some way down would often reset to the start position while you were on another page! Thank you so much!!!

>>17264 >The thumbnail grid now sorts files by 'number of tags' according to the current tag domain--set it to 'my tags', it'll be different from 'all known tags', etc.. >>17246 > Let me know if it gives you any more trouble! Awesome! Seems to work without issues! Big step in the right direction. That being said, i played around a bit and found, that collections maybe could also have a rethink and some similar change. Not sure if it's possible or if people need the current behavior, so tell me if my suggestions make sense: Change sorting to 'sort by collections: number of files in collections' for easier understanding of my problem, then 'collect by creator' for example. Also search for 'creator:*anything*' (because i don't know if the change could apply to 'system:everything/inbox/archive' searches). Now if you change the main tag domain button to something else than 'all known tags', the number of the collections in the thumbnail grid seems weird. For example, let's say i have 203 files with only the 'creator:artistA' tag in 'my tags'. I would expect the grid to show 1 collection with 203 files, but it shows me 4 collections: a 200-file-collection + a 1-file-collection + a 1-file-collection + a 1-file collection, even though the creator tag is the same for all of them (only creator:artistA). That happens because the three lone 1-file-collections have other creator tags in other tag domains like PTR, therefore also in 'all known tags' which the grid seems to rely on. Looking into 'all known tags', each of the 1-file-collections has artistA in combination with another creator tag, that the others don't have. Thats why each combination stands for themselves, means if a file has two or more different creator tags, it creates one collection. So what i would like: in 'my tags' the thumbnail grid would show only one collection with 203 files, because they have all only the 'creator:artistA' tag. Same collection behavior for all other domains that are not 'all my tags'. In 'all known tags' the grid behaves correctly at the moment imo and shows a collection for each creator combination. Also ones that arent in 'my tags' at all. I think for 'system:everything/inbox/archive' and some other system predicate searches that wouldn't be possible, since those show the same amount of files all the time, no matter what tag domain you chose. Here the number of collections and the number of files within the collections are always the same, but probably that's by design (because they don't rely on the tag list when sorting? Just an assumption by a layperson). With 'system:number of tags > 1' the grid behaves like with a 'creator:*anything*' tag search though again.

>>17272 >means if a file has two or more different creator tags, it creates one collection Correction: can of course be one different creator tag too. Important is, that each combination of creator tag/s creates one collection, which is easiest seen in 'all known tags'.

>>17264 >I fixed a couple of Drag and Drops--from thumbnails and page tabs--where if you released your click at precisely the wrong time, the DnD would stay attached to your mouse despite not having any mouse buttons down. The program now recognizes this and cancels the drag. I never encountered this issue, but I'm now getting false cancels when I try to drag and drop stuff, resulting in me having to do it several times sometimes. Can I toggle this back?

IT KEEPS HAPPENING

>>17257 >>17262 >within a single document To clarify, I think I might've misunderstood you here. If by "single-document" you mean "within a single parsing step" then no, it's not multiple posts' file urls being added simultaneously. These are separate post urls that have some file urls that are identical to each other. The post urls are added by the gallery search, then the file urls are added by entering each post url.

I don't know what I keep pressing that does this, but how do I un-highlight the query box? Whenever this happens, I can't enter a query with the enter key anymore and have to clicking search results from the list instead. I've been fixing this by duplicating the current page and removing the old one.

Is this a joke?

Something off in the logic here. Same number of frames, same fps.

Just updated from 606 to 611. One of my gallery downloads has been giving me really off statistics. This was a new download and none of these are "already in my database" nor "deleted" and only displaying 392 images I just downloaded(I did delete a few so its probably a little over 400). I even did a search on the artist and only 392 images show up. Even stranger is I've did a couple of gallery downloads before this one after updating and they all showed up fine with no issues or saying they are already in my database/deleted.

>>17262 >>17277 Thanks, I think I got you. If an import object--which is a single row in the file log--parses multiple URLs to pursue, it creates child objects, so you'll get: POST URL 1 POST URL 2 POST URL 3 converting to: POST URL 1 (successful, created three children) - FILE URL 1 - FILE URL 2 - FILE URL 3 POST URL 2 POST URL 3 And the problem is that when we hit a later Post URL, it wants to create a child for FILE URL 2 or something, and the file log says 'already hit that, bro', and skips adding it as a valid new child. I'm sorry to say, now that I think about this, that I can't see a clear and nice solution. My file logs are very strict about not duplicating data (which is mostly a backstop to avoid loops, but the fundamental data structure of a file log relies on each row having a unique key (URL), so I can't really write a hack in here to allow the dupe). I also don't have the ability to go 'we already did that here, but let's merge in our new metadata and re-run the write content updates job'. The add event is simply skipped, just as you say. I could think about a long-term solution where I say 'don't process children until the parent layer is done, and then figure out content update merge instead of skipping, but there are good reasons to process children immediately for sites with timeout tokens. So, this is a dumb and not excellent answer, but I think you'll have to hack it on your end for now. Mainly you want to trick the file log into not realising they are the same URL. Change the parser so it puts a meaningless extra bit of data on the File URLs you parse. This could be a parameter, let's say based on parent post id, that the server won't error out at receiving: If you are getting something like: https://n1.kemono.su/data/f7/ab/f7ab95bec4a7eaac9b5e5c6e80c5b9fd38443a7ac75b0e593e7c3c9ab0290152.jpg?f=nRGFiR1UNGX7St2H7uSbJQ33.jpeg and convert that to https://n1.kemono.su/data/f7/ab/f7ab95bec4a7eaac9b5e5c6e80c5b9fd38443a7ac75b0e593e7c3c9ab0290152.jpg?f=nRGFiR1UNGX7St2H7uSbJQ33.jpeg&whatever=9465655 Then hydrus will think it is a different URL. EDIT--But wait, now I see a real URL here, I see the 'f' you mentioned, and that it is already being stripped, and that could be our key here. No need to add a spurious new param if we can figure out why the always-unique 'f' is disappearing. So, go to ''manage url classes' and see if that file url matches anything. Maybe it is matching a kemono Post URL by mistake. If so, make a new URL Class just for the File URL (or perhaps edit an existing one) and add the 'f' parameter so it knows not to remove it when normalising. Unmatched URLs do not have 'excess' parameters removed on normalising, so I'm fairly confident your File URLs here are matching something. Once you can get the File URLs normalising with a bit of tokenised unique data, the file log should allow them. Let me know how you get on!

>>17268 Thank you for this report, and I am sorry for the trouble. Can you try the two sigterm tests under help->debug->tests, do not touch? One of those is real, and one just calls the thing a sigterm is supposed to call. Do either work for you under normal conditions? Trying to wrestle this thing into receiving system and session shutdown calls is hellish sometimes, I regret it isn't cleaner. I try to just say 'hey when the system tells me it is shutting down, I obey', but it seems things are not always so simple, and I think my legacy code is still firing in the wrong thread-order, despite many iterations of cleaning the shutdown code. I'm currently planning on implementing full 'this program is stopping shutdown' logoff stuff working on the Windows side of things, since I know I'm not setting the right flags for it and I need to do more research, so I wonder if I can get that working correctly, it'll help on yours too. If you boot hydrus from a terminal and hit ctrl+c on the terminal window, does that work? It should fire a quick shutdown that skips some stuff, including no shutdown splash window. I think that works as a certain sort of SIGINT, but I'm not super confident on the Linux side of things. >>17269 I know some guys have scripts that wangle auto-update with chocolatey or even winget. You can do whatever you like, with the sole proviso that any script you make must integrate a backup sync beforehand. As long as you make a backup, then it doesn't matter what fucked up thing could happen in the update step, including where I post like 'hey, every two years or so you have to do a clean install, and we are here again, so here are the special install instructions', you haven't lost or damaged anything. Tbh if you want clean auto-update, I'd say run from source and then just do git-pull once a week, again after backup. It takes like three seconds to run, and then you might want to rebuild your venv once per six months or so, which I make easy with the setup_venv script. https://hydrusnetwork.github.io/hydrus/running_from_source.html >>17272 Thanks. I agree. Now I've got the thumbnail grid aware of the current tag context, I need to deploy it to pretty much all tag computation on that side of things. I also want the thumbnail banners to filter to the current context. I hadn't thought of collection slicing, but that's another one to do. I'll see if I can do some this week, let me know how it goes. >>17274 Thanks, can you explain more what goes wrong? The current process on my side is: If the user has been moving the mouse with click down from the drag source for more than 100ms, start the drag and drop. If, 100ms after that drag starts, the mouse is now up but the drag is ongoing, cancel the drag. This second part is what I added last week, because there was a marginal moment you could release the click right at the 100ms moment and things would get confused and the drag would start even though the button was already up. So, are you just doing a normal drag of a tab/file and ~100-200ms after the start it suddenly cancels? Despite you having the mouse button down? Do you have anything funny going on with your mouse, like you have primary/secondary clicks swapped or anything? Maybe I am testing 'Left down' the wrong way in your case.

>>17275 Shame all around. I don't know enough about the Arizona political situation to guess what will happen next. It looks like the governor vetoed the same basic bill a year ago, so if the legislature are trying again, are they just posturing or did they change the language to satisfy her? My engineering brain says that you make kids safe by applying strict whitelists to kid electronics. A blacklist whack-a-mole on adult electronics is never going to work and only going to fuck with privacy. But when these bills pass and the proponents cheer how the sites now have reduced traffic across the board, you realise what it was really about. I know, I know, gasp, shock, and horror that politicians might be duplicitous. If it passes, I guess traffic will collapse, and on our side of things the hydrus downloader will break. Maybe Bad Dragon can spin e621 out to its own entity and move it to a more friendly state, but I always figured the site was run at a loss and it is really advertising for the core product, so it probably isn't feasible. All the more evidence that offline storage and management is useful. You simply cannot trust the cloud to be there in five years. That said, if e621 survived because of parent subsidy, and it had an outsized market share because of that, then its absence will presumably allow for a more healthy ecosystem of multiple normal boorus operating in different places and with their own tag taxonomies and so on. Might be a smaller space overall for appreciators of obese farting anthropomorphic lion drawings, but a less brittle one. >>17282 Probably 'insert' key. It enables some IME stuff for inputting kana and kanji etc... You can remove this under shortcuts and the 'tag autocomplete' set. >>17285 Yeah, that's odd. Normally the janitors have been removing all the 'clothing' stuff. I know 'thigh-highs' in particular has had a bad loop in it, so maybe they are fixing something. I don't see anything in the janitor chat, so give it a day or so and see if they did something already, otherwise propose a sibling change putting 'clothing:thigh-highs' to, say, 'thigh-highs', and I should think it'll be accepted. I'll make a job to check this in a few days. Might be something got fucked somewhere because of the history of this complex tag and I need to do something. >>17287 Thanks. I bet the duration difference is tiny, just a few ms. I'll have a look at the thresholds. >>17288 Thank you for this report. If this is very private, you don't have to post the log itself, but can you tell me more about the 'shape' of the information here in the file log? Do you have anything like this: URL | successful | (found 2 urls) Sub-URL | successful | (already in db) Sub-URL | successful | (already in db) URL | successful | (found 1 urls) Sub-URL | successful | (already in db) URL | successful | (found 2 urls) Anything like that, where some of your extra rows might be some 'container' URL that is spawning children, so you have multiple rows per file? Or is it a normal booru-style log where it is just a flat list of URLs that were executed on, one file per object? Or, do any of the URLs have weird token parameters, like 'http-url?tokAuth=eou4o954u9o5e4u95oe46uo'? Sometimes you can get the same URL twice because of some weird parse, but hydrus thinks it is different because of some unique value like this. Sometimes you get an 'already in db' result for something that was in the queue earlier, which sounds like it might fit your situation here. If those lines say 'already in db - was seen 6 seconds earlier', that's typical for this. And, what in your 'file import options' of that page are the 'presentation options'? Does it say like 'all files | must be in inbox | all my files'? Sometimes the presentation options cut down what you actually see from the master list by excluding previously archived stuff. If you click the little down arrow on the file log button and hit 'show all files in a new page', is it still 392?

>>17290 >Thanks, can you explain more what goes wrong? Testing it some more, it only happens when attempting to drag things outside of the Hydrus window. I will still see a little plus sign attached to my cursor for a split second even after the mouse is outside the Hydrus window, then the it vanishes. I'm not releasing the click, and my mouse works fine otherwise except for some occasional issue with automatically right clicking that I have yet to pin down, probably because it's a cheap mouse, but that's rare and this is consistent and repeatable. It happens about half or more of the time right now.

>>17285 This is why I manually tag damn near everything, don't use the PTR, and don't use siblings. This mess seriously fucks with my autism.

>>17292 Bro what the fuck is going on, I just tested what you describe and it is happening super reliably if I drag and drop out the main window before the second 100 ms timer hits. Somehow the 'is left down' state is not testing correct when the cursor is out of the client area. I will check this, sorry for the trouble! Slow-initial drags seem to fix this for now.

>>17294 >Bro what the fuck is going on It amuses me to see you use this sort of language given how you normally post.

>>17295 A raw look at what UI code does to me. An endless series of 'why the fuck is it doing that?'

>>17291 >Thank you for this report. If this is very private, you don't have to post the log itself, but can you tell me more about the 'shape' of the information here in the file log? Do you have anything like this: Never mind, I think I found out why. Its another Pixiv nonsense thing again. When it came to those image set collection Pixiv does, this specific artist would add samples + commission prices inside his collection. Say an image set with 20-25 images in the collection. More than 2/3 would be samples + commission prices. This artist basically reused those samples + commission prices in every collection. I deleted the first batch of samples + commission prices images thinking it was a one time thing and hydrus kept picking those up, adding to the file count. Different URLs, same hash. Some of which were downright identical. tl;dr Nothing wrong with Hydrus and just me being dumb and not paying attention.

>>17290 Say if an update messes something up it should only be the .db files that need to be restored/backed up right? The same library files (the pictures) should still work I assume?

>>17300 yeah the db files are where the update happens. It should be safe to run a backup of just those files right before an update, then simply restore those if it goes wrong

not really important but I just started using hydrus companion again after trying it once loooong ago and disusing it since it slowed my browser to a crawl for whatever reason but using now not only speeds up my workflow, I just noticed the URL database checking thing and that's super fucking helpful damn I should not have slept on retrying companion it's funny, cause I did the same thing with hydrus it self back in like '17 or '18, tried it a bit and hated it only to come back like 2 or 3 years later and see how amazing the program is

I've been testing more tools for a while now, but may I ask if there are plans in the future to run scripts automatically on import? Right now, whenever my subscription downloads archive files, I use hyextract to manually extract and reimport the data then deleting the original files, which is a real pain the more time I do it. This would be useful as well with AI tagging such as WD. Any thoughts on these?

>>17290 >Can you try the two sigterm tests Sure: >self sigterm Quickly closed gui, took about 20 seconds to clean up >self sigterm fake It also quickly closed the gui, also took about 20 seconds to clean up. >terminal C-c Prompted if I was sure if I wanted to shut down, I let my thread downloads finish before retrying. It also prompted for maintenance work, which I skipped. Took about 5 seconds to do some final cleanup once the "saving gui and quitting" job finished.

I had an ok week. I mostly did behind the scenes work, so the release just has some bug fixes. The release should be as normal tomorrow.

Has "parse advanced predicate string" ever worked correctly? (A && B) || (C && D) a OR c a OR d b OR c b OR d A xor B a OR b -a OR -b

>>17311 I have a duplicate filter task that maybe I should not be doing, because of the volume, but here it is. I have thousands of video frames, and I want to find the best ones. Each frame has a "frame:number" tag. I want to compare pairs of adjacent frames and use "this is better and delete the other", "same quality", or "related alternates" on them. That is, I want the duplicate filter to compare only 1 and 2, 2 and 3, 3 and 4, 4 and 5, 5 and 6 etc.

>>17314 those are both correct what's issue?

>>17316 Huh, guess they are. a OR c a OR d b OR c b OR d (a && a && b && b) = (a && b) (a && a && b && d) = (a && b && d) (a && a && c && b) = (a && c && b) (a && d && b && b) = (a && b && d) (a && d && c && b) = (a && b && c && d) (a && d && b && d) = (a && b && d) etc It's so unreadable though.

>>17314 >>17316 >>17317 Yeah, unfortunately hydrus can only handle Conjunctive Normal Form (AND of ORs) atm, no ORs of ANDs, so that parser takes arbitrary logic and outputs CNF for the normal search system to process. I'd like to figure out ORs of ANDs sometime, but the main barrier is just that it'll need some loop-de-doop new edit UI.

https://www.youtube.com/watch?v=LgCRbxXY-fs windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v612/Hydrus.Network.612.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v612/Hydrus.Network.612.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v612/Hydrus.Network.612.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v612/Hydrus.Network.612.-.Linux.-.Executable.tar.zst I had an ok week. I mostly worked on behind the scenes stuff, so I only have some bug fixes to release! Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights All Jpeg-XL files will be scanned for a bunch of metadata like 'has transparency' and 'has exif'. If you have any weirdly rotated JXLs, they should fix themselves soon after update. I fixed a couple bugs with the recent drag-cancel tech, which stops some weird drag states occuring. The test was firing falsely when dragging thumbnails very quickly outside the window, and it was throwing some errors when kicking in on macOS. Export folders handle sidecars a little better. Missing sidecars are filled in on every run, and if you have a 'synchronise' export folder, it is more careful about not deleting existing sidecars. next week I moved the duplicates auto-resolution system forward this week, and it is now pretty much all in place. The database module and daemon are live--but as-yet unpopulated. I now just have to overcome one more difficult hump, a nice preview panel so we can see the impact of a pending auto-resolution rule, and then we'll be ready for final integration and testing and letting advanced users try it out. I'm feeling good about it all and will continue.

Do the "Unless" look to you like they were intended for "Hide namespaces"? The items are uncheckable only if the first checkbox is unchecked.

Is there any way to search specifically for a tag, and only that tag without any siblings or parents? I mean like "solo_female" without "solo female". Or vice versa, only searching for "solo female" without any of its siblings. Or do I have to kill the sibling association each time?

>>17299 Great, no worries. >>17300 >>17301 Yeah, just the db files are changed in an update. If you use FreeFileSync, you can have it compare file-size/file-modified-date to calculate a backup. This runs very fast for an incremental backup job since it recognises all your media hasn't changed and will only overwrite your db files and a few thumbs maybe and add/delete any new/deleted media files. A neat trick, then, is you can simply tell it to reverse the operation and it'll do a complete rollback that is equally as fast and comprehensive. I do a backup before I update every week, so if it goes wrong, I can just rollback and it'll do the db files for me. >>17302 I'm glad you are enjoying things now. I've been doing a lot of optimisation over the past few years, and a bunch of API stuff is working a bit faster, I think, just because my own UI code isn't so clogged and it has free reign to fetch URL statuses and stuff freely. Let me know if things ever start getting laggy again. >>17307 Yeah. Please do a search in this thread and in >>>/hydrus/ for 'exe manager' or 'executable manager', and you'll see me talking about my general plans here. I want it, and it is becoming more important with AI autotaggers and upscalers and stuff, but it'll be a big project. >>17308 Damn, so that side of things seems to be fine, although 20 seconds feels too long. I'll do some work and see what I can do to better plug into the session signals, since it seems like hydrus isn't catching the SIGTERM here, or pre-SIGTERM 'hey session is closing are you good?' stuff, exactly as we want. This stuff isn't always easy to test, but let me know if the overall situation changes in future please. >>17315 Interesting problem! I don't think you'll be able to make a duplicate search that does 'namespace:n', 'namespace:n+1'. If you are not super strict about the n,n+1 requirement--i.e. what you really want to do is examine similar frames--then I'd just let the system work as normal, since it will recognise that frame 5 and 6 are very similar anyway. It might say that 5 and 7 are similar too, or 5 and 57, so if that is a dealbreaker then not. If you are very strict about only comparing neighbouring frames, I think, I don't know, you'd probably have to try and figure it out through the client api. Something like: https://hydrusnetwork.github.io/hydrus/developer_api.html#manage_file_relationships_remove_potentials - do this on all the frames, dissolving what hydrus has already recognised https://hydrusnetwork.github.io/hydrus/developer_api.html#manage_file_relationships_set_file_relationships - do this programatically on each n, n+1 pair https://hydrusnetwork.github.io/hydrus/developer_api.html#get_files_search_files - this to help you find the n, n+1 pairs' file_ids Then, I think the duplicate filter would present you the pairs as you want. BUT as you said that some were better quality, let's say you said 2 > 3, then 2 would represent 3's spot against 4, so you'd be presented with 2,4 anyway. I think my overall suggestion is to try this on a smaller sample of frames the natural way. Just run them through the filter, let's say without deleting anything, and see if the answer is anything like what you wanted with the n,n+1 idea. Just letting the duplicate filter find and compare pairs is probably going to be KISS here. >>17320 Thanks, bad grammar, I'll fix it. >>17324 Not yet, but I'd like to introduce it in the not too distant future. The PTR janitors want the same thing, and a bunch of debug situations want the splitting of fine differences here. It'll come in the form of a system:predicate that searches tags specially according to tag display context (with or without siblings) and tag service (i.e. other than the current tag domain if you want).

Hello. Using Hydrus but new to anything involving programming. I'm downloading high quality tumblr images via the "export your blog" function and at the same time saving the standard tumblr image with the url as the file name, adding file names as tags on import to hydrus then using the duplicates search to just have the urls linked to the higher res images. It sounds complicated but it is a million times better than having to filter through tumblr exports which are downloaded out of order and manually find the url for each of them while keeping track of what you already have. The problem is that you can't save an actual url as a file name because of the forbidden characters. However I have a Danish keyboard with acess to the keys æ ø å which never show up in urls (I browse english language tumblrs). So I just replace forbidden characters with them (I created a word macro for this) and for multiimage posts with the same url I just add å at the end of the file name to have unique filenames. And then just adding "source" as the filename namespace. For instance, if I wanted to save the images from this post on nasas tumblr https://nasa.tumblr.com/post/775384136434302976/love-letters-from-space I would end up with my hydrus images being tagged with: source:øønasa.tumblr.comøpostø775384136434302976ølove-letters-from-space source:øønasa.tumblr.comøpostø775384136434302976ølove-letters-from-spaceå source:øønasa.tumblr.comøpostø775384136434302976ølove-letters-from-spaceåå And so on... (and for tumblr art you can easially add a regex to tag the artist from the url) But then this still has to be converted. Is there a way to automatically change certain characters in tags to other ones? Like tag siblings or regex but only changing certain characters. Replace ø with /, æ with : and remove å.

>>17327 This was simpliefied a bit for readbility, the actual tag is uncluding https as part of the file name.

>>17327 I think that instead of trying to solve this weird painful problem, you need a better workflow for getting images into hydrus. Sidecars would be very helpful. Tell me more about the "export your blog" function. I thought you could only use that for your own blog?

>>17327 Nevermind, I think I see what you're doing with "export your blog" - you're reblogging a post you want and then exporting your blog, right? Unfortunately, that doesn't come with any metadata files that you could import with the image. Luckily, that's not the only method of getting the highest quality images. I tried making a parser, but tumblr does some annoying things that make it hard. Instead of reinventing the wheel I think the better way to do this would be to just download the images with gallery-dl and import them with sidecars. 1. Download gallery-dl: https://github.com/mikf/gallery-dl 2. Run it with the option --write-metadata to get information about each image in a nice json file: gallery-dl --directory "C:\gallery-dl\tumblr" --write-metadata https://nasa.tumblr.com/post/775384136434302976/love-letters-from-space This will put the files in a folder under C:\gallery-dl\tumblr. 3. Create an import folder in hydrus for that folder: https://hydrusnetwork.github.io/hydrus/getting_started_importing.html#automation 4. Create sidecar routers: https://hydrusnetwork.github.io/hydrus/advanced_sidecars.html#json_files The name of the blog is under "blog_name", the url is under "post_url", and the post timestamp is under "timestamp". That's all you really need.

Id like to ask for a new feature. Could we have some arrows to be able to skip to the next/last entry (thumbnail) in the 'manage tags' dialog? Right now we need to close the dialog, then do the right-click procedure again if we want to compare tags/number of tags (if you are a mouse-only user) or use F3 to open/close the dialog. The idea is, a click on the 'right' arrow would go to the next single entry, even if you had multiple selected before. Or maybe also without arrows, just to click on another thumbnail so the dialog gets updated to the one thumbnail selected? That way you could also select several thumbnails again and the dialog would update accordingly. But right now it does that windows clinking sound if you try to click the thumbnails behind the dialog, so it doesn't work. I needed to check for some files the number of tags in different tag domains, but it's a bit tedious to close and open the dialog again and again, even with F3, so that feature would be great. I don't know if anything would speak against it, from a usability perspective. But i also use two monitors, so idk...

>>17334 It's only one file at a time, but you can do that in the full size file viewer. Just know that tags entered and deleted there are applied immediately, rather than when you close the window via F3 or the apply button. The apply button actually doesn't exist in the tag editing dialogue window for the full size file viewer.

I had a great week. I made much more progress than I expected on the duplicates auto-resolution work, fixed some bugs, and figured out a new system predicate to search tags in a bunch of advanced ways (e.g. on a different tag domain, without siblings, in deleted/petitioned domain). The release should be as normal tomorrow.

>>17335 Oh thank you for the reminder! I actually used that sometime and forgot about it. Will help alot in many situations! :) For the others, i hope Hydev can look into my suggestions, especially for live updating of the dialog for multiple thumbnails. So you can click/select them and also maybe scroll up/down in the thumbnail viewer with the mouse wheel. The question would be, should the manage tags dialog stay in the foreground all the time or should there be the option to 'minimize' like other dialogs can, like the 'migrate tags...' dialog just as an example. Default it stays in the foreground but you can minimize it if you want. I think that would be the best.

>>17336 >The release should be as normal tomorrow. Forgot to say, thanks boss!

>>17336 >on a different tag domain, without siblings, in deleted/petitioned domain based I'm still not sure about auto-resolution because it could munch a bunch of my imported AI slop images that have/don't have AI metadata, since there's still no predicate about specifically that (and metadata doesn't always mean that it's not just photoshop exif garbage)

I'd love it if, in the duplicate merging options, there was an option to "copy from worse if none are on better" option for URLs. So it only copies URLs if there aren't any already, otherwise I get some images that have zero URLs despite having an online source, or many which are just an unholy mess of various URLs of dubious quality, few of which actually serve the same picture.

>>17337 Oh and to add another idea: I also would like hydrus to be able to sort for 'last tagged time'. I don't think hydrus records times when you tag a file, correct? If you tag a bunch of files after a complicated search, then delete the search and you would like to check for exactly the last tagged files, that would be a good way to find them fast. 'Last viewed time' doesn't work for that if you don't look at them in the media viewer. But im not sure if that would work retroactively (i don't think hydrus records tag times behind the scene?) and also if there should be a general tag time that considers all tag domains or per tag domain. Because when the PTR updates, the time would update also for all the updated files. But since the thumbnail grid is aware of the current tag context now, per tag domain would probably be better, and you could still change to 'all known tags' to make it consider all tag times. Maybe it's too much bloat, but just see it as another suggestion that could come handy sometimes :p

>>17319 I can't seem to drag to reorder tabs after this update. Is there a setting I'm missing?

https://www.youtube.com/watch?v=iyl3otdvXH4 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v613/Hydrus.Network.613.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v613/Hydrus.Network.613.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v613/Hydrus.Network.613.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v613/Hydrus.Network.613.-.Linux.-.Executable.tar.zst I had a great week. There's a few fixes and a new clever system predicate. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html system:tag (advanced) We've wanted a tool to do things like 'search for files with just this tag, no siblings' for a while. After recent conversations with users, I had some good ideas and decided this week to try and just knock it out. It went well, and advanced users are invited to try it. Let me know where it fails! So, in the normal system predicate list on any search page, you'll now see 'system:tag (advanced)'. You type a tag in manually, no autocomplete, and can set: - Which tag service to search in, if different to the current search. - Whether we should include siblings or just search for that tag exactly. - What 'status' the tag has, be that 'current', 'pending', 'deleted', or 'petitioned'. - Whether the file should have this tag mapping or not. This adds several new tools, all pretty clever. Some queries may run very slow, but I think it all works, and I've integrated fast cancel tech. I didn't have time to add it to the system predicate parser, but I planned for it and will try to get it done next week. I may add a proper autocomplete input to the panel, too, for quicker tag entry. next week I also had success with the duplicates auto-resolution preview panel I had been dreading. I was expecting to have to do a bunch of rewrites, but I banged my head against it for a bit and a solution popped out. The new dialog now asynchronously fetches a rule's search and presents the pass/fail thumbnail pairs in a couple of tables. I'd like to do a bit more--adding a regular media viewer to 'zoom in' on a pair, and only sampling results when the rule has a lot of pairs--but the difficult part is solved. With this, all of the complicated work in duplicates auto-resolution is finally done. I now just have to do final integration and testing. I am going to keep pushing, so with luck, I expect to launch the first version of this system for advanced users to try in v615. >>17342 Did your options->gui pages->BUGFIX: Disable all page drag and drop get switched on by accident?

Anyone else got their hydrus deleted by microsoft?

>>17345 Happened to me in one specific version 10-20 versions ago too. But only because my regular anti virus did have some problems and turned itself off, so microsoft defender was activated. My regular one doesn't put hydrus into quarantine. But i also didn't test the two latest versions of hydrus yet.

>>17345 >Threat blocked >Detected: Trojan... This is a years old issue. You see, Hydrus has an embedded server in its code and Windows confuses it with a trojan. Of course it is a false positive. To my knowledge, the procedure to avoid triggering the anti-virus is that devanon has to pay from his pocket to Microsoft an audit of the code to get it into a White-List. Naturally that's outrageous and never will happen. So, two choices for ya, either you White-List Hydrus in the anti-virus settings, or, you switch to a different operating system. >microsoft Ahem. Respectfully... you've got what you deserve.

Gelbooru admin is vagueposting about shutting down.

>>17343 Ah it turned out to be some strange windows issue. For some reason i could reorder chrome tabs but not firefox tabs

>>17343 >system:tag (advanced) I'd like most of all to do something like system:has tag in "all known tags", ignoring siblings, with status pending: "*:*_*" So I can more easily filter out all the trash tags before uploading to the PTR. Even better would be an "apply all siblings relations to pending tags" that would convert everything within the search to its sibling equivalent.

Jesus fucking christ why is sankaku such a fucking piece of shit hydownloader is failing to download from it again giving "os error" What does that even mean?

Has anyone been able to solve tbib's mass-downloading of dummy files? I can clear the file log from my subscription panel but it would keep downloading (=ignoring) the same amount of files.

The age of hoarding is about to be over Cloudflare stops all my hydrus attempts now

>sankaku now have daily login rewards Can someone explain to me why is that fucking shit site still being used

>>17334 >>17337 Thanks. This is actually tricky to navigate, which is why it is so awkward now. The main thing is with a 'modal' dialog (one which locks out the rest of the program), we have the idea of apply and cancel. If a user makes changes in the dialog but then backs out, we roll them back to what we had when we started. If, broadly speaking, I allow the user to continue using the rest of the program while the dialog is open, then I'd have to deal with hypotheticals like them opening up two different dialogs on the same file, or using a shortcut to apply some tags to a file after the initial dialog was open--in that case, my dialog is no longer neatly 'atomic', and we are dealing with a more complicated situation to rollback to or commit from. We already live in that world a little bit, since downloaders can apply tags to files in the background while a dialog is open, but it generally hasn't proven a big deal so far. Anyway, that's why the manage tags panel is an immediately-committing non-modal 'frame' when launched from the media viewer, but a modal apply/cancel thing when launched from (potentially a whole selection of) thumbnails. It saves me a ton of time and potential headaches, particularly with manage tags, which is pretty complicated and CPU heavy behind the scenes. There isn't a superb solution other than me knuckling down and writing better and cleverer code. I can't promise this will get better too quickly, but I do first want to make the manage notes, urls, ratings, and anything similar work like manage tags in the media viewer. I want you to be able to have smooth editing+navigation with one or more windows open off the media viewer, and as I develop that unified solution for multiple dialogs, I will keep all this in mind. The manage tags dialog pulls off the duo modal and non-modal capability with a bunch of hardcoded hacks, so first I will pull all that tech out and smooth it out and generalise it for any dialog so I can apply it to the notes and urls and stuff. Once the code is cleaner, I can think about make it complicated in a smarter way >>17341 >I don't think hydrus records times when you tag a file, correct? Yeah. I think this is a good idea. We have pretty decent timestamp management these days, and while I don't want to track every possible change, I'm all for a 'last time the human edited tags on this file on any service' timestamp and just hooking it into the same search and sort routines as something like 'last viewed time'. Yeah we can't fill this data in retroactively. I don't know when things were tagged in the client atm. >>17339 I will write up some proper help as this system launches, but I'll say here: this system will be highly user-configurable and always off by default. I aim to have a bunch of decent templated rules that users can try for common situations, but you'll always be the one pulling the trigger, and you will see the rules and see a preview of what is about to happen. I absolutely do not want users to half-accidentally fire something off and discover a bunch of decisions were made that they didn't want. >>17345 Sorry for the trouble. It felt like this was happening less these days, so it is a shame to see it back. It is a false positive. All the source is on the repo, and the build is created on the github cloud itself using this script: https://github.com/hydrusnetwork/hydrus/blob/master/.github/workflows/release_win.yml If this keeps hitting you, I recommend running from source: https://hydrusnetwork.github.io/hydrus/running_from_source.html

>>17349 Shame, if so. Feels like we are in an inflection point in a bunch of ways right now. If more and more of the imageboard community is squeezed out by the big guys and/or pressure of AI rippers, I don't really know what the future will be, whether that is a new community in a new place or a radically different set of protocols, or what, but I'm happy that hydrus, with its 'local only, no cloudshit' autism provides something of a backstop. >>17353 No worries. Windows has some funny rules about drag and drops between processes, usually in regards to programs that are not in admin mode DnDing to programs that are. Same deal with the clipboard and global shortcut hooks. If you are still trying to figure this out, on the Firefox side at least, if the main FF process is running as admin but the individual page processes or UI process(?) aren't, I wonder if that could be blocking the tabs through some convoluted rule. Or some Window Manager/UI-styler with the same thing. >>17354 >wildcards aiiiiiiieeeeeeeee I may be able to do this in future, but I can't promise anything now. I was thinking of it as I was developing the new search routine behind this predicate, and it'll be doable, but a ton of code cleaning may have to happen first. I was lucky in that the efficient tag mappings search for all sorts of domains was already done, so I just had to adapt it for the new predicate and add a bit of slow stuff for deleted and petitioned, but the wildcard tag definition search is not so pretty, and so implementing this in a way that doesn't take forty-eight minutes to search the PTR may need some prep work. I'll note it down, please remind me if three months pass by and nothing happens. Hard sibling replace is something I am thinking about seriously. It will start as a janitor-only ability on the PTR, but I'll extend it to normal users for local tag domains and we'll see how it goes and maybe extend it to normal users on repositories too.

>>17362 >Hard sibling replace is something I am thinking about seriously. considering the PTR is currently north of 87GB large – far closer to 100GB+ – the PTR really could use that ability, if not the ability to replace the mappings of one hash with another via multiple reference. Consensus of duplicate replacements could be used for that, uploaded (probably opt-in) by users. It shouldn't need any revealing information, just "I'm replacing the mappings of hash #1 with hash #2" if both are in the same public database's domain, no worse than uploading tags. Enough of those, and you can replace delete hash #1's tags and have it reference hash #2's tags. Given how many duplicates there are, it could prune a lot of the database.

Think I found a bug when exporting files and using json sidecars with the export folder in v613. I've set up an export process that uses routers for tags and urls and adds it to a json sidecar, but it seems that only one of the metadata routers is added to my json sidecar. The regular "export files" dialog seems to work just fine though, all the routers are added to the sidecar as expected.

>>17347 >>17348 >>17361 I force-reinstalled it with winget and it's working fine for now.

>sankaku I'm getting error 403 on search pages when using the one from the repo and pasting daily auth tokens like I always did.

>>17358 Because it allows big filesizes for long videos while allowing stuff that's outright banned or too low quality for most video-specific hostsings, and it has user defined tags for easy navigation.

>>17368 What a pain Hydrus stopped working for it like a year ago and now even that shitty gallerydl wrap isn't working

>>17369 the v2 api downloader worked fine until this week, just required lots of manual babysitting because auth keys and ddl urls expires all the time.

>>17370 >the v2 api downloader worked fine until this week, just required lots of manual babysitting because auth keys and ddl urls expires all the time. Well yeah, might as well not work, pointless to automate hundreds of artists if you need to babysit the downloader. >

>>17349 Gelbooru has been down for 7+ hours.

>>17372 they said it's a routing issue

Hey hydev, just got a sudden Hydrus while using sway (Wayland), looks like it's due to an MPV error expecting X11. v612, 2025-03-16 21:34:53: A media loaded in MPV appears to have had an error. This may be not a big deal, or it may be a crash. The specific errors should be written after this message. They are not positively known as crashy, but if you are getting crashes, please send the file and these errors to hydev so he can test his end. Its hash is: 530d68237026581637b578bab5f0f1b851c2033548ebc107dd60537fda48f43c v612, 2025-03-16 21:34:53: [MPV error] vo/gpu/x11: X11 error: BadDrawable (invalid Pixmap or Window parameter) I will be able to test more about it later, no time right now.

Thanks for recently adding the "is not filetype" function. Helps immensely since some filetypes are disabled right now due to an attack on the site.

>>17374 >sudden hydrus Uhh, sudden hydrus *crash*, I think it happened as soon as I clicked the file, I tabbed away for a second.

>>17376 again Just clicked on an animated gif while looking for a meme and hydrus froze. It's almost certainly a sway/wayland issue, I've never had hydrus dying on every video before. Its hash is: 3959d075181c1b1963127191349d2eb86756d012c96e1d93543d6608d1272dc3 v612, 2025-03-18 12:54:08: [MPV error] vo/gpu/x11: X11 error: BadDrawable (invalid Pixmap or Window parameter) v612, 2025-03-18 12:54:08: [MPV error] vo/gpu/x11: Type: 0, display: 0x7f344c0040d0, resourceid: 69, serial: 16 v612, 2025-03-18 12:54:08: [MPV error] vo/gpu/x11: Error code: 9, request code: e, minor code: 0

>>17377 More tests: Trying with dbus-run-session: X Error of failed request: BadWindow (invalid Window parameter) Major opcode of failed request: 4 (X_DestroyWindow) Resource id in failed request: 0x6200002 Serial number of failed request: 52 Current serial number in output stream: 55 v612, 2025-03-18 13:02:09: QObject::killTimer: Timers cannot be stopped from another thread v612, 2025-03-18 13:02:09: QObject::~QObject: Timers cannot be stopped from another thread I also tried Xwayland but it didn't start at all.

>>17356 >tbib The problem is with TBIB's search. Their search has been borked for ages, and they apparently don't care about fixing it.

>>17374 >>17376 >>17377 >>17378 Are you running from source?

>>17293 Do you use downloader siblings to "filter" known downloader tags? For example: "series:genshin impact" -> "game:genshin impact" I don't use the PTR either, but personally I get a lot of mileage out of downloader-only siblings

>>17380 Yes

>>17382 at the bottom of the hydrus_client.sh file, try replacing python hydrus_client.py "$@" with: env --unset=WAYLAND_DISPLAY --unset=QT_QPA_PLATFORM QT_QPA_PLATFORM=xcb python hydrus_client.py "$@" that might help if the issue is related to interactions with Qt. just a shot in the dark though

>>17381 >Do you use downloader siblings to "filter" known downloader tags? For example: >"series:genshin impact" -> "game:genshin impact" Nope, with one exception for yokai watch/yo-kai watch or whatever because I keep misspelling it myself.

>>17383 No dice but it at least segfaults instead of freezing. The coredump says the error comes from PySide6...libQt6Gui.so.6 Here's the stack trace, I don't know shit about debugging but the ??? entries are a little odd, I was suspicious of them at first but I see that libc has it too so I guess it's fine. Is that what you see if you don't have debugging symbols? #0 0x00007f59f87d1110 in ??? () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Gui.so.6 #1 0x00007f59f89ba6ad in QPainterState::QPainterState() () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Gui.so.6 #2 0x00007f59f8998a2d in QRasterPaintEngine::createState(QPainterState*) const () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Gui.so.6 #3 0x00007f59f89bfb52 in QPainter::begin(QPaintDevice*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Gui.so.6 #4 0x00007f59f6b9ee62 in ??? () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #5 0x00007f59f6b9d3b1 in QMainWindow::event(QEvent*) () at /venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #6 0x00007f59f6a0f632 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #7 0x00007f59fbc6ad3a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #8 0x00007f59f6a5ee26 in QWidgetPrivate::sendPaintEvent(QRegion const&) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #9 0x00007f59f6a5f62e in QWidgetPrivate::drawWidget(QPaintDevice*, QRegion const&, QPoint const&, QFlags<QWidgetPrivate::DrawWidgetFlag>, QPainter*, QWidgetRepaintManager*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #10 0x00007f59f6a72245 in QWidgetRepaintManager::paintAndFlush() () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #11 0x00007f59f6a6761c in QWidget::event(QEvent*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #12 0x00007f59f6a0f632 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Widgets.so.6 #13 0x00007f59fbc6ad3a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #14 0x00007f59fbc6e24d in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #15 0x00007f59fbf24223 in ??? () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #16 0x00007f59f9521992 in ??? () at /usr/lib64/libglib-2.0.so.0 #17 0x00007f59f9524dc7 in ??? () at /usr/lib64/libglib-2.0.so.0 #18 0x00007f59f9525570 in g_main_context_iteration () at /usr/lib64/libglib-2.0.so.0 #19 0x00007f59fbf23afa in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #20 0x00007f59fbc76573 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #21 0x00007f59fbc7300e in QCoreApplication::exec() () at hydrus/venv/lib/python3.12/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #22 0x00007f59f74ebcb5 in ??? () at hydrus/venv/lib/python3.12/site-packages/PySide6/QtWidgets.abi3.so #23 0x00007f59fffcf284 in ??? () at /usr/lib64/libpython3.12.so.1.0 #24 0x00007f59fff81d13 in PyObject_Vectorcall () at /usr/lib64/libpython3.12.so.1.0 #25 0x00007f5a0007aaa0 in _PyEval_EvalFrameDefault () at /usr/lib64/libpython3.12.so.1.0 #26 0x00007f5a0008287c in PyEval_EvalCode () at /usr/lib64/libpython3.12.so.1.0 #27 0x00007f5a000d7576 in ??? () at /usr/lib64/libpython3.12.so.1.0 #28 0x00007f5a000d7675 in ??? () at /usr/lib64/libpython3.12.so.1.0 #29 0x00007f5a000d7780 in ??? () at /usr/lib64/libpython3.12.so.1.0 #30 0x00007f5a000da6cb in _PyRun_SimpleFileObject () at /usr/lib64/libpython3.12.so.1.0 #31 0x00007f5a000dadc0 in _PyRun_AnyFileObject () at /usr/lib64/libpython3.12.so.1.0 #32 0x00007f5a00100ecc in Py_RunMain () at /usr/lib64/libpython3.12.so.1.0 #33 0x00007f5a00101418 in Py_BytesMain () at /usr/lib64/libpython3.12.so.1.0 #34 0x00007f59ffcdb22e in ??? () at /lib64/libc.so.6 #35 0x00007f59ffcdb2e9 in __libc_start_main () at /lib64/libc.so.6 #36 0x000055f117e62095 in _start ()

>>17385 >Is that what you see if you don't have debugging symbols? I actually don't think I've seen this error before myself, so I have no idea here. I have one more idea though that might help. Keep that "env" part that you added in the startup script, and in your mpv.conf (the one used by Hydrus) add this config option: gpu-context=x11egl Remember to actually go into Hydrus to update the config under the "media playback" settings, since I don't think Hydrus updates it automatically. (could be wrong though)

>>17386 Still nothing, don't feel too bad about not solving my issue anon, like 50% of the programs I use are having issues in wayland. I'm only using it because Xorg is having major problems, and I now know that it is an X issue (great). X Error of failed request: BadWindow (invalid Window parameter) Major opcode of failed request: 4 (X_DestroyWindow) Resource id in failed request: 0x2400002 Serial number of failed request: 52 Current serial number in output stream: 55 v612, 2025-03-18 19:01:52: QObject::killTimer: Timers cannot be stopped from another thread v612, 2025-03-18 19:01:52: QObject::~QObject: Timers cannot be stopped from another thread

>>17388 wow that seems serious. hopefully Hydev will know more about what those errors might mean then, at least on the Qt side

>>17374 >>17377 >>17386 I am no Linux or mpv expert, but the thing that stands out to me here is the: >v612, 2025-03-16 21:34:53: [MPV error] vo/gpu/x11: X11 error: BadDrawable (invalid Pixmap or Window parameter) line. I've seen trouble before with 'vo', which is documented here: https://mpv.io/manual/master/#options-vo so I think something like >>17386 's 'gpu-context' or 'vo' might be a good tack here. Maybe this? https://mpv.io/manual/master/#video-output-drivers-dmabuf-wayland https://mpv.io/manual/master/#video-output-drivers-wlshm Or take the other end and try 'sdl', which I think is the software-only rendering--if that doesn't crash, then you know 'vo' is the guy to hit. You can edit the mpv.conf in your db dir directly. It reloads in all mpv windows every time you ok file->options. >>17388 >>17390 This specifically is a bit odd and I don't totally understand it. In general, the easiest way to get hydrus (or most UI programs) to crash is to touch UI stuff from the wrong thread. Since UI things are often working with synchronisation-delicate stuff like the GPU, you generally have one ecosystem of threads that are set up to do UI stuff at the correct time and correct place, and non-UI threads have to post events that are then processed in the main UI event loop where it is safe. This error looks like a classic crash in that some non-Qt thread is telling a QTimer to do something. Probably, although I can't be certain, it is calling the C++ level destroy function on the QTimer, so my best guess here is that some external thing is saying something like 'hey, this object/thread/process/something is invalid, kill it now', and that's grabbing a QTimer in there and nuking it, and then it is sperging out because we aren't in the Qt thread, and that seems to be caught by the X Error catcher. It saying X_DestroyWindow is probably part of the whole thing. So, where's the QTimer here? I only use one, really, in hydrus, to do some animation-level timing stuff like thumbnail fades. That's owned by the main gui window and lingers until program exit, so this error could be a complaint on that timer during a forced whole-program shutdown somehow from an external thread, but I suspect instead this is a QTimer that's operating inside Qt itself (down again at C++, rather than my python code) to do one of a million tasks like animating a checkbox click transition or whatever. It may be that some of the mpv stuff works via this, and that's likely since that's what's causing the problem, but I would guess, right now, that this error is more of a symptom of error-reporting that is failing to handle and clean up a problem rather than the cause of the problem per se. The trace in >>17385 , which is all C++ level, seeeems to back this up. It looks like a PaintEvent, which is the most dangerous and crashy time. Probably the main mpv widget (on the Qt side) trying to draw itself to screen, and then it, idk, let's say it is asking mpv about something as it tries to draw stuff in the media viewer (not the hardware accelerated mpv window itself though--I'm pretty confident mpv draws itself to its own window id on its own thread using a different mainloop, which is another tangled nest that can cause crashes I'm sure), and because this version of mpv isn't setting the hardware drawing flag in the way wayland/X/whatever wants, it is raising an exception. Any unhandled exception inside a paint event is basically crash time. I still think the right tack for now is try stuff at the mpv.conf level. That way you are telling C++ stuff how to initialise and talk to other C++ stuff. Get mpv calling different OS-level graphics APIs and we'll hopefully find one that won't go bananas with this specific setup. Let me know what you figure out!

I had a great week. Duplicates auto-resolution is very almost done, and for the release there's nice new zoom-locking tools, some quality of life, and the new 'system:tag (advanced)' gets a bit more polish. The release should be as normal tomorrow.

I'm revisiting the hydrus install I used to scrape a bunch of cringe-inducing shit and exploded artist blogs from 5+ years ago (version 417) and I'm noticing something very disturbing: hundreds of files in client_files/f?? with no extension and no apparent file format other than some repetitive byte strings. It's not JPEG or PNG, 'file' says they're all "zlib compressed data" but that could mean fuck all. My mappings.db is also a hot mess but that's not a big surprise considering how much I was fucking around with the PTR at the time and I'm pretty sure I force closed it in the middle of a PTR update a couple times. I just want to find out if this is expected and if not, how fucked I am. I'd upload a screenshot showing what I'm talking about but the acid fatty won't let me.

>>17393 I'm pretty sure you are looking at PTR repository update files, means the files that get downloaded before being applied to the PTR. If you change your file domain to 'repository updates' and sarch for 'system:everything', you will see them (20,401 files, 14.4GB for me). If you can't see the 'repository updates' domain, you need to activate advanced mode first under 'help' at the top. Then you right click one of the files -> open -> in file browser, and it will take you to one of exactly those files. Deleting them after your PTR updates are done, should do no harm. They don't delete themselves i guess and act like a backup.

>>17343 >- Whether we should include siblings or just search for that tag exactly. How about finding tags that are really there and not added as parents?

>>17367 >>17355 api url changed, parsers, downloaders etc need to be updated.

>>17394 Neato. One more question: did the tumblr NSFW purge a few years ago totally fuck the affected blogs or is there still a secret handshake to get at original images? Right now when I look at a particular NSFW blog on tumblex random images are just 400px thumbnails.

https://www.youtube.com/watch?v=a_2VS4NFAXU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v614/Hydrus.Network.614.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v614/Hydrus.Network.614.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v614/Hydrus.Network.614.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v614/Hydrus.Network.614.-.Linux.-.Executable.tar.zst I had a great week. There's some new zoom-locking tools and quality of life. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights Thanks to a user, we have some neat new zoom tech for the media viewer. There's a new zoom button in the top hover window that lets you set different zoom types like 'fit horizontally' and 'canvas fill', and we also finally have zoom locking, so if you want to browse some media all at 100% zoom or whatever, you can just set that and set it to lock and it'll stay that way as you move back and forth. There's also 'pan locking', to help comparison between similar files, and I added the 'size locking' zoom tech from the duplicate filter, too. As a side thing, the 'eye' icon on the top hover also lets you disable the hover-window pop-in for the tags and top-right hover windows. The new 'system:tag (advanced)' predicate now has a tag autocomplete to make it easier to enter the tag you want, and the predicate now works in the system predicate parser. There's two new 'catmocchin' QSS styles submitted by a user. Check the contents of the files themselves for more ideas on style and other colours. I boosted the default 'memory reserved for image cache' from 384MB to 1GB for new users. If you have never touched the stuff under options->speed and memory, you might like to go in there and make sure your basic image cache amount is decent, especially if you get laggy image load on >4k images. 1GB is very good for most situations. There's an annoying bug in the media viewer where images will sometimes start to take two frames to zoom--one where they move to the new position, then the second, flickery frame where they resize. I discovered how to reproduce it and think I have it fixed! next week I did final integration on the duplicates auto-resolution system this week, and everything worked! I pretty much just have to write help documentation and unit tests for it, so I will focus exclusively on this and aim to launch the simple initial test for advanced users next week. If you are interested in this system, I enabled all the edit UI this week so feel free to go bananas and see what happens. All users see it now, too--it is a third tab on any 'duplicates' page. The dialog will not save any rules back, so it won't do anything, but you can see what I'm thinking of. Let me know if anything breaks!

>>17401 You're a fucking wizard, man.

Suggestion for duplicate processing. When I have large files (10+MB and 2000+ width or height) and I need to compare them in the duplicate processor, or just when moving forward or back in the media viewer, swapping between the files will have a delay, and during this delay the display is blank. This can make it difficult to spot minor differences. Can the first file linger until the last possible moment as the new one is loading?

Not really a hydrus question, but does anyone know if there a fine arts booru? Like a booru for classical paintings and whatnot?

I'm trying to set up hydownloader but can't get it to work over vpn or proxy. If I enable proxy in settings, in test run logs I get >Missing dependencies for SOCKS support. I tried installing requests[socks] as mentioned in gallery-dl docs but it did not help. As for VPN - I'm using a client with split tunel support, do I have to point it at the entire python.exe ? Also importing known IDs from Hydrus did not work, but that may be because of sankaku url change. Or am I doing it wrong? The importer ran successfully and displayed matching number of entries.

>>17401 I started looking at the UI for the dupe auto-resolution and I have a suggestion. I think you should change the wording for the comparison rules so that, instead of "A will pass:" or "B will pass" you should say "A will match:" or "B will match:". saying "pass" confused me and made me think that the one that gets that rule will be the one that's counted as the better dupe. "match" makes it more clear that it's a check that the file matches the pattern.

Just updated from v611 to v614. I accidentally zoomed in a video 2 times quickly, and it instantly froze Hydrus for about a minute with my cpu running a bunch, then Hydrus crashed. it only gave this one error in the log: python3.12: ../src/x11/x11-window.c:2344: eplX11SwapBuffers: Assertion `pwin->current_back->status == BUFFER_STATUS_IDLE' failed. I tried it again, but this time I zoomed in slower and 3 times, and it crashed again. It didn't do this before.

>>17427 I can't reproduce it with images, so I'm guessing MPV is involved with the error

>>17401 >Thanks to a user, we have some neat new zoom tech for the media viewer. I've been missing that! Have to resize the window with "Let's review our rule to make sure it makes sense" every time for the third column's head to fit, and the "edit rules" window once.

>>17289 (sorry I've been busy) >But wait, now I see a real URL here, I see the 'f' you mentioned, and that it is already being stripped unfortunately, I looked into it, and it's not being stripped. it only shows up when you view the file directly from kemono.su normally, but the downloader uses the api, which doesn't have the f parameter as part of the file url. that being said, I just used a regex substitution to recreate the parameter and attach at as the url to download. I hope you're right and that it works, but the downside here is that there's gonna be a whole bunch of duplicate known urls now, that only have an unnecessary parameter different between them. it's gonna make kemono.su urls messy unfortunately. but if it works, it works. I do hope that Hydrus eventually gets some sort of "metadata merge" for the downloader's file urls so that this hack isn't necessary anymore, but as long as this allows the tags to properly be added to all files (I'll hopefully see if it does when my subscription has updates) then it's good enough for now. Thanks for the help!

>>17401 It looks like the setting for making rules are just ordinary searches. is it stilled planned to allow you to compare the 2 files directly in rules, like "A has >=10% larger filesize than B" or something like that, or was that postponed? if you already mentioned it then I must've missed it. I hope it comes eventually because a lot of the rules I wanted to make would involve comparisons, but this still looks like it'll be pretty useful as is. pretty cool!

Super anecdotal, but just switched to Wayland (niri); I'm starting Hydrus with Xwayland plus the suggested `QT_QPA_PLATFORM=xcb`. I've switched to the native hydrus viewer for videos as mpv really sucks in my case (it creates two separate windows that don't go away and stay black after playing, not sticking to the main client or the media viewer). I'm happy with the native hydrus viewer and that setup seems to work pretty well for me, so thanks for providing that alternative hydev!

>>17363 We'll see how it all goes as these tools slowly roll out, and faster duplicate resolution. I added an advanced 'purge tags' tool for janitors last year, and that went well, although I'm sorry to say the absolute savings of these things feels like it is in the 1-5% range at best, so while it is nice for the human experience to clean up garbage, it isn't earth-shattering on a storage level. Unfortunately, also, for space-saving purposes, the PTR system doesn't yet have a way to completely divest itself of a tag mapping--when we 'delete' something, I just move its record to the master 'deleted' table. There are good reasons for this--to make sure that the bad tag is not re-added by the same misfiring parser or whatever later--but we'd like an option for a complete space-saving purge and/or orphaned-id cleanup. I'm thinking of these systems, and I expect in the coming years the PTR will get massive overhauls. The growth is delightful but unsustainable. >>17365 Thank you, sorry for the trouble. I will check this out this week or next. >>17366 Great. I am not an expert, but I believe most of the false positives here come from 'Smart Screen' or 'Automatic sample submission' or whatever MS call it, which is their cloud submission system that delivers real-time threat definitions. Afaik, when you run an (unsigned?) exe, MS sends a hash or profile or something of the exe to their cloud, and that process can result in the confident false positive report. When users (and maybe user behaviour?) report that it is a false positive, the cloud learns that that hash report was a false positive, so like ten days after release, Windows Defender no longer gives the report. The data they learn through all that, I assume, becomes the regular definition updates they actually push to Windows Update. Thus, although I don't think it is fantastic to recommend, turning off Smart Screen cloudshit under the Defender options--I just looked and it looks like the call it 'cloud-delivered protection' and 'automatic sample submission' now(?)--tends to reduce these false positives since your Windows isn't asking the realtime-yet-error-prone cloud system about the new hydrus exe. But I guess if you turn that stuff off, you aren't getting it when it does fire off correctly. >>17393 Sorry for the confusion. I'll write an FAQ for this--it has come up before. If you use zlib to decompress them, you'll see it is just giant JSON lists of numbers and tag/file definitions. >>17398 Pretty sure it isn't widely available in any way. I think for a while they let accounts see their own nsfw posts, so people could backup and stuff, but I don't know if that is still true. If you see some cached thumbs, maybe it is, or that could just be some external image host remembering them. They changed their policy again after another more recent change in ownership, I think, as well. I think you can have artistic nudity, gravure stuff, again, but nothing hardcore. I think I got my hydrus tumblr in ~2006/7 (EDIT: looks like June 2008), because they had the hottest post editor at the time. Would have never guessed all the changes since. You may well know, but it used to be where all the early sfm creators posted crazy stuff; you'd never think it today. I'm amazed it is still running, as I'm not sure they've ever had a profitable quarter. I'm expecting it to just shut down one day. >>17395 Thank you for mentioning this. I changed this for v614. I forgot that the storage/display tag domain dichotomy has the parent-gap also.

>>17403 Thanks; had a good few weeks recently. We can die any day, so keep on pushing. >>17423 Yeah I'd like some better render tech here. I don't like how ugly the media viewer is in the intermittent moments, and it'd be nice if it could wait on a media transition until the next one was ready. Perhaps I'll be able to do that when I finally get around to overhauling the awful layout system that backs it atm. For now, your best bet is to boost your 'image cache' sizes under options->speed and memory. I increased the default values here for new users in v614, so you might like to go in there and double-check you have a decent amount yourself. 1GB cache size and letting a single image eat 25% of that allows for a ~16,000x9,000px image to stay in the cache. If a file is in the cache, it should load instantly when you flick between two images. Make sure you don't have an accidentally anemic 'image tile cache' either. 256MB tile cache is great. >>17425 I know very little about hydownloader, but I have to add 'pysocks' to the hydrus requirements.txts to (iirc) get requests SOCKS stuff working. I dunno if 'requests[socks]' does that too, but you might like to try it. >>17426 Thanks, great idea. >>17427 >>17428 Thank you. I changed the media viewer to do 'setposition and setsize' on the actual media window in one call this week, to stop a flicker issue. Sounds like X11 doesn't like that for mpv. Sorry for the trouble--I will make a DEBUG option to go back to the old behaviour for mpv for next week. Let me know if that fixes it, and if it does I'll make it default for Linux or something. >>17430 >Have to resize the window with "Let's review our rule to make sure it makes sense" every time for the third column's head to fit, and the "edit rules" window once. Thank you, will fix!

>>17431 >I do hope that Hydrus eventually gets some sort of "metadata merge" for the downloader's file urls... Yeah absolutely. We'll need a lot more tools around this as duplicates auto-resolution takes off, too. I want a bunch of en masse URL management tools too to do 'hey convert/merge all these legacy http urls with their modern https equivalents' and so on. It'll just need an overhaul of URL storage&search, and some UI. >>17435 Yep, exactly. The first test for v615 is just going to be pixel-perfect jpegs and pngs, and I focused on getting the platform and workflow working solid. We'll make sure the system works at scale on that trivial and unobjectional rule, and once happy I can then add more levers and handles to the system. The 'A has x property' test is simpler than an AB, so I did it first, but I absolutely will figure out an object to handle 'A is at least five times larger than B' and so on, and I'd really like to do it like the 'A has x property' system where it works with existing system predicate tech in some way, so I'm not writing all new UI and hardcoded comparison blah for every new testable variable. This is inside baseball talk, but I'm thinking of adding a call to system preds where they eat a media and give up a number related to their type. So for filesize, they'd eat up a file and produce the num_bytes. Then, with minimum effort, I can have some generalised comparison UI and backing object like: A SYSTEM PRED | OPERATOR | MULTIPLE | B SYSTEM PRED Or maybe just the one system pred, I dunno if you'd ever want to compare A's height with B's width or something, but maybe that's something to think about later. Anyway, you'd say "A system:height | >= | 5x | B system:height", and then all I have to do in future weeks is add the simple 'hey yeah I can produce a number' method to other system pred types, and we magically get new tools with the same UI. Simple start, and iterable growth of new tools thereafter. We might figure out some more tightly coupled file-to-file comparison tech like 'A is 99.7% pixel-similar to B', but I'd like to see how the IRL works and we can get a real feel for what we are actually missing and what is worth dumping time into. If we can eliminate 80%+ of all the relatively easy same-file duplicates automatically within twelve months, I'll be delighted. >>17436 Sorry for the trouble here. I have heard this new wayland stuff is causing headaches all over the place, but I've also heard that it is shaking the rust off of bad old legacy systems, and I'm pretty ignorant about all Linux politics, so I hope it all just sorts itself out in a year or two and Qt and other players can adjust to whatever the new rules are. The hydrus mpv embed is amazing tech that I've butchered into shape in my crazy ass python Qt situation with duct tape. It has never worked in macOS. I am aware of another way of embedding it--iirc it is basically me spawning an OpenGL window and asking mpv to render to that, whereas atm I ask mpv to make its own window and attach it to my own window handle--which I'd like to prototype sometime as a more compatible alternative. A side thing as well--the native renderer is inefficient atm since it literally calls ffmpeg on the command line for a raw render on stdout, but there are one or two python libraries that may allow us to call ffmpeg in dll form, in our own process, which should reduce the overhead and overall jank in this system. It is something else we are thinking of as a possible project to try. And maybe one day I can figure out doing a 'software' audio renderer. We'll see how it all goes.

Really simple request; for Drag-and-Drop renaming in Exporting, could there be an option to choose which hash it defaults? The absolute majority of sites and boorus default to md5 hashes, not Hydrus' preferred sha256. I'd like it to set to export as md5 by default.

In file import options, can we get an option "if file already exists in database, overwrite its import date to now"? There are some situations where that will be useful.

Alright boys, been using Imgbrd grabber for a while but I'm getting tired of how it's getting buggier and buggier every update and they never fix videos. Tried to use Hydrus a month or two ago but all the words filtered the fuck out of me, and I feel like trying again today. Doing my best using the WORDSWORDSWORDS guide, but I still gotta ask, any ways to transfer bookmarks/favorites from that software to Hydrus? And anyone has a CSS/Style sheet for dark mode? When I use the color change thing it's still kinda half-white even with CSS override.

>>17449 never heard of imgbrd grabber so I can't offer help there, but assuming that bookmarks and favorites are something you can export to a file, Hydrus has a sidecar system for importing that can use sidecar files to import files with metadata. that might be able to help, but I never used it before because I never needed it. >darkmode go to "options → colors → current colorset" and change it to darkmode, then in "help" enable darkmode. I don't know why you have to set 2 different options, and I don't know why the darkmode checkbox is under help, but that should do it.

>>17450 >import files with metadata Yeah that one somewhat works, I'll see what I can do with that. >colors That one actually didn't work, but I managed in the styles tab by choosing dark blue. Thanks.

any ways to just browse images in this thing or is it always just automatic download of everything for a tag?

>>17452 what?

>>17453 any way to just look at the images and then decide what i want to save when i look up a tag or am i forced to dl everything and then delete what i don't want? like if i write artist name, am i forced to get everything from that artist or can i look up their shit and then decide what i want to dl

>>17454 no, you download everything, that's the entire point either write search requests for what you want or get an extension like Hydrus Companion that lets you pick and choose what you download

>>17451 Man, the import stuff is pretty nice but I wish I could figure out how to tag my old images better. Seems like I can put the folder they were in as metadata but it would've been more convenient if I could have set that metadata as artist tag instead since all my images were in individual artist named folders to begin with, would've saved a lot of work. Metadata artist name will have to do I guess.

>>17456 When you run a folder import you can add quick namespaces like "the second to last folder is artist:xyz" so if your path is /home/anon/pics/vincent van gogh/blue period you can automatically add "series:blue period" and "creator:vincent van gogh". If you already imported the images without deleting them you can re-import them with that rule and it'll add the tags. If you deleted them, you can use siblings to make them cleaner. Right click a tag > no siblings > add siblings to tag > leftmost box is "vincent van gogh" & rightmost box is "creator:vincent van gogh", that will automatically set all existing and future "vincent van gogh" tags to "creator:vincent van gogh". I'm using the standard hydrus tag namespaces but obviously you can use your own, it's your system.

Hi is it possible to add thumbnail support for archive file types? (.zip, .rar, .cbz, etc.) It can be as simple as reading a .thumb file if it exists inside the archive. I would love that since I can use hydrus to manage my texture files. Would help with stuffs like photobooks/artbooks/manga too.

keep up the good work dev, local booru was the missing piece to my self hosted paradise

>>17457 That worked pretty good, thanks. I had tried earlier but was confused because I wrote artist: but turns out you need to do creator:.

When sorting by a numeric rating, descending, with a limit, all the files on top have 0 rating.

>>17454 Install Hydrus Companion add-on in your browser, browse the website as usual and send the images you like to Hydrus through the companion add-on.

big n00b here, starting my selfhosting journey. Got a DS1515+ for free and I'd like to run hydrus on it so I have a 24/7 personal local only booru running. I tried to install it via docker but I'm too stoopid to figure it out. I got the suika/hydrus image from the registry, created a container and the log make it look like it's working but I cannot access it from the browser with http://yourdockerhost:5800/vnc.html or http://yourdockerhost:5900 using tiger VNC so obviously I'm missing something stupid and obvious.

>>17437 >Smart Screen I think that should only check exe on launch. I had hydrus running, downloaded some stuff, refreshed the inbox and got force-closed and client exe deleted with no recovery. Seems more like behaviour pattern recognition rather than simple hash popularity check. Though after reinstall all works fine with the same database same images etc.

Am I tripping and using the wrong blacklist or it straight up doesn't work when I download stuff? Had blacklist from tags -> manage tags in the all known tags & downloader tags and copy pasted my e621 blacklist and I still see vore/scat/etc being downloaded. The only thing the blacklist seems to do is remove the tags from the images. The only thing I can understand not working is the artist names since I imagine I'd need to add "creator:" at the start of every artist I want blacklisted because lol syntax but I don't get why the rest is still popping up.

>>17449 I always found hyrdrus to be a lot better when you take things more slowly and sort of ease into it. Don't just dump your whole collection and expect things to word out of the box. That's why I always recommend new users to only try a few images at a time just to get the hang of it. Keep doing what you've been doing but keep hydrus installed and always on your mind while still playing around with it until you're fully ready to commit. Definitely not brainlet friendly but goddamn amazing when you set it up the way you want it.

>>17472 >I always recommend new users to only try a few images at a time just to get the hang of it. This.

I had an excellent week. The first version of duplicates auto-resolution is ready for advanced users to try out. There's not much else in the changelog! The release should be as normal tomorrow.

>>17469 >Had blacklist from tags -> manage tags in the all known tags & downloader tags and copy pasted my e621 blacklist and I still see vore/scat/etc being downloaded. That section only controls display. What you want to do is go under "network" > "downloaders" > "manage default import options". Then click the "import options" button next to "default for file posts". Put your blacklist in "set file blacklist". This blacklist is will apply for all websites. You can set blacklists on a per-website basis in the section below.

>>17478 Thanks a lot. Some of these settings are put in some really dumb locations, legit spent an hour looking up on all the documentation and discussion to find where the correct blacklist is and it's always either the wrong thing or outdated images.

https://www.youtube.com/watch?v=aqzzowQcPZ4 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.macOS.-.App.zip linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v615/Hydrus.Network.615.-.Linux.-.Executable.tar.zst I had an excellent week. The duplicates auto-resolution system is ready for advanced users to try out. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html duplicates auto-resolution Any user can run this test, but I'll ask that you only go for it if you have a bit of experience with processing duplicates. Make a backup before you update, then go here: https://hydrusnetwork.github.io/hydrus/advanced_duplicates_auto_resolution.html Make sure your default duplicate merge options for 'better than' do not say 'archive both files' and then try out the jpeg/png rule. I want to know: if it all makes sense to you; where it runs smooth; where it runs garbage; do you get any errors. I'm also generally interested in your numbers. Out of ~800k potential pairs, I had ~6,000 jpeg/png pixel dupes, resulting in ~4,700 actual pairs (numbers shrink because multiple good files can share the same bad file). The preview panel ran pretty bad, about 30 seconds to load numbers, but the actual work was about 20k files/s for the search stage and 10 files/s for resolution. I cleared about 7.5GB of Clipboard.png in seven minutes! Thank you for trying it, and thank you for your patience as I figured all this out. next week Back to normal work, and I'll plan out future extensions to auto-resolution. Assuming the test goes well, I'll open up all the UI and invite everyone to try it out in v616.

Did the e621 tag importer broke again? I can no longer download any tags other than rating.

>>17484 https://files.catbox.moe/s4nzrh.png have to upload via catbox because "file format not allowed" on upload actually fixed, e621 changed tag format and added some shitty fucking "uploaded by artist" twitter badge in svg in the creator span that has to be filtered out note: this filters "contributor" to its own namespace, removes "creator:sound warning" because it's a trash tag and changes "creator:third-party edit" to "meta:third-party edit" via regex substitution.

When running version 615 on Debian 12, the program gives an error symbol lookup error: /home/%my user name%/Downloads/Hydrus Network/libQt6WaylandClient.so.6: undefined symbol: wl_proxy_marshal_flags what can be done? Try the Flathub version?

>>17482 so, I did this, went there add suggested, apply, and missed out on some of the cool stuff that was also there because I was half paying attention. now, my thoughts on this process. 1) I like that it automatically does everything and that's about it, what I dislike 1) that it automatically does everything let me try to explain, I had a bit over 19k pixel perfect matches. it went down to about to about 26,000 images before it decided to not do anything, and I had to hit work harder... not sure why, what I would like to see happen is instead of auto deleting them (well moving them to the trash) I would like the program to effectively earmark these, and let me decide on how to resolve them or when to resolve them, more or less, the program knows they are duplicates but lets me review them if I want to, I personally see this as a massive thing when its not as obvious as jpeg/png pixel perfects, given how complex this seems, double checking a few of these to make sure it was set up right before I press forward would be nice. now, because I was a dumbass and skipped the interesting things, I have to refer to the images on the help page, again well aware i'm a dumbass, but the image with the preview is interesting to me, the one that shows the images side by side in previews would it be possible to do something like that will the current duplicate filter? maybe as a companion to the large full image view? my use case is when I have an image set that may be 10+ images deep and all of them are minor variants, or when the differences are fuzzy enough that even larger differences still pop up as duplicates, hell, I have several thousand pngs that are just shitposts someone made that downgrade image quality while keeping a large file size and are seen as duplicates in the filter, easy for a human, even from thumbnails to figure it out, but would take quite a bit of time loading up one image over the other. overall I like this, but instead of fully auto resolve, can we have the program parse it, and then let us open it in some way to make damn sure its doing what we want before we ok the next action?

>>17489 Nevermind. "Fix" this by using Ubuntu

mpv added HDR support for Wayland. https://www.phoronix.com/news/MPV-0.40-Released

Hi! I'm having this rare but still intermittent issue where the api just stops responding. The issue persists until restart. Windows platform. Usually it happens after I try to access videos via the api in quick succession. How would I even go about finding the reason for this? The "api test" option in the debug menu has no effect until the restart as well.

>>17484 Ah, I thought I had broken something myself, glad I checked the thread.

For duplicate auto-resolution rules, I hope to see fuzzy filesize comparison (A's filesize is within x% of B's filesize), comparison of image resolution and comparison of aspect ratios. For JPEGs, also quality/compression profile. I think having these types of rules available would make a lot of work with variant sets automatable, and even be able to reliably detect obvious better/worse pairs.

>>17488 >removes "creator:sound warning" because it's a trash tag and changes "creator:third-party edit" to "meta:third-party edit" via regex substitution. Why would you do that? If I wanted those changes I could do them non-destructively with display filters and siblings.

>>17501 Because I cannot imagine a single situation where having "creator:sound warning" as a tag would be useful. In fact, the tag is far more likely to be wrong than the built-in "has audio" file property.

>>17502 I don't care about your lack of imagination.

>>17502 it's useful if you don't want your ears blasted by a sudden loud noise. what's hard to get about that?

>>17503 Then shut the fuck up and remove it from the parser filter for yourself. Here, I'll even give the exact path: Network -> downloader components -> manage parsers... -> "e621 file page parser" -> Edit -> content parsers -> creator tags -> Edit formula -> Big string processor button -> select "TAG FILTER: ..." -> X - > Yes -> Apply everything. >>17504 Then use the "system:has audio" predicate? Which does a better job? Out of 394 files with "creator:sound warning" in tags only downloaded from e621, 4 of them don't actually have any audio at all, and out of the remaining 527 files with audio, there's many that could be considered having "obnoxiously loud sound". I'd take it as a good, valuable tag if it were mechanically applied, like if the e621 website detected any sound past 30dB (arbitrary) on upload and added the tag (which shouldn't be under 'artist' anyway) but it's clearly not. To be fair, on the same logic I should also filter out the other various file tags like "meta:sound" or "meta:webm" because they're all useless, I actually forgot they even existed.

>>17506 If you're posting something on the internet for people to use, try to be user-friendly instead of enforcing your own arbitrary biases. Imagine if I posted a e621 parser that got rid of the species namespace and just moved them to unnamespaced, and then said "hurr durr i cannot imagine a situation where you would want the species namespace". You can see how some people might be annoyed by that, right? Especially in the hydrus thread were people are autistic about tags. I just don't see any reason to arbitrarily change the results to be different from what's actually on the page.

>>17504 If that's what sound warning was actually used for I'd agree.

>>17488 I'm dumb, explain this. To fix e621 tags not working I need to remove creator sound warning? I checked the catbox as well but it's just an image...

>>17504 >it's useful Sure, but only for you. Imagine if much of these moot feature requests are fulfilled, the software would be so bloated and slow that anons would be demanding for a Hydrus Light version. What about if you try the Mute icon before you load the videos instead?

>>17511 >Imagine if much of these moot feature requests are fulfilled what feature request? the reason they have the tag on e621 is to let people know that the video has a sudden audio spike. nothing about hydrus here so I don't know what you're responding to >What about if you try the Mute icon before you load the videos instead? because that just... mutes the audio?

>>17511 >there shouldn't be a jumpscare tag in any context >just like, don't play the video

>>17510 The image has the downloader data inside of it. Download the picture and put it somewhere you can find. Go to hydrus>network>downloaders>import downloaders. Click on Lain's face and find the image, it will tell you what you are importing, click yes and now the parser's been added. Now go to network>downloader components>url class links, find e621 file page, edit and set it to the new one, which should be e621 file page parser (1), matching parsers should be at the top.

In case anyone used the e621 with notes variant I updated it to work with the new parsing thing. I didn't change the sound warning stuff because I didn't feel like it. No image uploads so here it is: https://files.catbox.moe/t6jw3w.png

>>17512 >because that just... mutes the audio? Exactly, then you adjust the volume located right on top of the Mute icon just before to click on a video. Not a big deal, I think.

>>17514 >>17522 >>17488 Thanks

any ways to make some images children/parents of others? i'm seeing the children/parent tag thing but it doesn't seem to correlate

>>17526 No, not yet, you can set them as alternates*, which just indicates they are related, it does not have any sort of hierarchical info. I think parent/child images might be planned but I might be making that up. *select 2 or more images>manage>file relationships>set as alternates.

>>17447 Great idea, thanks. I'd still like to completely rewrite how export filenames are created. We want something richer than the current template string, and we need better fallback logic for when some data is missing. I'll write this down and think about it. >>17448 That's an interesting idea, thanks. I'll write it down. >>17458 We do it for cbz now, although my cbz detection algorithm is a bit shaky, so if your cbzs are read as zip, let me know. I think it would be neat for normal archives to pick a decent thumb, although I know some users don't want it because many results will be unhelpful (imagine if you were storing lots of technical brush zips and the thumb every time was just a 20x20 white square or other weirdness), so I'd introduce it as optional for arbitrary archives, I think. We have zip reading right now (python comes with it), but I don't think we have any tech to read rars or 7z yet. I think it is fairly easy to add, but I haven't looked into it properly yet. I'd like thumbs for cbr and is it called 'cb7'? I generally want more archive-parsing tech in future. You mention a .thumb file--is that something you see often? Is that a standard in a particular .zip format you use? Are there any others I could look for? My general approach here has been to list all the .jpgs and .pngs and just pick the one with the earliest filename, since that works for cbz. As I (slowly) roll this stuff out, let me know how it works for you! >>17459 Thanks, I am glad you like it. If you are a new user, then, in a few weeks, once you are comfortable with things, let me know what has been difficult and easy to learn. Keeping the help guides updated and overall UI easy for new users to pick up is a constant battle. >>17462 Thanks. I am afraid 'rating' file sort does not mix with system:limit yet. I added a tooltip to the file sort menu button a couple weeks ago so it actually tells you now which do and don't work. That said, rating shouldn't be too difficult to add, so I'll look into it.