You do amazing work Hydev. Thanks to you I can find any file I need to post in a split second.

https://www.youtube.com/watch?v=L5dcODmclFU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v584/Hydrus.Network.584.-.Linux.-.Executable.tar.zst I had an ok week working on some small jobs and new Client API commands. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I fixed a bug that was allowing wasteful file re-downloads from Pixiv and Twitter. I accidentally left a hole in the recent changes to the URL 'neighbour-testing' logic, where it tries to determine if a 'already in db/previously deleted' URL determination is trustworthy, and sites where posts can have multiple files were not able to return 'already in db' or 'previously deleted' until the file itself was redownloaded. I have filled the hole in--thank you for the reports, sorry for the trouble, and let me know if you notice anything else weird going on. This is a weird thing, but in the same way that if you double-click a tag in the normal search page, it adds that tag to the search, if you ctrl+double-click a tag, you enter '-tag'. It was difficult to pull this off due to ctrl+single-click doing a deselect (usually it was best done with ctrl+enter on keyboard), but I've smoothed out the click selection logic and it does, a bit more, what you think it should. Give it a go! If you ever use 'regex' 'system:url' predicates, they may run significantly faster this week if you mix them with other search predicates. regex URL preds now run absolutely last in the master file search, so they benefit if tags or other system predicates have already reduced the search space. Regex URL search is pretty much the worst-performing part of all hydrus file search, and there isn't much I can do about it, but I did a little research this week and I have a couple of ideas for the future--it might even allow for sometimes-fast regex tag search, but we'll see. client api Everything in the hydrus 'pending' menu (e.g. for sending mappings to the PTR) is now available on the Client API, with a new 'Commit Pending' permission needed to do it. A new command in the file relationships set also lets you remove all the potential pairs for files. next week I will make the tag siblings/parents dialogs load instantly. This job keeps on getting put off, but I am determined to finally clear it, even if it takes a couple weeks.

Given the recent e621 bullshit, and the potential for more stuff like it to happen there and elsewhere, one thing I'd like to be able to do is select an image in hydrus and somehow feed it a URL to tell it "Hey, even if you can't find this image here (because it was deleted or w/e), the tags and other data from this url belong to this image"

How do I set import options to only exclude deleted files from a specific file service? So I can have my SFW and NSFW importers separated but using the same database instead of two.

>>15723 >Thanks to you I can find any file I need to post in a split second Yup. Shitposting was never easier than now.

>>15724 >so they benefit if tags or other system predicates have already reduced the search space Does Hydrus do the same with tag searches?

>>15725 Can't you already do that with the url manager? >>15726 I think you can't do that.

>>15730 >Can't you already do that with the url manager? Not that I can tell. I'm pretty sure that just adds a URL to the list of URLs associated with a file, it doesn't pull tags, etc. from urls put into it.

>>15726 Are you using the 'import folders' feature? So you could add a SFW folder and a NSFW import folder and edit the 'import options' for each. this is a button in the center of the 'edit import folder' window when you create one or when you edit it, which might be hard to see. once clicked, you press the button in the new little window that pops up and change it to 'set custom file import options only for this importer'. there you can change the import destinations = local file service to which you like and also deactivate or activate the 'exclude previously deleted files' checkbox. with two import folders that would work for you?

>>15731 Oh, that's what you meant. Yeah that would be useful.

>>15724 >This is a weird thing, but in the same way that if you double-click a tag in the normal search page, it adds that tag to the search, if you ctrl+double-click a tag, you enter '-tag'. It was difficult to pull this off due to ctrl+single-click doing a deselect (usually it was best done with ctrl+enter on keyboard), but I've smoothed out the click selection logic and it does, a bit more, what you think it should. Give it a go! Thank you very much! I noticed one think that might not pop up in normal usage and only occured to me because i was testing this stuff: If the search pane has so many tags that the box is full and you have selected all tags, you kinda can't unselect any in a nice way. If you have only a few, you can click into blank space underneath to unselect, but if that space isn't there, the only way i know right now is to ctrl+left click one tag to unselect it and then left-click the same again to select the same and unselect all others. Would it be possible to allow the Esc-key to unselect all tags, just as it is possible to unselect selected files by pressing Esc? Alternatively an optional 'windows click behaviour' checkbox , which we spoke about some days ago, would be a way to solve this too -> > If you want a naked click to deselect what you didn't click, like Windows File Explorer does, I can write logic for that.

Not sure what the plans for the media viewers animation scanbar are, but i'd like to have some things that are missing at the moment: 1. Changing colors of the animation scanbar or even better make it optionally transparent with only a thin border, like Windows Media Player 12 has for example. Right now it is so bright, that i can barely see the frames/times. Only when i pause it becomes grey so i can see it well. I'd rather have it a darker color i think. Transparent might be cool and probably i wouldn't hide it and make it bigger at the same time. But i'd had to see it first to be sure. 2. Would it be possible to make it also optionally so, that if you chose to not hide the scanbar, the scanbar would be under the video and not overlap the lower part of the video, like in the VLC not-fullscreen view. that would mean a video with the same resolution as the monitor would have to be scaled so that black bars appear left and right (coz it would need to leave space underneath for the scanbar depending on the size you have chosen in the media viewer options), but that would make the frames/times always visible while the scanbar wouldn't overlap the video. 3. Maybe thats only a me-problem but i can't count how often i want to skip the time on the scanbar/timeline but dragged the video away xP. Maybe a checkbox for deactivating dragging for all animation/videos that have a scanbar? See it as an accessibility option for special people like me lol. 4. How about mouse wheel roll for fast-forward/backward in adjustable seconds (in options), while the mouse is on the scanbar? VLC also does that. Although i already see me skipping files accidentally like an idiot, id like to have it :D Those things added could change the viewing experience by a bit i think, at least for me, thanks!

Hi! One thing I noticed on v581: It was possible to add an artist to the different subscriptions by highlighting all subscriptions and clicking edit. That button is greyed out. It's now a little less convenient to add a new subscription across different sites.

>>15737 You should be able to change the scanbar color by editing a qss theme in the \static\qss folder. You can make a copy of the theme you use and edit that, then select it in hydrus, so that it won't get overwritten on update.

>>15739 Thanks, i might try to play with that. Is transparency possible? 'Background' color is the main color of the bar only while playing but not while paused i assume?

>>15740 No idea. Maybe try if rgba() works, but I haven't tried. https://www.w3schools.com/cssref/func_rgba.php Also no idea if you can change paused color, but that one doesn't seem to matter much as when you move your mouse away for the bar to hide, it will then use the other color again.

>>15741 >Also no idea if you can change paused color, but that one doesn't seem to matter much as when you move your mouse away for the bar to hide, it will then use the other color again. That seems to be a bug only when you have the checkbox for 'no, hide it' activated. i noticed that too. but as long the bar is only shrank and not hidden when moving the mouse away, it will stay gray. Thanks for the rgba idea.

Not sure if this is intentional or not, but clicking apply from the filename tagging window causes the import to commence, closing itself and the file import window, instead of saving the settings and closing the window, like most other windows in hydrus.

What happens to files freshly removed out of hydrus' trashbin when the client is closed quickly (while the files are being moved batch for batch into windows' trashbin)? Do they become orphan files? I wanted to delete 200 files, did not wait and just closed the client. In the end there appeared to be 35 files in the window's trashcan and running the orphan file search queue, it found 14 files only.

>>15744 And the 200 files in hydrus' trashbin are all gone? Interesting.

i need help with a problem. i moved my db to an external ssd, but there's a problem wherein the ssd improperly ejects whenever my pc goes to sleep, which i had just kinda been ignoring since it never really caused a problem (i know, stupid). my db got corrupted, but i had backups so it was all good. i made a backup just before moving it to my external ssd, but in the time that i migrated my db to that external ssd and i found it corrupted, it had imported some new files (from an import folder). i set my imported folders to delete the originals, so now these files are mixed in somewhere in my hydrus files, but i don't know where. i still have the old corrupted dbs. both mappings and caches are hit, and trying to recover them hasn't been very successful. i tried cloning them, but it doesn't work (my shell just spams Error 11: malformed disk image). is there another way i could locate those files. i want to reimport them. thanks whoever helps

>>15746 https://8chan.moe/t/res/14270.html#15144 > Check 'help my media files are broke.txt' in the db dir for info on how to resync it to your current client_files file storage.

I recently had to re-add Hydrus Companion to my browser. I copied over the API access key and it seems to be working for regular urls, but I'm not getting the cookies right, which I need to download things from Pixiv. Before, after clicking the button to send cookies to Hydrus, it would pop-up a notification confirming they were successfully sent to Hydrus. I can't recall if this notification appeared in the browser or in Hydrus, but I'm not seeing it anymore and trying to get urls from Pixiv that are r-18 fails if I'm not logged in outside of incognito mode, which means it won't work for subscriptions that I need to run in the background while I'm not logged in at all. Is there something I'm forgetting? I recall last time my mistake was attempting to send cookies from an incognito tab, but this time that's not the case.

>>15745 I've tried it again with a new batch of 145 files at ~300MB, closed the client as the files were removed from the trashcan, 18 made it into the trashbin until the client fully exited. Whilst exiting, the files were moved there very slowly, at one file per second whereas it would normally fill the trashbin at ~7files per second. Restarting the client show an empty in-client trash. 127 files are ... where? I'll check if they remain in the folders still.

>>15750 Nevermind, they have been deleted a while into this new client's session. Darn, I could swear I was up to something.

>>15723 >>15727 Hell yeah! >>15725 Interesting idea. I have a plan to rework an old system called 'lookup scripts'--which basically fetched a booru's list of tags for suggestions in the manage tags dialog--to instead just hit up known URL pages and grab all the normal metadata the downloader fetches, for retroative fetching of tags, post times, source URLs, whatever. Your thought here, to do it even on pages where the file differs, may slot into this sort of system. >>15726 >>15733 Yeah, you can set up an import route that positively imports to a certain place, but you can't do clever filters that filter import destinations based on individual local file service deletion records yet. I'll keep this in mind, but I don't think I'm ready to go this clever yet. >>15729 Yeah, pretty much all the different search routines take the work of the previous routine and use that as the base instead of the total possible file domain (so it might do 'system:has rating xxx' on the 1,200 files already positively matched, rather than the whole 700,000 file local domain). Tags are fairly early in the process though, since they tend to be very simple and fast. I try to do specific tags before namespace or wildcard tags. The general preference, obviously, is to do the simplest and most specific search procedures first to collapse the search domain down as fast as possible. Adding a specific low-count tag to any search will usually speed it up massively. The code is a mess and I do some unusual bullshit to wangle some id-based searching, but most of the method calls are english, if you are interested in the current order of operations. I haven't rigorously tested this order though, it is mostly just estimation/intuition and profiling/reorganisation when things do become a problem. Some things like OR predicates have multiple opportunities to fire, based on more complicated logic: https://github.com/hydrusnetwork/hydrus/blob/master/hydrus/client/db/ClientDBFilesSearch.py#L1293 starts for real here: https://github.com/hydrusnetwork/hydrus/blob/master/hydrus/client/db/ClientDBFilesSearch.py#L1414 Note that I do pretty much all of the search predicates separately and regularly dip out into python. A more usual database program might try to combine these into a single giga-query, but I only have time to KISS. >>15735 >Esc-key to unselect all tags Great idea! >>15737 Thanks, interesting ideas. I'm hamstrung with some of this stuff, usually the bells and whistles like transparency, since the media viewer is already held together with duct tape and string. I could see these things in future, but I have a backlog of very ugly code behind the scenes that I need to fix first. I'm investigating another mpv-embedding technique (a Qt-OpenGL thing) that may shine light on this situation and make it all more stable for everyone or may make things even more complicated and forestall new bells and whistles--we'll see how it works out. I can absolutely write some options to disable dragging on media with a scanbar, great idea. Mouse-scrolling over the scanbar also sounds totally doable.

>>15738 Thanks. I've been reworking my multi-column lists across the program to have better select/sort/scroll-to tech in the past few weeks, and it is going great except that I've decided to mostly move to 'edit one thing at a time' for technical reasons. Can I make your workflow easier by, say, adding a 'paste-to subscriptions' button? I keep meaning to add this for my own use, since I hate having to 'go into' a sub to do 'paste queries'. I could add that so one can paste into a sub just from looking at it, and for your case I could allow that to work on multiple subs at once. >>15743 Yeah, this is an ancient workflow. Were I making it again, I'd probably have that dialog be something you can go in and out of. It was intentional then, but I don't like it a lot now, so it is a candidate for rework. What would you like to see in a future version of this whole workflow? Showing tags and stuff on the initial drag and drop window, and then when you come 'out' of filename tagging it updates? Combining the whole thing into one dialog? >>15741 >>15742 The rgba won't work, I think, btw. That colour goes into a custom rendering thing I do (and thus do alpha on the underlying bitmap default, which will be all black or, perhaps, static), rather than setting the native actual colour of the scanbar. Qt doesn't generally 'do' transparency, when we are talking widgets overlapping, although I am no expert. I'll see what is going on with the paused thing not updating correct. I think for the paused colour it just like takes the current colour and reduces the brightness by 20%. >>15744 >>15745 >>15750 >>15751 >What happens to files freshly removed out of hydrus' trashbin when the client is closed quickly Yeah no worries these days, I put the files on a durable database 'to delete' table, and they are deleted over time in the background (and a sudden intervening re-import in the next few seconds does cancel that list). If there remains work to do, it continues working the list a few seconds after boot. If you have orphan files, they may be from a borked delete, but my guess is the delete was actually borked by something like a program crash that, let's say, succeeded in saving the file delete but failed to save to that 'to delete' table. There's a few gaps like that still in the program, but generally the normal 'physical delete' is reliable and you can close the client any time and not worry. Let me know if you discover any more problems though! >>15749 I am not sure, but if it helps, you can check what cookies hydrus thinks you have under network->data->review session cookies. Might be worth clearing everything under there for 'pixiv' in case there is some clash, but I dunno if Hydrus Companion cares about that sort of thing. Maybe there is an excess cookie somewhere messing up the logged in session.

>>15752 >I can absolutely write some options to disable dragging on media with a scanbar, great idea. Mouse-scrolling over the scanbar also sounds totally doable. Awesome, sounds good! How about color change of the scanbar? Do you suggest also editing the color through the way >>15739 suggested, or do you think you could add changing the color of the bar in hydrus options itself? Isn't it too bright for you guys too so you can't see the frames/times?

>>15749 >>15753 >I am not sure, but if it helps, you can check what cookies hydrus thinks you have under network->data->review session cookies. Might be worth clearing everything under there for 'pixiv' in case there is some clash, but I dunno if Hydrus Companion cares about that sort of thing. Maybe there is an excess cookie somewhere messing up the logged in session. Seems to have done the trick. I had 20 cookies on Pixiv and now I have 15. No idea why the cookie confirmation message no longer shows, but both manual downloads from an incognito page and subscriptions with no currently logged in browser for r-18 content are functioning now.

>>15753 >I'll see what is going on with the paused thing not updating correct. I think for the paused colour it just like takes the current colour and reduces the brightness by 20%. Well, it makes the bar brighter for me, so that's not it.

>>15754 Yeah, edit in the QSS for now. That stuff is still a little prototype--I only just added the stuff in the options->colours page to QSS the other week--but that's the way we are broadly going in future. Make a copy of the stylesheet you like with a new filename in the /static/qss directory and then edit the colours for the scanbar stuff. Then set that stylesheet in options->style. Now we know this way of sending colours from QSS to hydrus proper works well, I think adding a pause colour is a good idea. I don't know when it will happen, but I can expand this to have more colours. I don't know what it uses for text, I guess it uses the default for the current stylesheet, but if you want whiter text or whatever, that might be something else to play with. >>15756 Yeah I think it goes 'if it is dark, add 20%, if it is light, reduce 20%'. It is just some dumb old 'get an alternate colour' method I made a while ago, I use it in the duplicate filter too. We can do better.

>>15748 thank you sm. can't believe i didn't check that.

>>15757 >Yeah I think it goes 'if it is dark, add 20%, if it is light, reduce 20%'. It is just some dumb old 'get an alternate colour' method I made a while ago, I use it in the duplicate filter too. We can do better. But thats good. I worried that if i change it in the QSS to something very dark, the 20% darker while paused wouldn't make it look much different. If it will make it brighter then as you say, thats actually good to distinguish. Have a nice Sunday.

>>15749 At least on LibreWolf on Windows, the notification comes thru as a system notification from the browser. If you've done something to hide/disable windows notifications for your browser, that may be why they disappeared. Pic related.

>>15760 That's probably it. I stripped a lot more out of windows this time.

>>15724 > >I will make the tag siblings/parents dialogs load instantly oh nice I was just gonna ask if it was normal for the manage tag sibling dialog to hang for like 10 secs on open every time

>>15195 >.... you do have backups right? Right??? Surely you must be 16 to post on this forum. >Life pro tip: Always RTFM. Read the fucking manual. don't care didn't ask kys.

>>15764 >oekaki Saved.

>>15763 Hope you lost your files, child.

In a downloader's file log, clicking an entry focuses the cell, so pressing Ctrl+C will copy usually pretty useless contents of that cell instead of the urls of the selected entries.

Just updated to latest from v558 and i cant find the option to switch back grouping namespaces in the left side of the media viewer and manage tags permanently. Would appreciate any help.

>>15768 For me default they are grouped in the media viewer already, means first come the namespaced tags then the unnamespaced. But there was never an option to change the sorting of tags in the media viewer itself as far as i know, only in the thumbnail viewer. And if there was (which wasn't afaik) then why should an update change your settings? You can change the tag display in 'tag' -> 'manage tag display and search...', then chose the local tag service on the top tabs and then press the 'tag filter for single file views' button (single view = media viewer; multiple file views = thumbnail viewer) to whitelist/blacklist tags that you want to display in the media viewer. Is this what you are looking for with 'manage tags permanently'?

>>15769 >But there was never an option to change the sorting of tags in the media viewer itself as far as i know, only in the thumbnail viewer. It's all under options > sort/collect

>>15770 Oh indeed. Guess i have once set it up and never looked at it again. Thanks for the reminder!

>>15770 That did it, thank you very much. Dont know why updating changed those options though

>>15772 I think they weren't split before, so when the media viewer options got added, they were set to default.

>>15766 they're mixed in the Grand Hydrus Disorder, at best.

>>15753 >Yeah, this is an ancient workflow. Were I making it again, I'd probably have that dialog be something you can go in and out of. It was intentional then, but I don't like it a lot now, so it is a candidate for rework. What would you like to see in a future version of this whole workflow? Showing tags and stuff on the initial drag and drop window, and then when you come 'out' of filename tagging it updates? Combining the whole thing into one dialog? Honestly with how relatively complex the the file import process is if you do anything more than simply import a bunch of untagged files, it may be more prudent to break the file import process up into a multi-step wizard sorta thing. I imagine the process would work something like this: When a user drags and drops files onto hydrus they would be presented with a dialog similar to the review files to import window now, essentially just a list of files hydrus found to import, but with two check boxes: one for sidecar tagging, and one for filename tagging. If the sidecar tagging box is checked, the user will be presented with the sidecar tagging options in the following step. If the file name tagging box is checked the user will be presented with the filename tagging options next. Finally a confirmation dialog of sorts, listing all of the files and the tags that would be added. Could also add some maybe sometimes useful options here too, like publishing imported files to a specific page. Either way though I would advise against condensing the whole thing to a single dialog, doing so may make it rival the options menu in complexity...

>>15681 https://8chan.moe/t/res/14270.html#15681 Thanks for this. I've found two files on the btrfs with the most errors that are only partly readable, and then Input/output error happens. So it reports errors for smaller parts. There is an option that sounds like it could help recover some files before ditching the filesystem, but it seems dumb. --init-csum-tree create a new checksum tree and recalculate checksums in all files WARNING: Do not blindly use this option to fix checksum mismatch problems.

is it only on my machine or is wayland support kinda fucky? i'm running it on KDE. i need to launch it with QT_QPA_PLATFORM=xcb or i get all sorts of UI bugs (and mpv doesn't work).

>>15778 >Wayland + Plasma >mpv doesn't work I'm running on X11 and every time I ran into trouble, the solution was to build Hydrus from source. Not a big deal, give it a shot. https://hydrusnetwork.github.io/hydrus/running_from_source.html#walkthrough

I had a fantastic week figuring out tech to make the 'manage tag siblings' dialog boot and work faster, even if you have hundreds of thousands of pairs. I now have a 98% working prototype that does everything--and a few workflow improvements--without needing to load up all the pairs on every boot. Rather than do a release tomorrow, I will copy this work to the manage parents dialog, polish the whole thing, and catch up on other work besides for next week. v585 should be out on the 7th. Thanks everyone!

>>15780 as someone who adds relationships very often, this has been one of the biggest enduring painpoints for me. It'd be amazing if it was no longer an issue!

>Rather than do a release tomorrow, I will [...] polish the whole thing The only thing better than a release is no release, because our installs stay safe from a potential break and the next release will be even better.

is there a way to send multiple selections to external viewers?

>>15778 >wayland >kinda fucky no! I cannot believe it!

>>15782 That's because your genes are expressing a tendency for low impulse control, You see a new release and feel the urge to install it. Perhaps race-mixing was in your family lineage and nobody told you.

>>15780 >>15781 yeah this is a huge change for me, can't wait for this

hello sorry i am retarded. i tried following the instructions but no tags seem to show up even though i followed the settings. i thought hydrus would automatically add the tags when importing.

>>15791 Are you importing manually from your hard drive or from the internet?

>>15792 i thought that you could get the tags from danbooru, etc and then tag them with those

>>15721 Is there an option to change the layout of files page? I want to have preview window on the right, search and tags on the left, and wall of thumbnails in the middle.

Been using hydrus for a minute, but haven't used the gallery DL features. Trying to parse an exhentai URL, but it only seems to support e-hentai? Is there no downloader for exh?

>>15797 You can find one here: https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts Not sure if it's up to date though, I remember they changed the image download address a bit so I had to modify it.

>>15798 TY for the response. I did import that, but it still doesn't appear to be working. I'll play with it a bit later.

>>15791 I assume you mean import of local files, read this: https://hydrusnetwork.github.io/hydrus/PTR.html#janitors Downloading PTR will take you probably 2 weeks or so and you need at least 50GB of local space. >>15796 You can't put stuff on the right, but why not get rid of the preview window alltogether, since thumbnail are big enough (plus you can resize them in options) imo and just have the system predicates and tag selection box on the left. 1) options -> guy pages -> hide preview window (last checkbox) -> activate 2) options -> search -> autocomplete dropdown floats over file search pages (first checkbox) -> deactivate 3) directly under "2)" -> autocomplete list height -> i got 16, but you can play with it for depending on your resolution 4) options -> thumbnails -> play with the first options so you dont actually need a preview window anymore, otherwise just get used to mouse-wheel click a thumbnail and it gets opened without needing to double click And if you grab the side of the search pane, so you can make it bigger and the thumbnail grid smaller (or vice versa), you can make it so that the last thumbnail in a row just doesnt fit into the grid and therefore a free space with almost the width of a thumbnail will exist, which makes all thumbnails look "kinda" centered. Not exactly what you want i guess but a bit, minus the preview pane. >>15797 Does it maybe have to do with cookies and login, since in a browser you need to be logged in in e-hentai before you can see exhentai too? I don't use those functions though, so just a suggestion. network -> data -> review session cookies network -> logins -> manage logins

>>15800 >https://hydrusnetwork.github.io/hydrus/PTR.html#janitors Sorry, don't start from 'janitors', but from the beginning: https://hydrusnetwork.github.io/hydrus/PTR.html

>>15799 Try pasting the following into the content parsers tab of "ex/e-hentai.org post page parser": [30, 7, ["urls to source download (new format)", 7, [27, 7, [[26, 3, [[2, [62, 3, [0, "a", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]]], 0, "href", [84, 1, [26, 3, [[2, [51, 1, [2, "https://e(-|x)hentai.org/fullimg", null, null, "https://exhentai.org/fullimg/1264932/1/1dojppu94mr/001.jpg"]]]]]]]], [7, 50]]]

I happened to notice a ton of the files in my db folders have no file extension anymore. Is there any reason Hydrus is stripping file extensions from some files? I tried looking up the hashes of a couple and Hydrus returned nothing. I can't tell what they are since no file extension means no proper thumbnail in my file browser, and I tried adding a bunch of different file extensions to one of them to no avail. Some files I blanked out so I could post unspoilered.

oh no that's too much i just wanted to import my collection and then easily tag them from tags that already exist like through boorus but it's ok i'll just try and do it manually lol

>>15804 >oh no that's too much >i'll just try and do it manually lol Welcome to autism land. You are about to undertake a task that is far more time consuming then just getting the PTR if your computer can handle it, though depending on how many files you have. Yesterday marks two years since I started using Hydrus and manually tagging files. I project I'll catch up with my pace of file downloading within the next half year. Whoops, forgot the graph key.

>>15804 It's possible, but the files need to be exactly the same as the ones on the booru or it won't find anything. https://wiki.hydrus.network/books/hydrus-manual/page/file-look-up I also recommend reducing the wait time between gallery fetches to 1 second for this in options > downloading, as the default is like 15, which takes forever. You may want to change it back again if you plan on using multiple normal (ie. not 1 image per gallery like in this case) gallery downloaders at the same time in the future.

>>15806 >>15807 ok thanks for the tips i'll see what i can do yeah i love to collect pictures but it's just a hassle the way i have them organized now hopefully i can get hydrus to work for me

>>15808 If nothing else, you can easily replicate your file structure as tags in Hydrus, losing nothing while still gaining some of the benefits of Hydrus with minimal effort.

>>15803 Lol. I'm pretty sure those are the 'repository update' files which you are looking at. Activate advanced mode under 'help'. Then change your file domain to 'repository updates'. Here you can see them. Right click one -> open -> in file browser. Those are not your normal files. I guess you can delete them if you processed all the repository updates already. But don't quote me on that. Hydev once said that afaik. >>15804 Beside the link from >>15807 , you also can get tags from every new stuff you get through the in-client downloaders, means that you have to directly download them into hydrus and not some windows folder before. network -> downloaders -> manage default import options... Here you can setup if you wanna get tags and notes or not for each website or in general for all (upper right 'import options' buttons). You have to activate the 'get tags' checkboxes for the tag domain you want to have them first (after going into a downloader or the buttons on upper right top) and if you also want to retroactively apply the tags to a file that you already have in hydrus, but you found the link for it again, then you also should activate the two 'force page fetch even ... already in db' checkboxes. Otherwise it will not get tags/notes for old files. Then you can activate the url watcher. that will send your links after you right-click -> copied them from the browser adress bar, into hydrus, automatically: network -> downloaders -> watch clipboard for urls -> activate one or both options there whichever you need Or you can drag&drop the URL from the browser's adress bar directly into hydrus too. That way it you can have tags for your new files (often faster than PTR, since you have to rely on people to upload them to the PTR first) and also for old files (in case you find them on one of the websites that can be parsed with the in-client downloaders).

>>15810 >Then change your file domain to 'repository updates'. Here you can see them Forgot to say, also search for system:everything of course.

>>15810 ok thank you i'm trying that right now :3

also is there a way for the tags that are downloaded from pixiv to be the english versions?

>>15806 >inbox/archived relation R for respect.

>>15810 >I'm pretty sure those are the 'repository update' files which you are looking at. Yep. Changed the domain to repository update files. There's around 5,700 of them, and they total around 2GB. A bit curious as to what they do and why exactly they need to mixed in with the db files.

>>15814 I just do it in chunks. I still have several thousand files left in traditional folders, on top of those I'm adding with more frequency due to how Hydrus makes collecting easier.

>>15815 Oh, seems i forgot to explain better. PTR = Public Tag Repository = your repository files most likely. But you can also add other repositories, which most likely you didnt. So those should be the PTR update files. You are connected to the PTR right? services -> review services -> remote tab -> tag repositories subtab -> PTR All the updates you download are the files without extension you mentioned and in the 'repositroy updates' domain. They have to be downloaded first before they are processed, which ends up as updated tags+siblings+parents. After processing they stay in the folders and the domain. There is a button to use those files for example on other clients, i guess without the need of being connected to the PTR too, but not 100% sure. But then they still need to be processed into the database: services -> import repository update files... (on bottom)

>>15817 >services -> review services -> remote tab -> tag repositories subtab -> PTR And here they can be exported by the way, with the 'export updates' button kinda in the mid right.

>>15800 Thanks, but making thumbnails big is not ideal, since they would take up a lot of space, and I have a lot of images. Just thumbnails DB's size was ~50 GB. Previously, with FastStone or XnView i'd look at small thumbnail, then at quite big preview if I was interested, and then open it fully if I was further interested. Tag and search interface doesn't need a lot of width, so it's almost pointless to widen it, only for constraining thumbnail grid, but for the preview a separate column in interface would be very handy.

>>15819 >Tag and search interface doesn't need a lot of width, so it's almost pointless to widen it, only for constraining thumbnail grid, but for the preview a separate column in interface would be very handy. I agree that a right column where you could put in the tags box or preview would be cool. that would kinda 'center' the thumbnail grid. Or that column could show other content directly. maybe the notes or even better: alternates after you click on one thumb in the main grid. Right now if you grab the edge of the search pane, you can make it disappear by dragging it completely to the left. Why not having a right column that you can drag to the left to make it appear (and also add it under the existing 'show/hide sidebar and preview panel' and call it 'show/hide right/alernate sidebar' or so), maybe with some tabs to change what is displayed like tags, notes, alternates, preview etc. Who knows what the future brings :)

>>15817 >You are connected to the PTR right? No. I attempted to once before I realized I didn't have enough computer. Later I decided I didn't want other people's tags, only my own personal tags. I believe I had taken the steps necessary to delete what portions of the PTR I had downloaded since it was a waste of space. >services -> review services -> remote tab -> tag repositories subtab -> PTR The remote tab is entirely blank. However there is pic related.

>>15821 Having a PTR is good as a reference i think. Of course 80GB+ is alot, but it MIGHT be worth it. The PTR has its own tag domain and you still can have your own tags in the 'my tags' domain and add more domains if you want. There are ways to even mass migrate only certain tags tags you want from one domain to another, lets say PTR to 'my tags', with help of black/whitelists, which can save you alot of work/time. It looks like you can delete those 2.18GB i would assume if its annoying you.

>>15810 >if you also want to retroactively apply the tags to a file that you already have in hydrus, but you found the link for it again, then you also should activate the two 'force page fetch even ... already in db' checkboxes. Otherwise it will not get tags/notes for old files. You don't need to do that if the file doesn't have the source url attached to it yet. I retroactively get tags for "blank" files all the time and it just works.

>>15823 I had to activate those 'force page fetch...' checkboxes, otherwise the tags wouldnt get added for old files which have the same hash as the media of the link you drop into hydrus. Even if the 'get tags' checkbox is activated. Thats why the checkboxes are there in the first place. Are you talking about 'blank' files getting tags, because you use the PTR and thats where they get the tags from? Otherwise i can't follow you and what you mean with 'source url attached to it yet'.

Dunno if this would be useful for anyone else but how viable would it be to add a toggle to have folder imports go in order of date (last modified, created etc)? Use case being a batch of files being downloaded elsewhere outputs in order by date, but the filenames are hashes so the default view on Hydrus has everything mixed up. My current workflow to get around this is just renaming them all in sequence based on their last modified date.

>>15824 I don't use the PTR. By blank files I mean fresh files that have no tags, urls etc. yet, or have a little. What I usually do is that I download an image from something like twitter, which only gets the creator tag and twitter post + file urls, then I copy the md5 hash of that file and paste it into a booru gallery downloader, which will grab the tags and urls of that booru without even having those 'force page fetch...' options checked. Works even if you find the image on the booru yourself and paste the booru url into a normal url downloader manually. Sometimes I even download the raw image straight from the booru itself (because of a script I use in my browser that makes it easier to get file links than post links) and then retroactively pull the tags using the same method. I only have to check the options if the file already has a post url of that booru attached to it.

>>15825 Maybe it would be good to show a 'modified date' column, yes. Right now Hydrus does allow you to sort the import sequence only for name, filetype and size. It doesn't parse the creation and modified dates when showing you the files in the pre-import list it seems. I think only after you press the 'import' button, Hydrus parses the modify dates of windows (But not the creation times, since the files get copied to the database folders and would be the same as the import times). BUT i have good news nevertheless (if thats what you want): The pre-import window shows you a cog symbol button near the 'close' button on bottom right. If you click it and deactivate the checkmark of 'sort paths as they are added', the files get added exactly like they are in windows. That means if you pre-sort the files in Windows for 'creation date/modifed date', you select all files within a windows folder and drag&drop them into hydrus. But: - you have to grab the very first file when you drag&drop them into hydrus, not any other file. Otherwise the sorting gets wrong - you cannot sort the files in the folder and then think that you can go back a direction and drop the whole folder into hydrus, that won't work. you have to drag&drop all the selected files themselves, NOT the folder icon where the files reside >>15826 Ok that's interesting. I just said it cause i tested it with a danbooru file. 1) I downloaded it without the checkboxes activated -> no tags 2) Then i checked the 'get tags' checkbox (within the danbooru downloader settings itself, not globally which you can also do) and put the link in hydrus again -> no tags 3) Then i checked the 'force page fetch...' checkboxes and put the link in hydrus again -> tags appeared So i don't know where our differences are. Can you confirm that you haven't activated the fetch checkboxes globally too? See network -> downloaders -> manage default import options -> upper two buttons that say 'import options', so not the individual downloader settings themselves. If you haven't activated them there, i dont know really why that is :)

>>15827 >Can you confirm that you haven't activated the fetch checkboxes globally too? Yeah, defaults are unchecked and individual downloader tabs are set to default. I even tested this in a fresh install and it still works that way. Like for example I download the raw file first: https://cdn.donmai.us/original/23/70/2370ac8fc8c8f0ca843e1c0940df492d.jpg which just gets the file and nothing else. Then I paste in the post page: https://danbooru.donmai.us/posts/7214688 which only gets the tags without downloading the pic again, because it recognizes it. Are you sure the image you're trying to get tags for doesn't have the danbooru post url already?

>>15828 > Are you sure the image you're trying to get tags for doesn't have the danbooru post url already? Yes i think that was it, thanks! Because i downloaded directly to Hydrus without tags and not to a Windows folder, which i should have done, so it got the post url attached. Ok after some trying now i get it. I had a little bit of a problem to understand it first, since i thought when you said 'default' and suggested a 'fresh install', you also meant the 'get tags' checkbox deactivated. But that wouldn't work and wouldn't make sense i guess. But with at least one 'get tags' checkbox activated (depending on how many tag domains you have) and the fetch checkboxes deactivated, it works like you say, yes. As long no url is attached. So for newbies: after you still don't get any tags even after activating the 'get tags' checkbox, try the 'force page fetch...' checkboxes too. In my testing, i had to activate both of the fetch checkboxes. Either one them alone didn't work in case of danbooru.

How taxing would it be implement a function that shows you how many other people in the same tag repository have a specific image hash? It'd be very useful when gauging whether you should bother uploading to the PTR versus your own local tags, and for correlating between alternates, as well as PTR maintenance for unused hashes, but it definitely should be opt-in only, even if it's just hashes being swapped around. Probably something that only sends all the (selected) hashes you have to the PTR and sends out a simple integer along with every tag mapping for every file of the users that have said hash since last 'census'. I'd similarly like for duplicate detections to be correlated, but that'd be even more complicated and would need some kind of consensus error-checking system.

Got a small feature request for you, Hydev. When selecting "save this search" as a favorite search, can there be a dropdown list of existing favorite searches to save over? It's much more natural to edit a favorite while actually using it since you see the results, and then save this over the new favorite, but that requires either manually typing in the exact name of the existing favorite, or going into the manage favorites window and entering the changes there a second time. I do this regularly as I manage what files I have an haven't posted in various places, and the criteria for what files I think should be posted in a certain place regularly change.

Suppose I download an image from booru and import it's tags, later tags get changed on booru, some deleted, some added. I found the way to get newly added tags, but is there any way to remove deleted from booru tags?

>>15767 Thanks, I think that is probably default Qt behaviour. I will add a hook so it explicitly copies the URLs. >>15775 Thanks, I will keep this in mind. I think you make a good point about avoiding the mega-complexity of a single dialog. First thing I should do is simply rewrite the objects behind all this stuff so it isn't so ugly to work with. A lot of it is just a bunch of tuples flying around in my twelve year old code. >>15781 >>15790 Thanks, let me know how the new workflow works out. I've added the idea of a 'sticky workspace' although I think I've explained it bad in UI. Basically related pairs now stay in the list until you are done with the stuff you are working on, so I'm hoping it will be easier to make multiple edits on the same large group. Let's try it out on some real world situations and see where it does well and badly, and iterate. >>15803 >>15810 >>15815 >>15817 >>15821 Yep if you no longer sync with the PTR, you can delete those files. Just do the 'repository updates' system:everything search and ctrl+a delete, and it should work out ok. I'm still working on better 'clean-up all this excess shit after I remove the PTR' tech, but this will happen automatically in future. The files are just zlib-zipped json, if you are interested. If you open them up in python and zlib.decompress, you'll mostly just get some long lists of numbers and sometimes tags/hashes. I hang on to these files mostly for network isolation/durability reasons. If the PTR ever goes down, a client can still reprocess to fix holes or whatever, and a single client can potentially create a new PTR if that is ever needed. >>15813 This is crazy, I'm not the foremost expert in how Pixiv works, but their fucking API (https://www.pixiv.net/touch/ajax/illust/details?illust_id=121160658) is giving different results to my test browser session and to hydrus. There must be some 'hey my language is english' header that the API is parsing, and hydrus is not sending, and thus it is delivering the translation in the 'correct' language dynamically. It didn't used to work this way (translated tags used to just work); I will look into it, thank you for the report!

>>15838 >Yep if you no longer sync with the PTR, you can delete those files Woot.

>>15816 Based, I kneel. >>15819 >>15820 I can't speak with great confidence since I am always overwhelmed, but I feel like I am just getting the hang of Qt now, and just about clearing up some of the last sticky ugly hacks we employed to get through the wx->Qt transition. There's still more to do (I hope to make multi-column lists populate/sort quickly in the next couple months), but I think I can start seriously thinking about moving to a more dynamic and user-customisable UI within the next few years. I'm a boomer and think of the 2008-tier Eclipse editor when I think of modular UI design, but that's what I imagine in my head. Ideally I want you to be able to anchor and resize stuff in different places; I just have to do a whole bunch more cleanup to decouple all the bad code I've written over the years. That's the objective, and we are slowly getting there. >>15825 Interesting idea--I will see what I can do. >>15832 Impossible with current tech, I'm afraid. The PTR doesn't know who has which file unless they submit a tag for it, and even then it is only assumed (since the user may have since deleted the file, or have acquired the tag/hash combo through some more esoteric means like a Client API action from an external db). Best analogue for 'is this file popular in any way?' with current tech is just how many tags it has. The stuff on the boorus obviously has like fifty tags, whereas obscure memes will just have one. I won't add census/tracking as an opt-in to the PTR since I designed it to be maximally private. I also designed it to be bandwidth minimal, and various 'hey just so you know, I have these files' ideas that would require a client to regularly check-in with a server, either sending info or getting it (as in, with some Anons' ideas, of "hey, can the client not sync but instead ask the PTR for any tags it has for new files?", which is a similar concept), will increase network traffic, and server CPU (and now per-account storage), significantly. I originally planned to have a 'ratings repository' where we'd all submit ratings on files and it'd combine them into a regularly-updated aggregate, but I backed off since I ran into the same issues and it simply didn't fit my model. Shared duplicate info would be similar, and the moderation workflow would be complex. The PTR is cool, and hydrus repositories are neat ways to share content anonymously, but I'm not the guy to make anything even two steps towards aggregated social media. In future I am going to go even more isolationist as we train AIs on existing PTR content and start to drop common nouns from fixed-mapping sharing (e.g. when an AI model can recognise a 'skirt' on a novel image file, there's little need to share explicit (file-tag) mappings on a per-file basis any more). >>15835 Yes, 100%. I hate the 'favourite' search management UI, pretty much top to bottom! I keep wanting to update a search to the current but then remember how much of a pain in the ass it is. This would solve it nicely. >>15837 No, not really. Even if there were an explict record of 'hey this tag was deleted from danbooru' in some API somewhere (rather than just inferring by missing tags), hydrus just doesn't have good 'deleted' pipelines yet. I'm still thinking about all this, but I'd like to put work into it. I want better visibility of deleted content and better ways to pipe it around (and, as you say, explicitly parse it). You'd have to hack something together with the Client API for now.

>>15840 >In future I am going to go even more isolationist as we train AIs on existing PTR content and start to drop common nouns from fixed-mapping sharing I seriously, deeply doubt that AI will ever get to that point. Yes, it will get better and yes, confidence levels will rise, but efficiency won't plummet to such a level that you could expect anyone who can run Hydrus to also be able to run a local classifier model, especially at a rate that it would be able to classify the thousand to hundreds of thousands of images in a local repository. I may be wrong, AI accelerators may become so ubiquitous that there'll be something wrong with your system if you don't have one, but even that would be a ways off. Even now, classifier-taggers require multiple gigabytes of working memory and a good CPU/GPU to run, and that's with dogshit confidence intervals. To get the 99.9%+ confidence interval required to eschew common tags at all, you'd need a far beefier, more expensive model. And even then, I believe there are fundamental limits on how good a job classifiers can do versus what a user actually sees (like all those thumbnail switcheroo images where the thumbnail is one thing but the full image is entirely different – that will entirely fuck up any classifier, yet both have taggable content)

>>15837 You could delete the tags from the file you are trying to update manually before getting the tags again. Though you will have to also look for an option in tag import options that will ignore deleted tags, or you could clear deleted tag records using the tag migration tool.

>>15841 >efficiency won't plummet to such a level that you could expect anyone who can run Hydrus to also be able to run a local classifier model, especially at a rate that it would be able to classify the thousand to hundreds of thousands of images in a local repository if I'm understanding what you're talking about correctly, I'm already doing it right now, and yeah for vanilla-ish anime-styled artwork (exactly what most of the tags on the ptr are for) it works fantastically. I have a very weak pc as well and it still works, just kinda slow. it's automatic though, so I'm fine with it being slow since I can just let it run in the background and tag for me. getting this up and running was one of the biggest reasons I stopped syncing with the ptr. for me it's just semi obsolete at this point.

>>15832 >a function that shows you how many other people in the same tag repository have a specific image hash? A very, very, very bad idea as it will only help glowies to fish for anons with specific files in their drives. Are you in the ZOG's payroll, anon?

>>15843 Yeah, it works great for 90% of anime images. But the weird 10% it struggles with, and the moment you add significant amounts of 3DPD or worse, furry, it becomes unusable. False positives are also a big issue. It *could* be solved with a whole classification stack that identifies the type of image, then chooses an appropriate classifier to minimize resources used, or just one big fuckoff model to make it all work regardless of type, but they have their own problems. All I'm saying is that I wouldn't depend on it compared to a (semi) curated tag repository. The PTR might have plenty of misclassifications, but those can be fixed, unlike a model that will keep making the same mistakes until you swap it out. Direct classifier integration would be nice, but there's plenty of tools that can indirectly or directly interface with Hydrus to populate automatically populate tags.

>>15841 >I seriously, deeply doubt that AI will ever get to that point. I totally agree. A reminder that there is not such a thing as "A.I." as it is just a marketing ploy to sell a bunch of algorithms arranged in tandem as the ultimate hype while gaslighting humans into believing it is the next step in transhumanism.

>>15846 There is such a thing as AI and always has been, the term itself has just been absolutely annihilated by popular vernacular. Classifiers are going to get better, but not so good that it can replace you, the human looking at a screen and using Hydrus. It'll still be useful, just not a panacea.

In migrate tags, removing pending tags should probably not be called "petition".

>>15846 >mlp >green anon It's shit

>>15849 Uh, it actually petitions, so the tag has (+1) (-1).

Also looking forward to that patched Shimmie scraper. Every so often I find a cute little Shimmie and the new parser is great for quickly adding it in. So thanks for that! >>15847 Anonfillyposter is right. Obviously machine learning exists as a technique and produces tools, but to call it "AI" is absolutely a junk marketing term adapted from science fiction. It's not intelligence.

>>15854 Artificial Intelligence is any artificial algorithm capable of making decisions based on its environment. Its environments are pictures and text. It makes decisions based on those. It's artificial intelligence. A* pathfinding is also artificial intelligence. The bullshit marketing is temporary, but the field has existed long before it, just like quantum science has before quantum anything became the bullshit buzzword of the year. Ironically, the association with science fiction is the most likely source of actually bringing Skynet to life through upheld expectations. AI slop is trained on the very same datasets that morally panic about AI slop. Talk about AI candy.

I am experimenting with FreeBSD and having some issues. I want to have hydrus in that machine and I am wondering if anybody tried running hydrus on FreeBSD. If it doesn't work I can just ditch FreeBSD and install linux.

>>15857 I have problems with BSD itself and I want to know if I should try to solve them.

>>15846 >>15854 >>15855 Good enough poners. Let us not derail the bread and piss devanon off.

>>15857 >>15858 I guess your best bet is trying the source package. Then theoretically speaking, a Venv for BSD should work as it would allow Hydrus to run in a virtual Python environment. I'm dumb as fuck in programming so I cannot elaborate any further. https://hydrusnetwork.github.io/hydrus/running_from_source.html https://forums.freebsd.org/threads/how-to-install-a-virtual-python-environment.92015/ https://duckduckgo.com/?q=can+freebsd+run+linux+programs&ia=web

>>15860 I read about a compatibility layer for running linux binaries on freebsd but I was wandering if somebody already tried something like this so that I don't waste my time on solving whatever is preventing my freshly installed bsd from booting consistently.

>>15813 It is possible, and I had it before, but I removed it and I highly recommend you don't do this. Many (too much imo) of the "english" tags are just flat out incorrect translations, so if you have hydrus grab those, you'll be added tags that either don't apply or don't even make sense.

>>15847 I use an Ai tagger to tag my downloads, and it works pretty good! One thing I like about it, it can recognize untagged loli, and tag it as such. That lets me get rid of it before I start perusing what I downloaded.

>>15813 The only Ai so far that I have seen give decent translations is GPT-4. So, maybe someone could write code for Hydrus using GPT-4 API. You have to pay for GPT API though.

>>15864 Actually, I guess you could just copy them to the ChatGPT online program, and tell the Ai to translate them to English. Then you could replace them with manage tags. But that would be a pain.

>>15863 One problem with the tagger on loli though, it does well on anime / hentai, but not so great on realistic images, as it tends to have a lot of false positives on that.

hydrus removes leading and trailing spaces from tags, but not zero width spaces (​) so it leads to confusing duplicate tags sometimes. I think hydrus can safely trim those from the beginning and end of tags.

>>15862 oh idk i just want simple tags honestly i can't read the tags in Japanese and there's a million of them lol >>15863 how do you do that?

>>15868 Well, I use an old tagger that isnt being shared on github anymore, but I think this one is a child of it. https://github.com/Garbevoir/wd-e621-hydrus-tagger

>>15869 Ahh, I think this is the old one that I am using ( the ancestor of the one above)

>>15870 Forgot link https://github.com/Garbevoir/wd-e621-hydrus-tagger/tree/main

>>15871 The main thing to remember is, the lower you set the threshold, the more tags it will try to match to the pic. If your looking to tag loli that hasn't been tagged as such, set it low. It will tag it. Just experiment around with it to find out what you like. I use --threshold .10

>>15861 I guess nobody here knows. If I ever get to solving my freebsd problems and trying hydrus here I will report my findings here or somewhere devanon will see it. If I want this to get recorded where should I write it?

>>15873 >I guess nobody here knows. I guess you're right. You have to wait for devanon to show up and comment. Take into account the niche status of that OS and the stubbornness (aka fossilization) of its community and then you will understand why it hasn't many takers. If you are interested in a fringe OS in the style of TempleOS but with a modern look, then Essence might be your flavor. https://nakst.gitlab.io/essence https://www.youtube.com/watch?v=1PMf3FrFGD4

>>15875 I know that the popularity of *bsd systems lies mainly apple and playstation so no where near open source desktop oses. I use linux as my daily driver. I wanted to have some fun and experiment with a bsd ams experience the "more cohesive system" they boast. Currently stuck on mostly not booting after install XD.

>>15876 Have you gone to the BSD forums to ask about your boot issues. https://forums.freebsd.org

>>15875 >>Links to patreon and discord

>>15877 Not yet. If I can't troubleshoot it for myself I probably will. I haven't done that much yet.

>>15838 Got it, thank you for your work. Also, maybe it will be an improvement for fetching tags of already in db images, but currently it seems like they are fetched for each image individually (e.g. gelbooru gallery downloader). Gallery-dl uses their api like this: https://gelbooru.com/index.php?page=dapi&q=index&json=1&tags=1girl&pid=0&limit=100&s=post Which gives metadata on a whole lot of images at once in json format. If hydros downloaders could use it like this, it would make synching tags with boorus much faster and maybe less strenuous for the site. >>15842 Thanks, bit of a pain to run it manually like that, but I guess it's fine since it's not something that I would do often.

I'm having issues with the media viewer. Debian 12, Hydrus 584 built from source. If I play any video in the media viewer and I'm not using the native hydrus viewer for playback, the transitions for further browsing or zooming get messed up until I close and reopen the media viewer. Video playback and browsing and zooming all still work, but there are some jarring visual glitches when going from one file to another or zooming in or out. When browsing to another file, instead of just showing the next file, for a split second the current file will change position and snap its top left edge to the position of the next file's top left edge before showing the next file. The zooming is harder to describe but it seems as if after zooming, for a split second a section of the previous zoom level is overlaid over the new zoom section. The glitches only occur when I view webms/gifs in the media viewer with mpv or the Qt Media Player. If I use the native hydrus viewer for playback, I get no glitches. Anyone have any ideas? I'd like to be able to use mpv for playback.

>>15882 >Thanks, bit of a pain to run it manually like that, but I guess it's fine since it's not something that I would do often. Maybe i'm wrong, but i think you and >>15842 are talking about two different things. Aren't the options in the image that >>15842 uploaded show 'deleted tags' in the sense of Hydrus deleted tags? For example: - If you download a file with 10 tags from a booru, then delete 5 tags in Hydrus of that file, the activated option 'parsed tags overwrite previously deleted tags' means you will have all 10 tags after download/parsing again, when deactivated you will have 5 since you deleted 5. That is possible because Hydrus saves also deleted tags of files. You can see an option to show them on a file in the 'manage tags' dialog when clicking on the cog symbol. But you are talking about deleted tags from boorus on the website and not Hydrus, correct? >Suppose I download an image from booru and import it's tags, later tags get changed on booru, some deleted, some added. I found the way to get newly added tags, but is there any way to remove deleted from booru tags? Example: - A booru has a file with 10 tags. You download it and you have 10 tags in Hydrus too. Later the booru mods decide to delete 3 tags for whatever reason on that file on the booru. You want that those 3 tags get deleted on your Hydrus file as well, in short: 'updated'. If not then forget everything i said in this comment :P. I don't think there is a way to do that for your (the second) example. If there is, id like to know too. And what is YOUR way to add the extra ones that got added on the booru?

I had an excellent couple of weeks. The manage tag siblings and parents dialogs now load and operate quickly, even when the underlying service has hundreds of thousands of pairs. I also cleared a bunch of normal small work. The release should be as normal tomorrow.

>>15884 Yeah it's not possible in an automated way, that's why I suggested doing it manually, which you could do in bulk once in a long while. For that you will have to delete all (downloader) tags from your selection, then paste their booru urls into a url downloader, but you will need to check that option in the image or the downloader will skip tags that didn't change. So for example you have a file with 10 tags, 1 gets added, but 3 get deleted, so 8 total. If you simply redownload, you'll have 11 tags. If you clear your tags and redownload without the option, you'll end up with only 1 tag. With the option you should get the correct 8 tags.

>>15886 Thanks for the explanation.

https://www.youtube.com/watch?v=LREOmHLII70 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v585/Hydrus.Network.585.-.Linux.-.Executable.tar.zst I had a great couple of weeks getting the tag siblings and parents dialogs to load quickly. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html fast siblings and parents The PTR has been a painful success. It is great, and I am grateful for how it keeps growing, but every time we add ten thousand or a hundred thousand new things somewhere, it lags out some bit of UI where I never thought it would be a problem. Anyone who has tried to work with PTR siblings or parents knows what I am talking about--it can take five or ten seconds, every single time, to load the manage tag siblings/parents dialogs. The same is true for anyone who has programmatically imported siblings from a booru--adding 100,000 pairs can be neat, but editing them manually is then a constant frustration. So! I have rewritten both dialogs, and the long and short of it is they now load instantly. If they need to review some pairs to do logic (like 'hey, does this new pair the user wants to add conflict with the existing pair structure?'), it all happens in the background, usually so quickly you never notice it. We'll see how this actually works on IRL situations, but I do feel good about all this work. There was a lot to do, but it ultimately seemed to go well, enough that I had time for some bells and whistles. Beyond some optimisations and loop-detection fixes, there's a workflow change in that these two dialogs now have a 'stickier' workspace. The list of pairs has typically shown anything related to what you have waiting to be added, and I have now extended that to say 'and they now stay in view after you add'. Whenever you type in a tag, everything related to that tag is loaded up and stays in view while you work on other things. If you want to clear the workspace, you can just click a button to reset. I hope this makes it easier to edit and even merge larger sibling groups. This is all a big change, and I'm sure I've messed up somewhere. If you do siblings or parents a lot, give it all a go and let me know how it works out. The PTR really is huge, and some larger groups may still take a second or two to load--we'll see. other highlights Hitting escape now deselects any taglist! options->media viewer gets a new 'Do not allow mouse media drag-panning when the media has duration' checkbox. If you often misclick when scrubbing, try it out. I pared down the spammy 'added to x service 3 days ago' lines in the media viewer's top hover. It now pretty much just says 'imported, modified'. If you need to see archived time or something, note that the timestamps are still available on the normal media right-click menu, on the flyout submenu off the top row. next week I have lots of small things to be getting on with, so I'll just catch up on my normal queue.

>>15888 man thank you for the fast siblings and parents parents had always been a little slow for me but the sibling dialog took like 15 secs minimum for me and I just downloaded a bunch of images from a drawthread booru and was trying to sibling a bunch of monster types to species:* and it was unbelievably slow going and now I just finished thanks a ton hydev, this has been the biggest gamechanger for me in a while

>>15888 >other highlights >Hitting escape now deselects any taglist! >options->media viewer gets a new 'Do not allow mouse media drag-panning when the media has duration' checkbox Thank you so much for making those personal additions! I like to imagine a world where not only I do use those but many others too so you didn't spend time only for my personal gain :D I want to report two bugs: 1) I think you fixed that the animation scanbar didn't stayed 'grayish' after you pressed pause on media with duration AND the scanbar was set to hide when the mouse was away. Before it stayed white in that combination. BUT the scanbar stays grayish only for the first time: you press pause let's say in the center of the media -> then move the mouse down to the scanbar -> it appears grayish, which is correct -> you move the mouse cursor up so the scanbar goes away -> you move curso to scanbar -> and from now on it becomes and stays white and the scanbar nub disappears also. Additionally, i found that the scanbar color of animated .webp is aqua or cyan. Is that on purpose? Just tested only two though. Animated .webp attached (not sure if upload will work) 2) when you activate the 'no, hide it' checkbox to hide the scanbar and 'apply', the pixel size of the scanbar defaults to 5px after you deactivate the checkbox again. That means if you had set something you don't necessarily remember like 17px and activate the checkbox and apply, the next time you want to deactivate that checkbox it is set to 5px instead of 17. It doesn't remember it. You once did something to remember the 'how many tags to show in the childrens tab' option in the 'tags' options (on the very bottom), because it was defaulting to 20. Maybe you can do that here too? I actually just tested the 'threshold' option on the bottom of the media viewer settings too. If you activate the 'do not use' checkboxes and apply, then reopen the settings, it changes to 1 on every single one of them, which is not the default really. So the problem is, once you click those and apply, you probably can't go back since you forgot the real defaults. I hope there are not many more checkboxes that have that behavior in hydrus. Going back to defaults is kinda ok i think if it can't be remembered, but the threshold ones are clearly wrong. I did some video recording of my settings back then at least, so im kinda save.

For sites with expiring links, is it possible to make next search/gallery page download always wait until all queued files have finished downloading?

>>15888 Seems like the parent management page is taking ages to load with this new update for tags with lots of children, though only once, and then quickly thereafter. Is this intended behavior?

>>15884 Anon >>15886 explained it, as for adding extra tag, you just check force fetch page in tag importer options.

>>15894 I guessed so, thanks.

>>15888 Hydrus freezes when I add a new parent to a tag now. I can search for tags under the "set parents" section, but as soon as I click on one it slows down until it stops responding. It also freezes sometimes if I use the media viewer in fullscreen mode. Using the Windows version.

>>15892 Actually, it slows again as soon as I try to re-search a tag that recently had new relations added. If I had a slower computer it would probably freeze like >>15896 It takes anywhere from 10-30 seconds to load the relations for a tag that has around 5-30, and it can take several minutes to load the relations for a tag with 30-120. Interestingly, it only took a couple minutes to load something like my genre:action tag's relations, which are around 1,250. Did this update improve loading relations for tags with large numbers of relations, which probably massively benefits the PTR, at the cost of decreased efficiency when loading relations for tags that don't have that many relations?

>>15897 Seems to have started working quickly regardless. Strange. Might have needed to do a lot of preliminary work after switching to the new system?

>>15888 > > So! I have rewritten both dialogs, and the long and short of it is they now load instantly. That's really good > now have a 'stickier' workspace. The list of pairs has typically shown anything related to what you have waiting to be added, and I have now extended that to say 'and they now stay in view after you add'. Sometimes it was hard already to find the entries directly related to the entered tag. In "edit subscription query", the "tag import options" button name does not show if the options were modified.

The derpibooru downloader downloads descriptions without the links in them. :(

>>15898 And now it's fucked again. Just took 30+ seconds to load 2 pairs.

>>15902 I had some problems too searching for tags in the tag parents dialog for the PTR, without activated 'show all tags' checkbox. just the typing alone was slow, like a delay of 300ms or so between each character. activating the 'show all chains' checkbox kinda freezed it with the typical 'hydrus clieant 585 (no response' window that comes when stuff gets slow or crashes in Windows. at that time i didn't restart the pc or the client for some time, could have something to do with that, or not. after restarting the client it got fast again. other than this i didn't have slowdowns or crashes because of the sibling/parent dialogs, but i also didnt use them. I'll check from time to time how it behaves after having the client open for some time.

>>15903 > just the typing alone was slow, like a delay of 300ms or so between each character My search in the parent management window is instant so long as it's not loading pairs. If it is, I get the same delay as you. Are you sure it wasn't in the middle of loading pairs for a tag you had already entered?

>>15904 Im not sure how you mean that. It doesn't load tags for me really when i enter tags since now it is supposed to be fast. So there is no loading really. Except the 'show all tags' checkbox which crashed it once, but the slowdown was without it activated. Maybe the 'show whole chains' checkbox was activated tho. But i don't think it was during 'loading'. Can you give a little tutorial how to reproduce it what you say? What checkboxes and what tags if you don't mind?

Now I'm getting even more weird behavior. It said "loading", but unlike usual during pair loading, it allowed me to enter a new pair before it finished loading. Normally the button just doesn't work until the pairs are loaded. >>15905 I haven't touched any of the checkboxes, so this is all happening as I normally use the program. What I am talking about being slow is the loading of pairs, which is indicated at the top by the "wipe workspace" button. Hydrus is having issues loading the pairs related to tags that have been entered, not just typed, into the set children/siblings/parents boxes. During this delay, typing in tags and the tag search autocomplete in this window are slowed for me.

>>15906 Also, upon clearing that popup, it loaded the pairs, but simultaneously cleared the already entered tags so I had to enter them again.

>>15906 I see you have no PTR, so you can't give me a tag to reproduce a long loading. To search a tag with many tags, my mind came up with entering 'series:pokémon'. It has ca. 600 pairs, and with the 'show whole chains' checkbox activated it has ca. 16000 pairs. And both are almost instant (max half a second), that's why i can't really type in stuff while it's 'loading'. That's what the update is supposed to do. But the slowdown i had was weird behaviour that didn't occur yet after restarting the client, so let's see how that manifests in the future. I can't help you really. do you use an HDD or SSD? Windows or Linux? Hydev will most probably give you the answers to whats the problem on your side, today or tomorrow i think.

>>15908 SSD, Wangblows 10.

I just noticed some files have this (1) (+1) tag instead of only (1) like in pic related. This is a literally a non-issue, and it seems like it appeared on files pending deletion after a duplicate filter. What does it mean, though? Tags that were merged?

>>15841 >>15843 >>15845 >>15846 >>and more Yeah, sorry, I don't mean some clever talking Cortana is going to tag your images via neural splice SINGULARITY NOW BROS, I just mean stable-diffusion-like models are getting better and better and converting image-to-text and vice versa, and we are seeing it work already with the hydrus Client API and the danbooru/e621 models. All indications are that this tech will improve further for some years, and modern GPUs are probably going to have even more hardware acceleration and stuff for all this, so I expect hydrus to use it more too, and I would like to have more ways for the manage tags dialog to provide suggestions and perhaps have the client call external programs to auto-generate some tags on import etc... Then, if we can recognise regular nouns like 'skirt' or 'blonde hair' in an arbitrary image, be that anime only or real life too, then the PTR wouldn't really have to keep sharing those words so much. The same may be true for some characters or series names, but that is less generalisable and I imagine more prone to false negatives and positives. Any future hydrus plugins to this sort of tech will need careful filters to ensure we don't try to apply 'title' tags and other specific stuff to our model training or suggestions. Also, in terms of precision, we don't need 99.9%. If these programs can offer, let's say, 90% true positive, and if they offer an "I am 85% sure of this tag" metadata so we can filter out unreliable suggestions, then we are getting a huge productivity boost. If 90% of the 'skirt' tags you would add appear without you having to do anything, then that's a ten times multiplier on your human time, minus the problems caused by false positives. The only question, then, is where the threshold should be to keep those false positives low enough to be a worthwhile pain. Since these models appear to be getting better every few months, we know the ratios are only ever-more in our favour. There's also some more esoteric ideas like automatic archive/delete filtering (or some new pre-filtering workflow or filtering context layer like 'I think you will like this' we invent to prep your human work). This is stuff we can think about in some years if models become trivial to train and run. We'll see how it all shakes out. >>15849 >>15851 Thanks, something here seems off. I'll check what it is doing in this case; I think it might only be doing petition, and not rescind pend. >>15867 Thank you, I will check this out! >>15857 >>15860 >>15861 >>15873 >>15875 I am afraid I not at all a Linux expert, and when it comes to anything unusual like FreeBSD I really no next to nothing. If you have some explicit error messages, I can look at them, but the answer is probably going to be 'try running from source', and then, if that fails, it'll be looking through StackExchange for posts about getting OpenCV through pip on FreeBSD or whatever the exact problem is. You will be much more adept than I am at that. If you do learn anything, please do send it in. You can email me if you like, or just post here, perhaps with a pastebin if it is too long. A couple of anons have written whole guides that I have attached to the help here and there, in a similar way.

>>15882 >If hydros downloaders could use it like this, it would make synching tags with boorus much faster and maybe less strenuous for the site. Unfortunately, almost all the big booru engines supply their tags in an 'unnamespaced' way in their APIs. I guess internally, boorus store 'samus_aran' as 'hey this is a character tag', as per here https://donmai.moe/wiki_pages/samus_aran , where it is inherently green, and it will be grouped with other green tags in the normal html post view, but that green namespace is not generally explicitly spoken when the tag is referred to technically in URLs or the API. Hydrus could have tag siblings for every booru 'samus_aran' to 'character:samus aran', and perhaps we will in future import all this data to get a nice mapping, but we don't right now, so in order to get nice character, series, creator tags, we need to grab the html. If the APIs started separating the tags based on their 'artist-tag', 'character-tag', and similar, then we would be able to do this, but I think this is just an unfortunate difference of hydrus and booru design--I declare namespace explicitly; they do not. >>15883 Unfortunately the stock answer on this is 'I got mpv to work through duct tape and spit, so if you have anything unusual as OS or Window Manager, I can't guarantee anything'. My media canvas code is also pretty shoddy in the way it lays some things out, which is probably the cause of the resize/position flicker you see. On OSes happier with Qt, that stuff tends to get folded into one frame, so you never see it, but if the OS forces a repaint on every update or whatever, you then get the flicker. I don't have a nice answer for you right now, especially since you are running from source already, but I am planning to add some DEBUG checkboxes for mpv in the nearish future that will change how I load and swap out videos. Mostly I will be going in the direction of 'use a new mpv video for every video', rather than the current recycling tech that I do for stability purposes, which may not help your situation much, but we'll see. I'll also be working more on cleaner layout code in my media canvas, which I hope will improve you. You might like to try using a slightly older or newer Qt version. If you rebuild your venv and select the (a)dvanced setup, you'll have several Qt versions to choose from. I expect your flicker behaviour will change, so perhaps one is better than another? If you do find a good one, please let me know and I'll update the help etc.. There may also be a magic environment variable you can run, like QT_QPA_PLATFORM=xcb, that will launch hydrus under a different Window Manager or whatever, but I'm afraid that sort of stuff is beyond my expertise so I can't talk too much about it. >>15889 >>15892 >>15896 >>15897 >>15898 >>15902 >>15903 >>15904 >>15905 Thanks. I am glad it is working in simple cases, but I was afraid we'd run into some of this. I encountered a couple of the random slowdown events when I was doing final testing on the PTR scale. For a couple of years now I've had odd reports from one or two users that doing x or y sibling/parent fetch was inexplicably taking 20 seconds or more on the PTR. I've never been able to figure it out, and some of those users had it because of hard drive problems, but there is some search index weirdness at times. The underlying database structure isn't too big (couple hundred thousand rows at most), but it can get complicated, and it looks like my new fast-search system for siblings and parents dialogs is running into the same problem, and in that annoying 'sometimes it just takes ages what the hell' way. If you get this, please try doing a few queries with help->debug->profiling on, and send me the profile. I will be doing this myself and we'll see what the hell is actually going on here. Selecting 100 rows out of 200,000 should be like 50ms at most, but if it is doing it backwards and reading 200,000 100 times, that could really add up. Sorry for the trouble, I'll keep pushing!

I got a about a 25-30 second delay here when trying to load pairs for a tag with only 7 pairs.

>>15914 Also, it was the last thing I did before closing the parent management window and then closing Hydrus, if that helps you find it in the log.

>>15890 Thank you for these reports. I will see why the scanbar is not getting the right redraw calls when it unhides. It turning cyan is intended, but it is odd--that's my native viewer, which animated webp uses (mpv doesn't support animated webp yet), and the teal area is a little visual indicator of the frame buffer my internal renderer has pre-drawn. My native renderer is a weird debug thing, I am pretty embarassed about it, hahaha, and it has a few weird old quirks like this. I expect I'll steamroll over it one day with an overhaul that brings it all up to newer standards and more Qt-friendly code. (I wrote all this shit back in wx, and it is secretly a really ugly software renderer that is eating bmps piped over from ffmpeg and throwing them on screen). And yeah, thanks for the note about the bad 'noneable' controls. Some of this stuff I can't fix nicely (with a memory of what you had set before) since I store the same 'number or none' in the same options cell, but I can at least make all those controls default to a nicer number than the '1' you are seeing. I just need to go through them, or programatically figure out a nicer initialisation/setting routine. Another requested thing would be 'reset this page to defaults', which I agree I'd like but it'll take some thought. I'll work on it. >>15891 Not yet. I know exactly what you are talking about though. Unfortunately my downloader doesn't have quite the tech to support this, and there will be an awkward problem to get around in how subscriptions target a downloader like this, but I hope to have some options around this in a future iteration of the downloader engine. I am sorry to say it make have to wait for a larger overhaul of the whole system, since the current behaviour is pretty core to the whole thing. We'll see if I can tuck this sort of thing into a related 'retry later' tech I want to add to handle some error states. >>15900 >In "edit subscription query", the "tag import options" button name does not show if the options were modified. Thanks, I will check it out. >>15901 I will check it out. I don't know how this thing works, but if it is just pulling the visible 'text' of the html, and the URL you want is in <a href="xxx">, it may be tricky to get that in a neat way. The hydrus note tech is only plaintext for now, so no proper rich text or links or anything yet. >>15906 Thank you, it looks like I missed something in the pair-loading-queue logic too, causing that error popup. I'll look into it, and sorry again for the trouble. >>15911 If you go into 'manage tags', I suspect you'll see those tags in one service, probably your local 'my tags', and also pending to the PTR. That taglist you are looking at is probably in 'all known tags', which merges all services and can cause some odd count summaries like that when two services agree or disagree on a tag. When you commit the tags, the (1) (+1) should merge to just (1). If you don't have the tag in multiple services, let me know, because that could be a miscount. >>15914 >>15915 Perfect, thank you. I see the slow parts, and will investigate this this week. "2024-08-10 15:32:18: Profiling db job: read tag_parents" if you want to see yourself. This shit is supposed to take a few milliseconds, and it reliably is instant on a smaller test service, but there's a couple of 13 second delays for you in one method. I will be putting time into this this week and am determined, if I can, to get it working correctly.

>>15916 >"2024-08-10 15:32:18: Profiling db job: read tag_parents" if you want to see yourself. <Take a look <cumtime is a stat Lmao.

>>15916 >tag in multiple services >pending to the PTR You're correct in both accounts! By clearing the "pending tags", that (+1) disappeared. Thank you~

I had added to 4chan parser a "downloadable/pursuable url" content parser that tries to get urls from the comment (text of the post), and just found that a picture gets the urls mentioned in any post it was attached to assigned to it as urls. Aren't those urls for different files Hydrus must download?

>>15919 And I would like those urls not to get the *tags* the picture is supposed to get.

>>15908 (me) >>15914 I also found a very reliable way to reproduce the 'loading...' right next to the 'wipe workspace' button. I ask all you guys to try it too (at your own risk, don't forget backup) and mention here what happens on your end. So you do the following: 1. open 'tags' -> 'manage tags parents...' 2. PTR tab (if you don't use the PTR then your out of luck here or lucky, whichever you prefer :D 3. no need to activate any checkbox 4. type 'dragon ball' and chose (it is the first option so actually no need to 'chose') the namespaced 'series:dragon ball (ca 216.000)' , enter, it should give 74 pairs at the time of writing this comment 5. type 'dragon ball and chose (the second option for me) the UNnamespaced 'dragon ball (ca 8.400) -> series: dragon ball', enter it will start 'loading...' for over a minute at least and while it does that the client will be unresponsive or very slow. it is important you have to enter the tags from 4. and 5. in exactly that order, otherwise 5. will give 0 pairs and 4. will give you 74 pairs without loading/crashing/slowdown. if for some reason it works fast for you, close the 'manage tag parents' dialog and open it again. with two tries it is guaranteed for me. with only one try after starting the client it works almost guaranteed (i remember it worked at least once but i am all over the place now, perhaps it is even guaranteed on first try) Good luck fixing this Hydev.

What is stored in Hydrus's backup file? I want to move my installation but I also want to do a "fresh install" and then get my backup. I'm just worried about losing something

Any way to set default tag sorting, grouping types? It's annoying having to set it every time I open an image.

>>15922 https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up Read the 'backing up' chapter and 'clean install' bit higher up, which you probably want. Also, im not sure what you mean with Hyrus backup file. There is not one file where everything is backed up within. You have to backup your whole Hydrus folder or at least the db (database) folder, which is the most important for your media and tags. You want to do a clean install with your database and files saved but settings etc put to default? Then you have to backup the db folder and do a new/clean install and put it back in. Read the link to be sure. >>15924 file -> options -> sort/collect on the left -> read the options here, but what you want is probably the third from top 'default tag sorting in the media viewer' I would advise anyone new to Hyrus to read the whole website (or at least the 'getting started' part) and go through all the 'file -> options' at least once. will take some hours but you will find out alot of helpful stuff.

I right-clicked a tag, opened the parents window for it, entered two tags and click "add". > Hey, somehow the "Enter some Pairs" routine was called before the related underlying pairs' groups were loaded. This should not happen! Please tell hydev about this.

Can you make so that 'dateparser' would stop respecting locale when converting timestamps automatically? Like a toggle in "edit conversion" window or in settings to set it globally? I have situation like this "7/18/2023 8:32:00AM" In my locale, for some reason client freaks out and says 'nope, can't do it with this date' But, if i would boot up client (using setup from source) with 'en_US' locale enforced with setlocale function in client boot script, it will happily chew it and spits out correct timestamp. I had to remove last 2 characters just to make my homebrew parser thing to work, and i don't want to do that setlocale ugly hack.

is there a way to retrieve the hash(es?) for a file my client knows about but doesn't have?

>>15912 Actually, the autotagger I use on my downloaded booru files works pretty good! I'd give it 97% correct, as theirs a few exotic items like "spreader-bar" that it occasionally makes mistakes on. But everything else is fine. It even tags things like "hair bow". It seems to be less reliable with 3D / realistic stuff cause it's been trained on anime drawings. So, if your tagging your Playboy collection, it might mess up on a few rarer tags. But it's still pretty good, and I use it on all my collection.

I think it'd be cool if there was a way to zoom in on the file history graph. it's a bit hard to see some of the differences between the lines when the entire history of your database if always visible. Just seeing the last year would be helpful.

>>15928 I know a way kinda yes, but with the exception of files that have no deletion record AND never had any tags. From those you can't get any hash afaik. For the others the SHA256 (i think) only. First you have to activate the advanced mode: help -> advanced mode (so the checkmark is there). Now you can take a look at other file locations where the 'my files' button in the search pane is. You need to check two file locations now, when pressing on the 'my files' button. 1. Change file location to 'deleted from anywhere'*. Now you see files that you deleted from any location, but they contain also ones deleted from a location but still in other places and still in your client. I think you only want the files that are not in your client anymore which are the blurry thumbnails with the red thumbnail background. Now use the system:fileservice seach predicate. Search for 'is NOT currently in all local files'. This will give you only deleted files that arent in your hydrus anymore -> the red thumbnails. Now you can select all those, right click -> open -> in a new page. This will send them to a new page with the search predicate 'system:hash is X' visible in the search box on the left. Shift+double left click on the 'system:hash is X' will open the edit window where you can copy all the hashes nicely. *This file location will give you files that you have deleted WITH a deletion record saved. Also those without a tag, which is not the case for 2. (see below) 2. Change file location to 'all known files with tags'**. To make only the ones visible that aren't in your client anymore, use the system:fileservice predicate again and do the same as above. You can just open the page from 1. again and change the file location to 'all known files with tags' there. The correct search is already typed in. Then you should have only red thumbnails with tags that you send to a new page too and there you can check and copy the hashes like described in 1. **This file location will give you files that you deleted WITH or WITHOUT a deletion record saved, but at least had/have one tag. Once you delete all the tags from a file from there, you will not find the file anymore (at least within the client) when you deleted the deletion record also. If you saved the deletion record, it is still in 1. ____ Note that theoretically you could only need one of those two file locations, but depending on what you did (saved deletion record or not, tagged files or not) you need both probably. This way you can also retrieve hashes from the PTR, when chosing the 'all known files with tags' file location + PTR tag domain. Search for files, select, send them to new page and get the hashes from the 'system:hash is X' in the search box, exactly as described above. Not sure if there is a better way for all of what i described.

Dev, a few years back, I asked you if Hydrus Network could remove all traces of a file ever being in the database (hashes, URLs, etc.). You said that it currently couldn't, but it was something you planned on eventually. I haven't kept up with development, so did this ever happen?

edit parser - example urls - edit - Esc without editing - cancel It asks if I want to save the changes.

I'm trying to download stuff from reddit, but I'm only able to download images - videos returns an error. What should I do?

>>15933 https://8chan.moe/t/res/14270.html#15536 Not yet, but still planned. Read the first answer from Hydev and the corresponding post he is answering to.

>>15913 >You might like to try using a slightly older or newer Qt version. If you rebuild your venv and select the (a)dvanced setup, you'll have several Qt versions to choose from. I expect your flicker behaviour will change, so perhaps one is better than another? If you do find a good one, please let me know and I'll update the help Thanks, but it seems like different Qt versions don't affect the flicker for me. I have Python 3.11.2 and tried rebuilding with the different available Qt 6.4.x and Qt 6.5.x versions, but there was no change. Tried the environmental variable too. I remembered I had no problems on Windows, so I did some investigation. On my Windows machine (i7-2600, 8GB RAM, no graphics card) I was running Hydrus v578. I wondered if it was the version so I tried v584 on Windows and noticed the same flicker I get on Debian: with mpv, after viewing any video in the media viewer, any subsequent zooming has a split-second resizing/repositioning flicker until closing and reopening the media viewer. The difference is on Windows I don't get any flicker while navigating between items with the media viewer, only when zooming. After trying more builds, it looks like v579 was the first version on Windows where I get the zoom flicker. So I tried v578 on Debian building from source, and... both flickers are still there, no change. Oh well. It's not a huge problem for me. The browsing flicker isn't that bad. The zoom flicker is bad but I don't zoom that often and I can always just close and reopen the media viewer to remove the flicker anyway. I just started using Linux a few months ago so there's probably a lot more I could try on my end that I have no idea about.

>>15935 >leddit videos https://flathub.org/apps/com.github.unrud.VideoDownloader It works flawless.

I had a great week. I fixed some bugs, finished some advanced multiple local file service features for the Client API, and got siblings and parents loading faster, particularly for the new dialogs. The release should be as normal tomorrow.

Is there any way to find related files by tags? Maybe by searching for files with some of the tags suggested as related, but including the less popular tags, too. I imagine it could take a long time, but not as long as the siblings window was taking.

Any tips on increasing the performance? I get freezes for 5-10 seconds every few minutes. My DB size is 98 GB, media files are 210 GB. All on fairly fast NVMe SSD. Got 64 GB RAM and 7800x3D. Shutdown maintenance was recently performed. Picrelated are my current speed/memory settings.

https://www.youtube.com/watch?v=CVAaLlOUD00 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v586a/Hydrus.Network.586a.-.Linux.-.Executable.tar.zst Hey, I did a hotfix to fix a stupid bug when moving from videos to images. If you got the release within twenty minutes of this post going live, get the updated v586a above! I had a great week getting siblings and parents lookups running faster and finishing some long-planned Client API work. The update may take a minute this week! Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I am happy with the new siblings and parents dialogs, but unfortunately the fetch jobs were frequently running super slow on the PTR. This has been a long time problem in other sibling/parent places, and I could never really figure it out. This exposed the problem better and I simply put a bunch of time into the database sibling/parent storage structure and search code this week, and I think/hope I have fixed the worst of it. I also fixed the crazy-long lag spikes we were seeing, which was, unfortunately, just me being stupid last week. If you sync with the PTR (or not!) and have had slow sibling/parent lookups (including in places like the tag autocomplete results list), let me know how it goes! If you have the media scanbar set to hide completely when the mouse is not over it, I think I fixed the issue where it would come up blank if the media was paused while the scanbar was hidden! The options widgets that are an editable number with a checkbox beside saying 'no limit' are now initialised with a nicer default number when they start with 'no limit' checked. Previously, this stuff was all initialising to 1 every time, which wasn't always helpful if you actually wanted to go edit it. I'm pretty sure all the 'noneable' integer widgets in the options dialog now soft-initialise to the actual defaults those numbers are on a fresh client. If you use multiple local file services, then when you middle-click a tag in the media viewer, the new search page now correctly retains the original file domain of the media viewer. Although I use multiple local file services myself on my IRL client, I do not browse around in them all that much, so let me know where else this sort of stuff defaults to 'all local files' or 'all my files'. I have removed a hard limit that said 'don't run an import folder for more than an hour'. If you have a mega import folder with hundreds of thousands of files, let's see how it goes. If you have done Client API or tag migration work and ended up with some bizarre tags that are both pending (+1) and petitioned (-1) to the same service, check the changelog this week! client api Thanks to a user, the Client API call that renders images can now output in jpeg and webp, can change the quality of the output, and will render to a target resolution! I also think I finally finished off the first full version of 'multiple local file services' Client API support. You can now set a custom import destination for the 'add file' and 'add URL' commands, and you can now copy files from one local file service to another using a new 'migrate_files' call. next week I've been working on several things recently that can populate a multi-column list with a hundred thousand or more rows, and it has reminded me that my core list code relies on an old hack in it that makes initialising and sorting such big lists super laggy. I have researched how to improve it and hope to do so next week! Unfortunately, I do feel myself going down with something, so it might be delayed.

>>15938 Thanks, king. Some people only post their stuff there, so cut me some slack. >>15938

>>15935 >>15938 I haven't tried it in a while, but when I last used it, yt-dlp also worked well for reddit vids

>>15945 oh wait, that's because the flathub application is a frontend for yt-dlp. makes sense

>>15945 >>15946 Oh shoot, it DOES work too! Nice, yt-dlp is so versatile.

>>15947 > it DOES work too!

>>15946 >the flathub application Incorrect, FlatHub is a website hosting programs in "Flatpack" format, like the "Store" in Mac. Flatpack allows to ship all software dependencies in one package that in theory are able to run in ALL Linux distros. Kinda AppImage files but centralized in one website.

1a. Is it possible to install several Hydrus clients with the .exe at the same time and run them simultaneously? Like install to folders A, B and C (all on C:)? I don't wanna try it and risk something get overwritten, but maybe it should be possible since Hydrus is completely portable? I have a main .exe install and just tried it with extracting the ExtractOnly.zip version to the Download folder, which works and runs at the same time. Im fine with them being in compeletely different folders and each with their own database. But not sure about several .exes, and if there would be several start menu folders etc. 1b. Can the .exe and the .zip versions update each other? Means if i have an .exe install, can i extract the .zip over it and vice versa? For example if i have the .zip version on an external hdd/sdd (E:) and i chose to move it to my main disc on C:, can i then update this version with the .exe and if so, would it have any implications for the update process? For instance, a 'clean install' was necessary for some .zip versions in the past. If i would start updating the .exe over the .zip , is that already a clean install like the guide is suggesting for the .exe version? 2. What is the best way to mass rename a tag. I assume right now the way is to search for the tag, then select all files, then 'manage tags' and add the new tag and delete the old?

>>15954 >I assume right now the way is to search for the tag, then select all files, then 'manage tags' and add the new tag and delete the old? Yup. That's the way.

>>15953 it's not incorrect. it's hosted on flathub, so it's a flathub application. I know what flatpak (not flatpack) is

>>15954 >mass rename a tag the majority of the time, it's better to make a new sibling. in fact I can't really think of any situation where I'd specifically want a one-time "rename" unless the tag is actually just wrong outright

>>15954 Not sure if you can have multiple installs using the installer, but for extract only you can. Also you can use multiple separate databases with a single install. Just make a new shortcut and add -d="path to different db folder" at the end of the target path.

is it possible to make is so that files that are downloaded and have a known url of a certain type get a specific tag added to them? I feel like there's a way to do this but I can't think of it.

>>15960 Network > downloaders > manage default import options... then you double click a url class and add an 'additional tag' to a tag service you need. Is that what you want?

>>15961 kind of, except that I need that for a file url class, not a post url class. Since those are missing, I'm guessing this isn't possible.

>>15958 underscored tags.

>>15963 that would definitely be a case where you should just make a sibling relationship

>>15963 >>15954 Definitely what >>15964 says. This is precisely what siblings are for. If you set a sibling like "some_random_tag" --> "some random tag" then you will never have to correct underscores for that tag again. Mass deleting "some_random_tag" and adding "some random tag" like you describe would have to be done periodically to keep things tidy. With siblings it's one and done forever.

>>15943 This seems to have fixed those lag spikes for my local sibling tags. Great work.

>>15965 >Mass deleting "some_random_tag" and adding "some random tag" like you describe would have to be done periodically to keep things tidy. With siblings it's one and done forever. IMHO, keeping things tidy is the proper way to do it. Siblings are a Mickey Mouse way for hiding wrong, not wanted, and deprecated tags.

>>15968 >Siblings are a Mickey Mouse way for hiding wrong, not wanted, and deprecated tags. If you don't use siblings for that, then what are siblings for?

>>15965 yes, but then you will have both underscores AND non-underscores, easily doubling the amount of tags you have underscores are a very simple case for automatically and permanently converting a set of tags based on sibling relationships, there's no reason not to if you can

>>15969 Well, when I begun using them was for what I really thought were for, to center (ping point) on a tag that encompass a general idea. For example: - ideal tag: npc - replacing tags: homo, libtard, covidian, sheeple, retard, low iq, commie, brainwashed, idiot, collaborator, automaton, parasite, degenerate, golem, adversarial, ... But soon I found out that all those secondaries tags were out of my sight and I wanted them back for a more precise search. So, I dropped Siblings usage for good to never touch it again.

Pasting a tag generated from 'system:file service' doesn't seem to work. For example I paste in 'system:is currently in my files' and hit enter, but it does nothing (the tag won't be added). Not sure if the functionality was added, but I remember you adding support for a lot of system predicated before, so maybe it stopped working.

>>15969 Having searchable variants. Supposedly the correct danbooru tag for tutorials is 'how to', so adding a sibling 'tutorial' to that makes me able to just search 'tutorial' if I ever forget that the proper tag is 'how to'. Anyway >>15954 I think the cleanest way of doing it is editing the database itself. You simply replace all underscores with spaces in the table that has all the tags. That way you won't create duplicate entries (the tags with underscores would still stay in the db even if no file has them) and you don't need to create siblings. Though if you regularly pull bad tags with underscores from somewhere, then siblings would probably be the better option. If it's just a one time user error, then you could try the db approach. >>15975 That's what parents are for.

>>15975 yeah, that's what parents are for. a tag implies a parent VERSUS a sibling replaces a tag parents are for every instance of X automatically also tagging with Y, whereas siblings assume you meant Z when you tagged X

>>15975 Use parents, npc. :^)

>>15943 After I change parents, a db lock delays things in the parents window. The tag on the left takes time to appear, the add button takes time to activate, and autocompletion takes time to start. I am not sure it saves any time.

>>15981 I had that issue at first on v585 as well with both parents and siblings. Some heavy database magic basically freezes everything for a few minutes. After 2-3 times of that though it finished whatever it needed doing and both the parent and sibling dialogues are basically instant now. If you're running a build older than v585 then it will unfortunately always be like that. Parents and siblings were massively optimized in v585.

How do I force Hydrus to reimport files from a booru? I've recently split out a new downloader tag service for a specific site and it won't import the tags into that specific service because it detects those files as already in DB.

>>15983 Immediate update: Reimporting new tags works with Gallery, but not URL Import. Inconsistent behavior, URL Import should also check for new tags, or at least be configurable to do so (unless it is and I'm missing it)

>>15983 A manual way to do it would be to export the files with tags as a sidecar. Then re-import them with the sidecar tags to be sent to your new tag service. It will say everything is already in the database, but it should still add the tags to the service.There might be another way to do it, but that will work as long as you're okay with the tags being in both the old and new services. I do sometimes wish there was a right-click option in the file log to force re-importing.

>>15983 Migrate the tags you need from service to service using the tag migration tool in 'tags > migrate tags...' or you can find it in the tag manager under the cog icon (this works on the files you're editing). If you really need to redownload, then go to 'network > downloaders > manage default import options...', open the url class of a website you want to download from, flip the drop down to custom and check the two force fetch checkboxes. Also check the tag service checkbox you want to import your tags to and uncheck any you don't want.

v586, win32, source AttributeError 'NoneType' object has no attribute 'GetAnimationBarStatus' Traceback (most recent call last): File "D:\hydrus\hydrus\hydrus\client\gui\ClientGUI.py", line 8293, in REPEATINGUIUpdate window.TIMERUIUpdate() File "D:\hydrus\hydrus\hydrus\client\gui\canvas\ClientGUICanvasMedia.py", line 2652, in TIMERUIUpdate self._animation_bar.setGubbinsVisible( False ) File "D:\hydrus\hydrus\hydrus\client\gui\canvas\ClientGUICanvasMedia.py", line 1230, in setGubbinsVisible self._DoAnimationStatusUpdate() File "D:\hydrus\hydrus\hydrus\client\gui\canvas\ClientGUICanvasMedia.py", line 911, in _DoAnimationStatusUpdate current_animation_bar_status = self._media_window.GetAnimationBarStatus() AttributeError: 'NoneType' object has no attribute 'GetAnimationBarStatus'

The 'hydev is stupid' hits keep coming this week as it seems I broke tags->manage tag display and search due to another dumb typo. It is fixed for source users now and everyone in v587, sorry for the trouble! >>15917 Btw this turned out in large part to be a mis-profile. Although the db search was running slow in certain situations, the big 30-600 second delays we were seeing here were due to a super stupid busy-wait I accidentally had in the asynchronous caller, and since the tech by which that tech worked was a low-level C++ thing (afaik), it was holding up the CPU in a way the profile wasn't catching and recognising was actually work occurring in another thread. This was probably combined with the thing actually being profiled was itself also out of the python GIL and in SQLite's C++ dll land itself. So, it wasn't so much a slow db search as much as a choked CPU core. Running this through PyCharm's debugger actually froze both the IDE and python's processes in a way that it was impossible to kill them via Task Manager or command line, and then it froze the whole computer! An interesting lesson learned--don't be casually foolish with thread.Event. >>15919 >>15920 Interesting. 'downloadable/pursuable url' generally means 'go get this thing', but I guess if the import object already has a URL, it becomes 'associate these things', like a source url. I expect this is an accidental inheritance. I will investigate the logic here, but your situation is a little unusual, so I cannot promise a clean solution right now. It might be that even after I fix it the child object still gets the tags and other soft metadata from the parent import object that spawned it, so my ultimate advice might also be that you need to make a whole separate subsidiary page parser or something here so the import objects that are getting the tags are in a whole separate thing to the comment-URLs you are parsing. As a general disclaimer, as I am sure you know now: my downloader is not great at pulling URLs from, like, kemono post comments. It was designed for boorus, so clever stuff like this will trip it up. I'll see what I can do though! >>15921 Thank you! Should be fixed now, let me know if it gives any more trouble. >>15922 It is just a copy of your four database files and the client_files structure. Nothing too clever. Check the backup help as >>15925 says. As long as you have those db files and your client_files stored somewhere, you can recover. If you are moving your install, check this one too: https://hydrusnetwork.github.io/hydrus/database_migration.html Have a poke around the 'db' folder and see which part is which. You can break anything if you just have a look, and let me know if you have any questions about anything else.

>>15926 Thank you for this report. I think I have this fixed in v586 now, but there may still be a hole in the logic somewhere. Let me know how you get on! >>15927 Do you happen to have a copy of that error traceback? Should be in your log, if it made a popup in the main gui at any point. Normally dateparser is rock solid, so it is interesting it can't figure this out. dateparser is a super simple library with basically only one method. I just did a test in console here, and it looks like that main parse call will take 'locales = None'. I can add an option for this, and (if you know how to get into your venv of your source install) can you check that it works for you? - open a terminal to your install dir - source venv/bin/activate (or "CALL venv\Scripts\activate.bat" in Windows cmd) to activate the venv - type 'python' and hit enter to open the python terminal, then do: - import dateparser - dateparser.parse( '7/18/2023 8:32:00AM', locales = None ) - exit() Does that work ok for you? >>15931 Yeah I completely agree. I keep thinking I should add a date range and stuff to it. It is totally possible, I just need to get around to it. >>15932 Perfect answer. >>15933 >>15936 I am closer to this point, and I was recently able to add some new tag-scanning tech to the client database and things did not explode, so I am feeling more confident about finally flipping the lever on this 'scan the whole database for no-longer-used master records' system. Although I've felt pretty bad about my work, this has been a good year for cleaning bad old code and old systems. We are getting there, but there is still more to do. >>15934 Thanks, I am sorry, I know how jarring/annoying this is. I don't know for sure what is causing it but I will make sure I drill down and figure it out. I have some ideas. >>15937 Thanks for the update, and sorry for the frustration. Although Qt can be tempestuous with its updates, it has been getting much better on this stuff in recent years. I'll clean things on my end too; please keep me updated on how things go.

>>15989 >I am closer to this point, and I was recently able to add some new tag-scanning tech to the client database and things did not explode, so I am feeling more confident about finally flipping the lever on this 'scan the whole database for no-longer-used master records' system. This is good to hear. Thanks for all your work, Dev.

>>15940 I quite like this idea, and I've thought of some sort of 'soft/fuzzy' search for a while, but I don't know a huge amount about how those sorts of recommendation algorithms work. The 'related' tag suggestions system in 'manage tags' is neat (turn it on in options->tag suggestions if you don't see it), so I wonder if we could do something similar. Like if I made a catch-all tag predicate that held several tags, a bit like an OR tag, but it said 'weight all these tags according to namespace and then find any files that match any of this taglist with total search-match weight > x', that might do the job here. >>15941 Sorry for the trouble--that is not normal! Please hit up help->debug->profiling and either pastebin the result here in the thread or send the profile to me me via email or whatever. You might also like to try pausing tags->sibling/parent sync and database->file maintenance and database->db maintenance->(deferred stuff) to see if things suddenly pop back to nice. Try pausing the work in 'normal time' first. That options page is good for improving the speed of the media viewer, and the general rule is you can push the numbers up a bit to improve performance it you are looking at giganto pngs and such. If you are getting slowdown doing other stuff though, it is probably my background maintenance code being super rude on your PTR store or similar. >>15954 >>15959 Yeah, check these two pages: https://hydrusnetwork.github.io/hydrus/database_migration.html https://hydrusnetwork.github.io/hydrus/launch_arguments.html#-d_db_dir_--db_dir_db_dir You can run multiple databases off the same install, be that the exe installer or the zip. You just launch the exe with that launch arg pointed to a new place and you are good. Check help->about once you launch to make sure all the paths are as expected. You can have separate installs/extracts pointing to different locations if you need to, but it'll just add complexity unless you need to run different versions at once for whatever clever dev reason. Only thing you must not do is run one database with multiple installs at the same time! This shouldn't work on a local machine, but if you fuck around and set a db_dir that is on a network location (i.e. on another computer), then my 'hey this database is already in use mate' checks will not work and you'll run into Database Locking ClownTown, doubly so because you are over network I/O. >>15962 Yeah I think this is not possible yet. File URLs aren't really clever enough to talk to the tag import system, but I'll make a note to look at this sometime. If you are doing this via a subscription or a permanent watcher page or something, I recommend just setting some tag import options with forced 'additional tags' there. If you are really hard up for a solution, you could make a search for [ 'system:url: has url (file url class)', '-your_tag' ], and then load that up once a month and go ctrl+a->F3->add 'your_tag'. >>15976 Ah, thanks, I am not sure if I ever got to those when I was doing this work. I know I did not get 100% coverage, and particularly on awkward more human/english predicate texts. I will check it out, and I'll probably relabel all these tags to something more happily parseable like 'system:file service: is currently in my files'.

>>15978 >I think the cleanest way of doing it is editing the database itself. You simply replace all underscores with spaces in the table that has all the tags. Unfortunately, the database is more complicated these days, and you can't do just the one table any more. There are also issues with resolving/meerging conflicts, if you have both 'the_tag' and 'the tag' in the tags master table. Broadly speaking, I strongly do not recommend manually editing the database on an IRL database. If you really want to do this, note that I have 'tags' and 'fts' sub-tables in client.caches.db that replicate the tag text data in the 'subtags' master table, so you would have to edit those too. And the 'local tags cache', now I think of it, and perhaps some other little corner somewhere. If you need to merge tag definitions, then you'd have to update all the autocomplete count caches, which is impossible to do with simple math since the display context applies and merged sibling data and it is simply too complicated to do in a few lines of SQLite or python or whatever. If you really got into this, or were doing it programmatically, I think I'd say, "Yes, you can directly edit the subtags table in client.master.db, but once done you need to run x, y, z database regen routines to let hydrus recalc cache stuff using proper code". It would probably be the mappings cache, tag text search cache, and local tags cache. Easier and better to just use siblings, even if you have to bodge part of the solution with the Client API perhaps. Ultimately, I think I should probably write a 'sibling-replace all underscored tags to their space-having variants' checkbox, and/or make that a 'hard-replace' option. After the success of the PTR janitors' new 'purge tags' system, I'll be bringing that to all users for local services and extending it with our first dedicated 'hard-replace' tech. It should all work on very large lists of tags, PTR-scale operations, so this should become real in the mid-term future and not so difficult for me to add a 'do all underscore shit' mode to it. >>15981 >>15982 This v586 is supposed to eliminate the super long fetch delays we saw in v585. If you are still getting delays in v586, please try using the dialog with help->debug->profiling on, just as we did the week earlier as I worked on v586, and we'll do round two. If you are still on v585, please update--I made some bad decisions in that first draft that are now fixed. >>15987 Hey, I am sorry for the trouble. Did you happen to get the release within about twenty minutes of me making the post? I screwed up one damn line and it slipped through testing. The new links on >>15943 now point to a v586a hotfix that should fix that bug, so just redownload and install and I hope you'll be fixed. Let me know if you still have trouble!

>>15993 >Yeah, check these two pages Thanks for answering, also all other people that answered! Hydev, could you answer 1b too please, would be interesting to know. >>15994 >I strongly do not recommend manually editing the database on an IRL database Editing the databases isn't something i plan to do. I'm scared :o >I'll be bringing that to all users for local services and extending it with our first dedicated 'hard-replace' tech That's cool. That will have tech that can replace namespaces too right? like replacing the namespace 'filename:08951abkoe53872kdfia84' to 'title:08951abkoe53872kdfia84'. this is something you can't really mass rename/replace right now afaik. looking forward to it.

>>15978 >>15979 >>15980 >parents Yeah, it looks like the right tool, however, devanon mentioned a few times the issue of circular dependencies and that Hydrus' logic is not so good at it. For example it works well the following, as every children is specific to its parent: - Parent: show:star trek - Children: ---> character:captain kirk ---> device:tricorder ---> species:klingon But what about if things get a bit messy with children with more than one parent and children being also parents in many other categories? Circular dependencies might pop up, then the question would be, can Hydrus manage that, if not, could the DB get damaged because of logic's inconsistency? So, I'm really hesitant of using parents beyond a couple of children deep and I know for sure that my chain of children can go way beyond 10 layers deep.

>>15998 >But what about if things get a bit messy with children with more than one parent and children being also parents in many other categories? Children can have multiple parents just fine. For example 'character:gawr gura' can have 'series:hololive' and 'series:hololive english' as parents, it will simply add both tags when you add gawr gura. Or you could do a chain like 'character:gawr gura' > 'series:hololive english' > 'series:hololive', which is probably better as hololive english should also automatically add hololive, and if you add gawr gura, it will add both. Combining both methods is also not an issue, you'll just have an extra redundant entry. It really depends on how you do it, don't overthink it. >Circular dependencies might pop up, then the question would be, can Hydrus manage that, if not, could the DB get damaged because of logic's inconsistency? You'll get an error when you try to create a parent/child relationship that would cause a loop.

>>15998 You'd have to have some pretty fuckin weird tagging habits to end up with circular dependencies.

>>16000 >>16002 Thanks anons. I'm going to give it a thorough test.

>>15989 >Do you happen to have a copy of that error traceback? Should be in your log, if it made a popup in the main gui at any point oh fuck me, sorry about this, i didn't worded correctly. what im trying to say is that it can't understand that date string on my locale, for example, i put that date thing in 'single example string' textbox then i try process it using string converter, add 'datestring to timestamp (easy)' step, then client freezes, couldn't do it and gives up with "ERROR: Could not apply "datestring to timestamp: automatic" to string "7/18/2023 8:32:00AM": Sorry, could not parse that date!" no error logs on my side >test in console here's the thing, even without locales parameter, it works fine. trying looking through debugger i can see that it loaded english language and parsed it ok, so i tried set 'languages' to have only poland (['pl']), it didn't understand with that and returned nothing, even with locales set to none only after i set languages to have only english (['en']) it worked fine, again even with locales set to none >(if you know how to get into your venv of your source install) can you check that it works for you? i tried to plug it in init file (71 line ithink) in dataparser folder, nothing plugging languages trick in it also nothing i tried plug it in ClientTime file in ParseData function still nothing so i said "fuck it" and plug this "locale.setlocale(locale.LC_ALL, 'en_US')" into it and worked fine. i guess that's struggle when you live in country with 24 hour clocks, i guess if you need some tips on that front because i couldn't debug it further myself, try set your systems locale to be in country with 24 hour clock (like Poland as i demo'd)

need help. my client becomes unresponsive every time I click anything that's not an image. Though, when interacting with just images, everything works fine. I looked in the client logs, but it ends when the error starts, so there's nothing useful there. I generated a profile but I'm not sure how to use it. Btw, when I am in profile mode, everything slows to a crawl; even after turning it off, everything runs painfully slowly, thumbnails fail to load, and nothing shows up when searching (no autocomplete or results; it just says 'no searching done yet'). Funnily enough, the issue is present even with a fresh install of hydrus. I installed from source, and imported a random video to the new client. Just selecting the video made hydrus unresponsive. I can select a bunch of videos with cmd+a and change tags and such, but clicking a single video makes hydrus unresponsive every time. I'm running the latest version of hydrus, with mpv turned off at setup btw. I'm not sure what's causing it or where to start; it didn't happen when updating hydrus, or my system, so I'm stumped. Any help would be appreciated.

>>15982 586a, and it happens again and again. >>15994 > If you are still getting delays in v586, please try using the dialog with help->debug->profiling on, It's 586a. I'll try later. When I tried enabling it, not only did it slow everything down, but there were messages like "another profiling something is already running".

>>16010 >need help. Hardly you are going to get it when your request is flawed. You don't mention OS neither Hydrus version. >I looked in the client logs, but it ends when the error starts, so there's nothing useful there. Produce a screenshot or text for examination. >I installed from source I suspect you fucked the "Options" up. Yup.

When searching I don't have the automatic wildcard search any longer, and it only shows up when I press the "*" key. Any idea how to bring it back as it was before?

>>16014 Sorry. I'm using MacOS, and the latest version of Hydrus. I was using version 578 when the issue arose, but since updated to 586 to see if that would fix anything, which it didn't. I should've mentioned that I've been using hydrus for a few months just fine and it's an issue that came out of seemingly nowhere. I used the recommended options, but I don't even know if Hydrus is the problem since the same issue happened on a fresh install. I'm asking here because I don't really know what to do, and someone else might have an idea.

>>16016 >I used the recommended options I'm not familiar with Mac but most likely you need to tweak those options. Go to Help/About to find out what library versions are installed and then reinstall the venv accordingly.

found a bug with the parents and siblings management windows. shrinking the window horizontally doesn't resize the text boxes and panels where the parents, children and siblings go, so the box on the right will just be cut out of the window entirely. it's supposed to be resized so that they're both visible in the window at all times.

>>16015 I think that behaviour was changed on purpose in version 582. Check out the those: Question https://8chan.moe/t/res/14270.html#q15492 Answer from Hydev https://8chan.moe/t/res/14270.html#q15536 Update notes https://hydrusnetwork.github.io/hydrus/changelog.html#advanced_autocomplete_logic_fixes Guess that's it?

>>16019 Welcome newfriend. There is no need for full links within a board, or even between boards. A regular post reply link will do just fine, and a slightly modified one will work between differing boards. >>15492 >>>/t/15536

>>16020 I asked myself that question when i made the post. But didn't wanna try and fail :P thx Are normal users able to edit posts btw?

>>16021 No, but you can freely delete your own post and remake it. If you want to do so across browser sessions, you can edit and save the auto-filled password field under the More button.

>>16021 https://8chan.moe/.static/pages/posting.html

>>16022 >>16023 Nice. Thanks i will take a look at it later.

I had an ok week. I was not able to finish the list rewrite I had planned, but I did clear a mix of small jobs. The release should be as normal tomorrow.

>>16019 Thanks anon.

>>15753 >adding a 'paste-to subscriptions' button A fantastic idea. On the rare occasions where we add a new subscription, this would save a lot of time.

Sometimes I need to find a file by url quickly, and it takes over five steps to do it. Also, I think it would make sense to search by a page url from "manage urls".

>>15721 Is there a way to use hydrus on android device?

I've been getting really long (going on 25 minutes now) waits on adding parents or siblings dialogs since updating to 586a. Gentoo Linux running from source. I've been killing the program because it's frozen for like 5 minutes and my patience wears thin. Nothing in the logs, but tailing the profile does show actions happening. I'll upload the profile when I get tired of waiting or it unfreezes. Looking through the profile I don't see any sensitive info but I don't really get most of it, is there any info I should censor?

>>16029 hydrus.app can do just about everything lolisnatcher can view Hydrus These both require Hydrus running on your deskop, there's currently no way to run the db from Android.

Alright yeah I got tired of waiting, enjoy 45 minutes of profiling.

https://www.youtube.com/watch?v=czrmBIHANV4 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v587/Hydrus.Network.587.-.Linux.-.Executable.tar.zst I had an ok week. I didn't have time to finish my big list rewrite, so I'm just rolling out some little jobs today. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I made another stupid typo last week, breaking the tags->manage tag display and search dialog! Fixed now, sorry for the trouble. The top-right media viewer hover now shows all the local file services a file is in. For most users this will just be 'my files', but if you use multiple file services, I hope this will be a bit cleaner than the spammy labels I removed from the top hover the other week. I think I'm ultimately going to make these into buttons or add checkboxes or something so we can have one-click local service migrations in future. I cleaned up the code that handles the two resizable splitters/sashes that separate the normal media page sidebar from the preview canvas and the main thumb grid. There was some ugly stuff in there, and I think I have fixed some odd layout problems certain window managers had. That said, this stuff can be temperamental, so if you are on a weird OS and your pages suddenly lay out crazy, please roll back to your v586 backup and let me know. next week I will finish this list rewrite. I have all the code pretty much done, and I feel good about it, but I need to do a ton of testing and polish. It should let us view huge lists with far less UI lag. >>16033 Thanks, sorry for the trouble. I will check this out next week.

So... Any plans for making clip files viewable?

>>16037 >viewable Huh?

>>16039 U kno... So you can see the whole image in the viewer and not just the thumbnail

>>16040 Double click on it, then use the zoom in and zoom out icons.

>>16041 I mean for .clip files. They are only supported as thumbnails currently. I'd like to be able to see my work within hydrus without having to open clip studio paint, and without having to save everything as a .psd

>>16042 >They are only supported as thumbnails currently There's the answer so. The format is not supported.

>>16043 You have terrible reading comprehension my first post was literally asking if there were plans to support that file type

>>16044 >clip files Not the same as .clip files, fren.

>>16045 What is a clip file?

>>16046 A cut or section of a video file as the result of an editing operation.

>>16020 really? even regular old 4chan will automatically convert a full url into a regular post reply link when you click post. i thought these spinoff boards were supposed to be more feature-rich...

>>16050 KEK This thread looks more like /b/ everyday.

>>16050 >Removing natural newfag filters is a feature On cuckchan maybe. Here, that sounds like a bug.

>>16050 >>16021 >unabashed newfaggotry Don't see that very often these days.

>>16055 but how would something like that even work? how would you tell hydrus which "object" you're tagging? and what if you're not tagging an object, but something more abstract, like a genre, or a medium?

>>16056 yeah that's a good point, i don't know what the ux would be like but i think object instances could have ids behind the scenes for disambiguation. let's say a file has two person objects, internally the objects would look something like this. >person: { tags: [red_hair, brown_eyes], id:1 } >person: { tags: [blonde_hair, green_eyes], id:2 } that's not what you'd have to type but basically just the logic of how they'd be represented internally. like right now it's "file has tags", but it'd be cool if it could be "file has tags and/or object(s) with tags". also the ids would be per object per file since they're just a way to distinguish different instances of objects in a file. for genres and mediums, i think those are already covered by namespaces.

>>16055 reposting to clarify it'd be cool if tags could also apply to objects instead of just files. let's say i'm looking for files with any person who has both red hair and green eyes. i can search for the tags red hair and green eyes, but they could apply to different people (file has a person with red hair and brown eyes and a different person with blonde hair and green eyes). unless i'm mistaken i don't think right now there's a way to search for only files containing any person with both red hair and green eyes. this isn't a feature request, more like a feature daydream. i'd use the feature if it existed but i assume it'd be be a lot of work to implement and i'm sure there's a ton of other stuff to work on that's more important/useful. i just think it'd be cool. the idea's probably come up before but i'm new to the threads.

>>16057 >but it'd be cool if it could be "file has tags and/or object(s) with tags" It would be pretty amazing to be able to define subjects this way, that way when you're looking for a single character with combinations of traits, regardless of if they're the only character in an image, you could actually do that instead of how boorus work now where there's essentially a lot of "false positives" or "noise" or whathaveyou that interferes with your search that prevents you from searching for characters with multiple traits effectively unless you limit yourself to images tagged "solo". But this sounds pretty unfeasible for the current structure of hydrus and would introduce so many new logic problems that have to be implemented for it to function with other features of Hydrus. It would be a massive overhaul of the whole system that I can't imagine happening for a very long time, if ever. In mean time, to simulate this, the most I've personally done is create combination tags, usually with the two tags combined being parents, for certain commonly looked for combination traits that would otherwise be hard to find. Things like "fit female" (sex:female + body:muscle), "fit male" (sex:male +body muscle), "futaloli" (sex:futanari + body:loli), "shortstack" (body:large breasts + body:wide hips + body:short) et cetera.

>>16057 >>16058 I see, I get it now. personally what I've been doing is basically what >>16059 does. If there's a case where I want to tag that a specific character has some specific trait, then I essentially tag the file with a "compound" tag that associates some notable property of the character with the trait. this is just a complicated way to say that I frequently add tags like "male sitting" or "female with missing eyes" or "embarrassed tomboy" and stuff like that. It works surprisingly well, because most of the time, if there are multiple characters in an image, there will be some difference between them that I can use to tag them separately. the main downside is that you have to add all of the proper parents for each of these specific "compound tags" that you create, but you only have to do it once for each, so it's not as bad as it might sound. ideally, if there were a way to add relationships (parents and siblings) to groups of tags all at once, this problem would be essentially solved in a way that's compatible with how Hydrus already works, but Dev seems very hesitant to add new features to the relationship system due to it being complex, so I wouldn't hold my breath on that being added.

>>16060 >but you only have to do it once for each, so it's not as bad as it might sound. The issue is, with this method, being thorough is unfeasible because there's too many possible combinations. The object oriented method anon described makes those combinations naturally part of a search, just like with regular tagging, which raises the limit on the number of tags in one theoretical compound tag to as much as you want. >if there were a way to add relationships (parents and siblings) to groups of tags all at once You can already do that. Have you not used the parent/sibling management windows? You can add dozens of tags to a single parent at once, or dozens of parents to a single tag, or create a web over multiple parents and children, though I don't see any good use cases for the latter. You can also highlight multiple tags anywhere else, right click them, and select "add parents" for groups of tags you already have pulled up that you don't want to re-enter in the parent management window.

>>16062 >You can already do that I don't mean group as in "a set of tags that I type in" I mean "a category of tags that I define now and works for all tags now and in the future". there's no way to define relationships like that. >being thorough is unfeasible because there's too many possible combinations this is why adding parents to groups (or you could call them categories or sets to make it more clear) would help. you would tag the "sets" then any tags in those sets automatically get all the appropriate relationships. this is the simplest feature I can think of that would solve the issue, since it wouldn't change the definition of what a "tag" means like adding objects to them would. it would just expand what kinds of relationships you can make. I agree that being able to tag objects would probably be more intuitive for the user than what I'm describing, but I have no idea how something like that would even look as far as UI and management goes. for each file, you'd have to remember which "object" (in the file) is bound to which object (in the db) to make sure that you're not accidentally adding the wrong tags to a character. I just don't know how something like that could work without being so advanced that no one would ever use it. although to be clear, I doubt either of these will ever be implemented. the idea I'm talking about adds to the relationship system in a big way, and the object idea would be a fundamental change to tags, which is the most basic and core feature of hydrus, and kind of the entire point of hydrus to begin with. I'll live with that though. what I'm doing now has a lot of busy-work unfortunately, but it does work right now. >being thorough is unfeasible oh right, and to be clear, it is a lot of work, but it's not truly "unfeasible" because I'm doing it now... and it works. it took a while to "rig up" all the relationships, and unless you really care searching precisely (remember that tags are for searching) I wouldn't recommend it, but now I can basically tag who's doing what to who and who has what hair or what height or skin tone while they're doing it, and then 30 parents get added at once. /ramble

Hi! has somebody ran into this issue? v572, 2024-08-22 19:59:00: shutdown error v572, 2024-08-22 19:59:00: A serious error occurred while trying to exit the program. Its traceback may be shown next. It should have also been written to client.log. You may need to quit the program from task manager. v572, 2024-08-22 19:59:01: shutdown error v572, 2024-08-22 19:59:01: Traceback (most recent call last): File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/ClientController.py", line 2128, in ShutdownView self.DoIdleShutdownWork() File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/ClientController.py", line 768, in DoIdleShutdownWork self.MaintainDB( maintenance_mode = HC.MAINTENANCE_SHUTDOWN, stop_time = stop_time ) File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/ClientController.py", line 1428, in MaintainDB self.WriteSynchronous( 'maintain_similar_files_tree', maintenance_mode = maintenance_mode, stop_time = tree_stop_time ) File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusController.py", line 982, in WriteSynchronous return self._Write( action, True, *args, **kwargs ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusController.py", line 244, in _Write result = self.db.Write( action, synchronous, *args, **kwargs ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusDB.py", line 956, in Write if synchronous: return job.GetResult() ^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusData.py", line 1387, in GetResult raise e hydrus.core.HydrusExceptions.DBException: error: unpack requires a buffer of 8 bytes Database Traceback (most recent call last): File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusDB.py", line 619, in _ProcessJob result = self._Write( action, *args, **kwargs ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/db/ClientDB.py", line 10718, in _Write elif action == 'maintain_similar_files_tree': self.modules_similar_files.MaintainTree( *args, **kwargs ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/db/ClientDBSimilarFiles.py", line 707, in MaintainTree self._RegenerateBranch( job_status, biggest_perceptual_hash_id ) File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/db/ClientDBSimilarFiles.py", line 455, in _RegenerateBranch ( new_perceptual_hash_id, new_perceptual_hash ) = self._PopBestRootNode( useful_nodes ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/db/ClientDBSimilarFiles.py", line 345, in _PopBestRootNode views = sorted( ( HydrusData.Get64BitHammingDistance( v_perceptual_hash, s_perceptual_hash ) for ( s_id, s_perceptual_hash ) in sample if v_id != s_id ) ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/client/db/ClientDBSimilarFiles.py", line 345, in <genexpr> views = sorted( ( HydrusData.Get64BitHammingDistance( v_perceptual_hash, s_perceptual_hash ) for ( s_id, s_perceptual_hash ) in sample if v_id != s_id ) ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/jzbwvkhsw4izd1q9yj9sk5n07hx4iq63-python3.11-hydrus-572/lib/python3.11/site-packages/hydrus/core/HydrusData.py", line 392, in Get64BitHammingDistance return bin( struct.unpack( '!Q', perceptual_hash1 )[0] ^ struct.unpack( '!Q', perceptual_hash2 )[0] ).count( '1' ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ struct.error: unpack requires a buffer of 8 bytes

>>16064 I haven't that I can recall but >v572 I wouldn't remember that many versions back. v572 was released in April

The below discussion made me notice a simple feature request I'd like. Can the display window for currently searched tags get one of those "mouse hover and drag" borders at the bottom? I'm sure it can probably be adjusted somewhere else, but this would be more intuitive for when you're search lots of tags at once and want to see them all at once. Second pic related. >>16063 > "a category of tags that I define now and works for all tags now and in the future". there's no way to define relationships like that. The closest I can think of is making all tags in a namespace have the same parent. Like, all pokemon:* tags would have series/IP/whatever:pokemon as their parent. But I don't think that's a good fit for compound tags, which can easily encompass multiple namespaces. >this is why adding parents to groups (or you could call them categories or sets to make it more clear) would help. you would tag the "sets" then any tags in those sets automatically get all the appropriate relationships. I don't understand how you're defining this as different from what I described. All you're doing is adding another parent in the chain essentially from what I can tell. >Group of tags -> set -> parent How is this "set" different from a parent itself? >I agree that being able to tag objects would probably be more intuitive for the user than what I'm describing, but I have no idea how something like that would even look as far as UI and management goes. for each file, you'd have to remember which "object" (in the file) is bound to which object (in the db) to make sure that you're not accidentally adding the wrong tags to a character. I just don't know how something like that could work without being so advanced that no one would ever use it. The way I imagine it is simple. For tagging new files without tags, add buttons and/or shortcuts to add or delete objects. In the list of tags for a file in the tag manager, each object would have one line just like a tag, and would have tags beneath it with an slight indentation indicating they belong to that object. To add tags to the object, simply click it in the list of tags. Adding a new object should automatically select that object. As long as the object is selected, any tags entered would apply to it, instead of just generally to the file. Objects would be placed either at the beginning or end of the tag list with a sort toggle. For viewing tags outside the tag manager, you would be able to toggle if objects are displayed or all tags are displayed normally. Also, objects should be collapsible just like sibling and parent tag displays. >you'd have to remember which "object" (in the file) is bound to which object (in the db) to make sure that you're not accidentally adding the wrong tags to a character. Easy, just add the character's tag to the object. If it's an unknown or nameless character, there's really no workaround. Just pay attention to what tags are already in an object as you enter new ones so you don't start slapping them on the wrong object. For searching, firstly similar to parent child relationships, any tag within an object would apply regularly to file as well. So if an object is tagged as something, the file is also effectively tagged the same. You could add a button to enter object search mode, or a new key shortcut similar to how you can for creating an OR tag search. In object search mode, the object being searched will display in the tag search window just like in the tag manager. Taking up one line with tags applied to it being below it and slightly indented. All tags entered apply to the object until you leave the mode, either with the button, the key shortcut or clicking outside the object and its tags within the search window. Using the hotkey or button again will add another object to the search, and clicking on the object or any of its tags in the search window re-enters object mode for that object. I think all this would be very intuitive, but the issue is the background logic, the actual implementation of such a system, might be too difficult. I don't think it should interfere greatly with processing times, but I'm no dev. It would definitely be a massive upgrade to tagging structure that would be one more thing putting Hydrus lightyears ahead of any booru. I wouldn't be surprised if boorus adopted such a system, but on further thought, this thread can't be the first place to have thought of this, so maybe it's not practical?

>>16066 >For tagging new files without tags, add buttons and/or shortcuts to add or delete objects. Scratch that. Only an adding button is needed at most. Deletion is just like tags. Just double click.

>>15988 >Thank you! Should be fixed now, let me know if it gives any more trouble. Works perfectly now as far i can tell on my end, good job!

>>15996 >Thanks for answering, also all other people that answered! Hydev, could you answer 1b too please, would be interesting to know. Hydev, i wanna add another question regarding this topic, 1c: When you install the .exe, on the second installation page 'select components' there is a drop down menu that lets you chose 'install' or 'extract only'. Do i understand correctly, that 'extract only' does the same as the 'Extract.only.zip'? So i don't need the .zip if i use it for an external drive for instance and can only keep the .exe and save some incredible ~300mb (since i keep the latest versions of programms i use)? Afaik the only differences between installing and extracting is, that install does overwrite everything that it needs to overwrite, adds new fies and also deletes unused old files, whereas extraction overwrites/adds too, but doesn't delete any files, even if not used. So in rare cases a 'clean install' is needed. That's basically it?

Is there any way to edit the gallery-dl config file for hydownloader? Like the part where you can add extra sites (picrel)?

>>15996 >1b. Can the .exe and the .zip versions update each other? Means if i have an .exe install, can i extract the .zip over it and vice versa? For example if i have the .zip version on an external hdd/sdd (E:) and i chose to move it to my main disc on C:, can i then update this version with the .exe and if so, would it have any implications for the update process? For instance, a 'clean install' was necessary for some .zip versions in the past. If i would start updating the .exe over the .zip , is that already a clean install like the guide is suggesting for the .exe version? Yeah, should be fine. The installer (which is InnoSetup, if that's helpful) basically just does most of a 'clean install' and then does some Windows system stuff to A) add the shortcuts to start menu and B) add some stuff to the registry for uninstall purposes. It is all very simple, as far as these things go (mostly since I am not expert in that stuff), and does not affect the running of hydrus itself in any way. Worst case is Windows might get confused about your actual install dir, or an uninstall might miss some files to clean up. The hydrus exe never checks your registry or AppData folder or any of that shit. It is always running in 'portable mode', so if the install dir looks like an install dir, it'll run. The actual install script: https://github.com/hydrusnetwork/hydrus/blob/master/static/build_files/windows/InnoSetup.iss >That will have tech that can replace namespaces too right? Yeah. I think I've given up on the idea of a soft virtualised 'namespace sibling'. The logic would be possible but almost certainly a gigantic pain. We'll see if hard-replace covers most of the situations we care for. PTR is awaiting a huge 'artist:' -> 'creator:' migration in a similar way. >>16004 >>15998 For my part: A) don't worry too much about loops, my code is very strict about such things and generally won't allow you to enter one as a human. B) if a loop does get in to the db (mostly this means legacy data from the PTR, back from the days when it was easier to add loops), the database breaks the loop pseudorandomly, so while I'm confident it is robust, it unfortunately can give some dumb results (e.g. giving 'tricorder' to every 'star trek') if fucked with Thankfully, the sibling and parents systems are now perfectly 'virtual', so if you do get any errors, you can go into the siblings/parents dialog and fix stuff. The very recent asynchronous work I did on these dialogs exposes bad/loop pairs much better and helps you to break them manually to fix these old issues. I can't promise these systems are always amazing though. The logic has knocked me about for years now. Every time I think they are simple somewhere, there's a new set of headaches to deal with. Let me know if you run into any crazy slowdowns or miscounts.

>>16005 Thanks, I was able to reproduce the error from your post here and I think I have it fixed. I misunderstood exactly what was going on here. The new system in v587 basically goes 'try it in local locale, if that fails try it in english', assuming that the non-locale fallback is going to be english 99% of the time. I hope this covers most error situations, but let me know otherwise. Maybe japanese would be a good second fallback, although my guess is their timestamps are a subset of 'en'. This was a surprise btw, I thought dateparser worked on anything, but it caring about locale is odd. You can say hace 3 horas or whatever and it'll parse in 'en', so I guess it is a bit mixed. Anyway, let me know if you still have any problems here. >>16010 >>16016 Sorry for the trouble. I know macOS has had some pretty weird 100% CPU issues when trying to position certain elements in the media viewer before. This has usually been related to (historically) fullscreen borderless modes or, more importantly, mpv embeds. When you say you have 'mpv turned off', does that mean that under options->media, your animations/video/audio are set to 'native viewer' or 'open externally button'? If your source macOS install has somehow discovered an mpv .so file to use, I suspect it is trying to load by default and you are running into the current state of mpv dev in macOS, which is: I am afraid it is broken 100% CPU. If you are definitely set to not use mpv for any media, but you are still getting 100% CPU on a non-image click, then yeah I think let's check the profile logs. After you generate a profile, go file->open->database directory, and it should be there, a .log file. You can pastebin it here or zip it up and email it to me. Give it a look, but it shouldn't have any identifying information. If it is hundreds of MB, then see if you can cut out any repeated section and just paste that. In any case, let me know how you get on. >>16018 I am afraid I cannot reproduce this--when I try to resize, it is a little jank but it does scale things down. Can you say: A) which OS you are, B) which options->style style and stylesheet you are? >>16028 If the URL is already on a file, does pic related do it? Or are we talking file urls that might not be in this menu list? Adding this function to the manage urls dialog is a good idea though, so I'll add that. >>16029 As >>16031 says, there are some third-party apps that can talk to the hydrus 'Client API' to wrap your personal PC's collection in a web booru layer, and some are very good, but I doubt hydrus will ever run natively on android, I'm afraid. Even if a team of phone-competent programmers took up the work, it would be too much work, like it might need a complete UI overhaul since I don't think Qt will run on android, and the kind of stuff hydrus wants to do is probably more than a phone wants to give permission for or can generally handle. Can you even invoke ffmpeg on a phone? Hydrus is a PC program.

>>16076 >I don't think Qt will run on android https://doc.qt.io/qt-6/android.html

>>16033 Thank you, this was useful. I don't see anything super horrible in database terms, I'm sorry to say, but I do see that some maintenance jobs are all clustered up and causing what looks like a traffic jam. Please try turning off: - duplicates page->preparation->cog icon->search for potential duplicates... - tags->sibling/parent sync->do work in normal time (although note this will stop quick recalc of parents, which I imagine you want) Those two seem to be the biggest problems here, although there's something else I can't identify that's causing the actual UI lag. Maybe it is some UI-update reporting after these jobs are done. I suspect it is potential duplicates mostly knocking you about, but let me know what happens when you turn both those off. If things are still garbage, might be worth doing another profile. I do not see a 25 minute delay, or anything beyond 4 seconds, so unless there's some very subtle pile-up or deadlock going on here, I don't think we captured it this round. >>16037 Probably not, I'm afraid. I'm subject to whatever simple/popular libraries can support, so if PIL/Pillow can read it, I can show it, but otherwise we are hacking some bullshit. I think for .clip files we do something where we read the file itself (it is secretly a .zip) and then extract a .png preview either as a raw file, or maybe we extract it from an internal sqlite file inside the zip. So we are cheating. If there's a package an pypi that can read clips natively to a raw bmp, I can probably figure out an answer here, but these 'rich' application formats that have multiple layers and all sorts of vector effects and things are probably just too complicated for us to show 'properly'. At least for now! >>16056 >>16057 >>16058 >>16059 >>16060 >>16062 >>16063 My general thoughts on this are: it sounds like a neat idea, but I think the technical requirements and the endpoint workflow make it not worth it. Others have thought of this, with different solutions like coordinates for tags (like booru translation boxes), or sibling/parent-like tag relationships, or nested namespaces in some sadpanda male:penis sort of way, and ultimately, in the end-state, I think it means thousands of hours of extra programming and tagging work to shrink a results set from 17 files to 3. It is easier just to apply your human eyes at this level. For common situations, your 'male sitting' answers are the way, I think. Often mixed with parents. Maybe in future, if we end up with models that can auto-tag in essentially zero time, we could explore richer tagging metadata, but for now I'm at my limit of capability with siblings and parents. I won't try anything more complicated, and I have no idea what the UI workflow for this sort of stuff would be. How would you enter a search phrase for this stuff simply and quickly, and what would the UI look like? There is a (tempting) danger in autistic navel-gazing, in a project like hydrus, and we are wise to shake ourselves out of it. Don't try to create a utopian mind palace, just try to add some simple tags that apply to your real-world problems. The two master rules of not going crazy: 1) Tags are for searching, not describing 2) Only tag what you personally search for

>>16077 Sorry, I meant 'our weird Qt situation'. I'm on python Qt, not the normal C++ Qt, which, if it is possible to run on android, is I'm sure an absolute nightmare. And then I do some weird shit that may not be supported by whatever Android and Android-Qt allows. Trying to mishmash all our bullshit into their Java wrapper is probably not a recipe for success, not to mention reworking all my duct-tape PC centric code to phone-acceptable stuff, so you'd probably be looking at a large rewrite anyway even without the python/C++ issues. Bottom line is we absolutely cannot import the hydrus codebase to an android Qt environment and expect anything to work without a ton of work, and, most importantly, expertise from phone developers. If you were going to do it, you might as well rewrite the whole thing in Electron or something. Native android hydrus from me is a technical no-go. >>16064 Thank you for this report. You somehow got a bad phash in your database. I will see if I can write some better error handling to recover from this situation. You might like to check 'help my db is broke.txt' in the install_dir/db directory. I am not saying definitively that your database is broke, but you might like to just run whatever the Linux version of chkdsk is, and crystaldiskinfo, to make sure your hard drives are healthy. This bad hash might be the result of a hard drive blip. It would be worth running the 'pragma check_integrity' thing on your client.db as well--just check the document. >>16069 >When you install the .exe, on the second installation page 'select components' there is a drop down menu that lets you chose 'install' or 'extract only'. Do i understand correctly, that 'extract only' does the same as the 'Extract.only.zip'? So i don't need the .zip if i use it for an external drive for instance and can only keep the .exe and save some incredible ~300mb (since i keep the latest versions of programms i use)? Afaik the only differences between installing and extracting is, that install does overwrite everything that it needs to overwrite, adds new fies and also deletes unused old files, whereas extraction overwrites/adds too, but doesn't delete any files, even if not used. So in rare cases a 'clean install' is needed. That's basically it? I think you are basically exactly correct. The only thing is I think the pseudo-clean-install, which is the 'InstallDelete' section here https://github.com/hydrusnetwork/hydrus/blob/master/static/build_files/windows/InnoSetup.iss happens in the 'extract only' case too. I'm not sure if there is a way to turn this off, so I guess 'extract only', which does not do the two 'desktopicons' and 'programgroupicons' 'Tasks' and does not set up Uninstall info, really means 'install but do not register with OS'. Anyway, for your purposes yeah you can treat it as the same as the zip. It does what you want.

Minor UI annoyance but there ought to be an "all my files" and "all local files" buttons under the pages navbar menu: pages -> new file search page -> all my files (not present) Like there is under the new page picker popup: pages -> pick a new page... -> popup [ file search -> all my files ]

Is it possible to add catbox collections as supported in the regular import page? They easily work if you use the "download all files linked by images in page" setting in the simple downloader but don't work on the normal url downloader page. I only have one catbox collection link on hand and it's NSFW WIP loli so idk if you want to use it for tests or not but it's here if you do: https://catbox.moe/c/5bub9c I think you need an account to make collections and I can't be bothered to do that.

>>16078 I think the freeze has something to do with sibling/parent sync in normal time, I had it off for a while and no freezes. I turned it back on and it just happened again. I wasn't profiling so I don't have any data this time.

>>16083 Oh I should mention, it's specifically in the "choose a reason for this parent/sibling" dialogue

>>16085 Yup, definitely related to the "choose a reason" PTR dialog box. I also noticed that when I sigterm the initial process it cleans up the db just fine but the UI doesn't fully go away, I have to sigkill the process again for it to leave properly.

Is there a fast way to download an image from Twitter together with author tag and post url?

Is there an easy way to export tags with the spaces replaced with underscores? I want to upload a bunch files to a booru

I will remove "lore:trans (lore)" tags from any PTR files that are merely dickgirls, and no amount of wishful thinking can stop me. >>16088 gallery-dl, as always, though it needs an account for NSFW and a configuration file if you want the author tag and post url. I believe hydownloader has integration for gallery-dl metadata files. (it's not a fast configuration) But there's also shitter downloader plugins that add author tag and post ID to the filename.

>>16078 Speaking of tagging daydreaming, would it be possible to append variables to every tag per image (defaulting to none)? I'm thinking "weight" and "confidence" floating variables. Default to 1.0, but a float from 0.0 to INF carried along with a tag when specified. 0.5 means it's weak, 2.0 means it's particularly strong, with no real intended "scale". This'd be more useful for the AI-classified future, but I've already wanted to use something similar to find images that are really strong representations of a tag. AI slop is already capable of inferring "strength" of certain tags, and there's no particular reason why either perceived weight (or at least confidence) couldn't also be carried by a database in the near future. Classifiers tell you their confidence already, it's just a matter of noting it down. It'd be handy to search for only confident detections, or to manually check low confidence ratings within the database itself. I don't know whether it'd be better to do this within Hydrus itself or yet another sister application. The real issue would be displaying these.

>>16076 >Can you say: A) which OS you are, B) which options->style style and stylesheet you are? I'm Fedora Linux 39, and my stylesheet is the default. I'm running from source and the Qt version is 6.6.0 I can trivially reproduce it because it happens every time. I restarted hydrus and it still happens I was wrong about it being both the parents and siblings pages. it's only parents. siblings seems to resize correctly

I had an ok week. I finished the list rewrite, so all multi-column lists across the program now populate and sort far quicker, particularly when they have tens or hundreds of thousands of items, and I fixed some bugs. The release should be as normal tomorrow. >>16093 Thanks--maybe fixed tomorrow, let me know how it goes. >>16083 >>16085 Thanks--maaaybe fixed tomorrow, let me know how it goes.

>>16076 >let me know if you still have any problems here. >checks locale in client >locale: Polish_Poland/pl_PL me: client, whats timestamp of this "7/18/2023 8:32:00AM"? (using converter in string processor) client: oh! it's "1689661920" >sets locale to "English (USA)" on windows >restarting client >locale: English_United States/en_US me: client, whats timestamp of this "7/18/2023 8:32:00AM"? client: oh! it's "1689661920" >sets locale to "Russian (Russia)" >restarting client >locale: Russian_Russia/ru_RU me: client, whats timestamp of this "7/18/2023 8:32:00AM"? client: *lags for 5-10 seconds* client: ERROR: Could not apply "datestring to timestamp: automatic" to string "7/18/2023 8:32:00AM": Sorry, could not parse that date! inb4 "am i a joke to you?" to be honest, i don't know either on how it ended up like that this thing is so weird

https://www.youtube.com/watch?v=X7OpjB_8sHQ windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v588/Hydrus.Network.588.-.Linux.-.Executable.tar.zst I had an ok week. Multi-column lists work faster across the program. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I finished my list rewrite. Multi-column lists look and work exactly as they did before, but they initialise and sort faster. I still have some optimisation to do, but my test list of 170,000 items now sorts in about four seconds. More generally, many normal delete and insert events should have just a little less lag. I hope this makes dealing with large file logs and so on a bit less of a hassle! Otherwise, I fixed some visual bugs and cleaned up some similar files maintenance code. next week I want to optimise some db maintenance code and otherwise just do some simple cleanup.

>>16096 While on DB maintenance, could you please add an option to vacuum databases on another disk? I know it can be done with some hackery, and I've done it before, but it would be a lot more convenient to specify them explicitly. The default of "/tmp" for vacuuming large databases (like PTR mappings) also isn't great.

>>16094 >let me know how it goes. the parents window issue is still there in v588. the window won't shrink horizontally so the right side of what's supposed to be there just gets cut off.

Is there any way to migrate parents and siblings from one service to another only when at least one of the tags in each pair are actually present in "all my files"? I asked about this about a year and a half ago and you said that you had plans to implement a way to do that some time in the near future. I know it's probably not here yet, but I figured I'd ask again anyway. I stopped syncing with the ptr about 2 years ago, but I still have the service in my db because I don't want to add a whole bunch of relationships I'll never need to my local service, but I also don't want to just lose everything either. You're already able to do the equivalent of what I'm talking about with mappings, taking the mappings for just the files in "all my files" and discarding the rest. basically what I'm asking for is the same thing, but for relationships, where you take all the relationships that involve tags in "all my files" and discard the rest. it's not urgent of course, but it'd be nice to finally get rid of the service and hopefully shrink down my db in the process.

>>16081 Thanks. I'll probably hide this behind an option/advanced mode. I don't want to direct new users to those file services much as they often confuse. I also need to rewrite some layout GARBAGE I accidentally did the other week that broke the 3x3 buttons on the new page selector when there are less than four buttons to show. Also I'd really prefer to tie the favourite searches system into page selection. I open a new page and don't care about the file service since I'm just loading a favourite search anyway 95% of the time. >>16082 Sure, I'll check it out. Can't promise anything, but a quick look suggests this wouldn't be too tricky. >>16083 >>16085 >>16087 Thanks again for this. I investigated what could be causing a freeze during the 'enter a reason' text dialog, and the odd thing is that I discovered a possible way to enter a complete deadlock, i.e. the whole program freezes indefinitely, but not a way it could freeze for 25 seconds. So, maybe the deadlock wasn't happening the way I thought, and/or python/Qt had a way to get out of it (seeing some recursive call or similar), or I did not catch what was causing the delay here. Anyway, I fixed the major problem I saw. That said, I had a good look at your profile, and while I see some ~1.3 second database delays, I see nothing that would obviously cause a long delay from the db end, so I think we are looking at some UI traffic jam. Let me know how you get on and I'll keep working at it. >>16088 We have very good default support for single tweets. Just drag and drop the tweet URL onto your client and it should work, video too, with creator and URL and tweet text as the note. This works through the excellent fxtwitter and/or vxtwitter services. You can also just batch up the tweets you like and then paste into a hydrus urls downloader page. What we can't do is search twitter. (Elon now charges, I think, $5,000 a month for this) I personally use yt-dlp for many video tweets, too. It works out of the box for everything except 'nsfw', which needs login credentials. >>16089 Not right now, but I hope to add some tech around this subject over the next year or so. I want to finally push on 'here is a tag replace rule, apply it to everything', with particular attention to the underscore cases.

>>16092 That's a fun idea, and I agree a system like an automatic classifier would be the place to figure it out. When I first started the program, I expected to write all sorts of custom sort and presentation rules for number tags, stuff like a 'cup size' namespace that would convert from 'a-cup' to a sortable value, but ultimately it never panned out. The overhead of implementing these systems, and then that on the user of actually maintaining and editing them, always trumps the simplicity of 'huge breasts'. It would totally be possible to store tags with a confidence value. Instead of the classic (file_id | tag_id) mapping pair, you'd have (file_id | tag_id | confidence_float) or (file_id | tag_id | confidence_float | weight_float ). It would make a slightly bloatier database, but not dreadful. Probably not appropriate for the PTR or any other mass-share, since a couple billion floats will add up, but it could be good for a local store. I suppose you could start thinking about special search or display tech that said 'find me all the "skirt" with confidence > 0.8', or for weight 'find me all the "bikini babe" that consumes > 65% of the screen'. I think the confidence variable might be moot for storage, since I imagine we'll narrow down specific models, or perhaps all models, to a good confidence threshold and not broadly be interested in altering that after the fact, and thus the confidence is something we play with when we add tags, i.e. in deciding yes/no to add, not after they are added. But yeah I'd only think about this if we could generate these variables automatically. A human is never going to be able to generate reliable 'weight' numbers, nor have the patience to do that more than 50 times. The more I think about an automatic weight, the more I like it. >>16093 >>16098 Damn, thank you. Some of that might be the list being rude about its minimum width, so can you try right-clicking on the header and selecting 'reset default column widths...', and then I expect you'd need to close/reopen the dialog. Does that fix/improve anything, or does it want to be huge again? Ideally, when you make the dialog wider, all the spare space is going to be eaten and then surrendered by the 'note' column of that multi-column list, but if the other stuff gets wide, it might be screwing up the whole thing. There's a confluence of several kinds of shit layout code going on here, just years of me reinventing the wheel on 'man it would be better if dialog panels set their own scrollbars bro', and 'man it would be better if dialogs resized themselves dynamically bro' and it failing in certain situations. >>16095 Thanks, I'll poke around again. The language/region/locale parameters here are funny; they seem to, inside dateparser, to be converting to 'xx_XX' locale strings for a lookup dict, but when I actually looked I didn't find the classic 'en_US', so I opted for 'languages = [ 'en' ]'. I guess it is somehow getting tripped up with some bonkers 'en_RU' mapping or something. You obviously have your own experiences, but I'll state for my part that calendar calculations are the absolute worst thing I have ever come across in computing. Everyone involved in designing calendars and times and locales over the past ten thousand years seems to have been non-engineer astrology-brained. I was fucking boggled beyond belief to learn about this some time ago: https://en.wikipedia.org/wiki/Unix_time#Leap_seconds There was the chance, finally, to make a non-fucked-up time format we could all sync to, literally just count one second for every second bro, and the lizardmen simply wouldn't permit it. >>16097 Very good idea. I will see what I can do. >>16099 Sorry, I don't think I have the tech to pull this off just yet. You might be able to figure out part of this with the Client API, but it would be a pain in the ass. We need better ways of selecting parents and siblings. I just made the dialogs work a hell of a lot faster, so perhaps we can start doing some better import/export logic there, with mappings lookups, and add the same to the 'migrate tags' dialog. I'll make a job to think about it.

>>16101 >can you try right-clicking on the header and selecting 'reset default column widths...', and then I expect you'd need to close/reopen the dialog. Does that fix/improve anything, or does it want to be huge again? Actually that fixed the problem, and it looks like it's not going back to cutting off when I reopen it! Nice and thanks! I wonder what column width really had to do with the issue though. I could see all the columns fine so that wasn't the problem. It was that the "frame" wasn't getting narrowed properly when the window's width was reduced, so the window was just cutting it off. it didn't seem to be related to the lists. A minimum width that's too high should never result in broken windows like that anyway. Is that an issue with Qt itself?

For the auto deduplication tech, will there be a way to "rank canonicity" or something? Put simply, an artist's official Pixiv is more canon than a booru page, so a pixel dupe with a Pixiv hash would be more "correct" and preferable than the booru hash. Does that make sense?

Sorry if you've already been asked this hydev, but would it be possible to add custom quick actions to the duplicate filter? One example I can think of is "this is a variant set" which would only transfer some tags over such as creator or series, but not all tags. Adding another quick action just for the 1% of images that are variant sets probably wouldn't be worth it for you but it would helpful for users to have the ability to do stuff like that.

Hey, I'm pretty sick, so no release tomorrow. v589 should be out on the 11th. Thanks everyone!

>>16112 Hope you get well soon, bro!

>>16112 Ganbatte

>>16112 >I'm pretty sick F Get better devanon.

>>16112 I'm channeling my energy through the ether to allow you a faster recovery!

Is there a reason why the SHA256, Blurhash and Pixelhash show in parentheses but MD5, SHA1 and SHA512 don't? SHA512 is a bit longer and could maybe shortened by '...' before displaying in parentheses. I wanted to compare the filename from a sankakucomplex file to the hashes, to see if the filename is in fact one of those hashes. But i had to first copy the MD5 hash and paste somewhere to be able to compare. If it would display, i wouldn't need those extra steps. If it is possible, please make them display too Hydev. Wish u a fast recovery!

I recall reading here a month or 2 ago that there was a way to still have subscriptions for x/twitter but it'll only grab the first 10 or so tweets. I can't find it in the defaults or the cuddlebear repo though. does anyone know what that was? I'd like to have that hack solution and just force the subs to check more often.

>>16102 Great, I'm glad it was something simple in the end, even if it is all my fault. This was one of my 'reinvention of the wheel' moments of genius, I decided to make a new sort of dialog panel that would handle some sizing things more dynamically, and while I am happy that hydrus generally does not suffer from cramped UI that you often see in other techy 'dense' programs, and it generally successfully eats up monitor space for guys on 4k etc..., we do get the opposite problem here where if the internal widget in the panel is adamant that its minimum size is 1,200 px wide, then the dialog is going to force that and set scrollbars. My multi-column list is one of those things that will insist that its columns are the same size as the previous time it loaded, with the exception of the last column which is supposed to be the resizable one. I think in your case the 'note' column was entirely hidden behind the scrollbar. Basically resetting the column widths back to default allowed the dialog to refigure itself properly. I'm slowly fixing some of my bad layout decisions, some of which were spawned in the wx days, to more Qt standard, and I'm happy every time I do so. I'll keep working here, but I am also fond of a couple of the dumber ideas I've had for resizing dialogs, so we'll see where we end up. >>16109 Interesting idea. My intention for the future state of the duplicate comparison system (and the auto-resolution system, which will use the same tools), is to have a new 'metadata conditional' object that allows you to deeply customise comparison scoring. The metadata conditional is going to be an algebraic lego brick that will plug in all sorts of cases and will say 'file x has y property'. I think you'd be able to say 'if file A has a pixiv url and B has a booru url, give A +20 points', so I think you'd be able to do this in time under your own steam. I can't promise the system will be this clever for a while though--we'll be starting with simpler stuff like 'file A has >2x the number of pixels of B' and so on. >>16110 This is a very good idea, thank you. We need more customised control here, and will want more of it in future. I'll have a think. >>16115 >>16116 >>16117 >>16118 Thanks, no worries. Just one of those things that completely knocks you out a couple days. Doing 7/10 now. >>16119 Yeah, it is a silly thing but the media object that backs thumbnails in hydrus knows about the sha256 and pixel/blur hashes (the media object loads them from the db as it is created, since they are used for some UI stuff), but the md5 et al need to be fetched from the database on an as-needed basis. Since you are only ever looking at one file here with human eyes (and thus it won't be db-expensive), I'll make a job to populate that menu with a db request. I know how to do this quietly and quickly in the background these days. >>16120 I think it is fucked now and the guy(s) who were trying to maintain it just gave up. afaik it was using like one of those 'other tweets by this user' boxes that you see embedded in blogs to get those 10/20 tweets, and I guess that API route is now blocked or obscured. If you want to search tweets, you gotta give Elon $5,000 a month! https://developer.x.com/en/portal/petition/essential/basic-info?plan=pro I have a memory that vxtwitter or fxtwitter were looking to provide some easy search layer, but I think again it was ultimately blocked by obfuscation on twitter's end. Twitter are actively trying to make it difficult to search, so we are unlikely to see a good solution here.

>>16121 >Since you are only ever looking at one file here with human eyes (and thus it won't be db-expensive), I'll make a job to populate that menu with a db request. I know how to do this quietly and quickly in the background these days. Good stuff! 👍

>>16100 Sorry for the delay in getting back about the UI hanging, I haven't gotten around to updating since it seemed that disabling the realtime parent/sibling sync seemed to completely stop the hanging but it happened again just recently so I finally upgraded and seems to be resolved now, thanks hydev!

Looks like idolcomplex added new pointless namespaces. I've seen anatomy, automatic, fashion, object, pose, and setting.

When you check the queries of a subscription it gives you an option to check only living ones, only dead ones, or both. It'd be cool if there was also an option to only check unpaused ones, since that's what I basically always want, and I got bit by accidentally running all the paused queries that I didn't want to run yet, by using that button and not realizing that it would check paused ones as well.

>>16125 I second this feature suggestion.

Hydev, give it to me straight: is Hydrus ever going to work on Wayland?

>>16128 Nothing is ever going to work on Wayland. It's by design.

hate to ask as I'm sure it's an option i'm overlooking but is there a way to change times from "2 months 9 days ago" to timestamps?

I had a good couple of weeks. I mostly worked on code cleanup and optimisation, so large clients should feel snappier. The release should be as normal tomorrow.

>>16128 >Wayland Another layer of abstraction to bloat even more Linux and complicating software maintenance. Keeping things as simple as possible would be better.

>>16132 easing maintenance and making session management simpler and more secure is exactly why Wayland exists. You have no idea why x11 needed to be replaced. >>16130 in the options go to "gui" then under misc, check "prefer ISO time"

https://www.youtube.com/watch?v=Ka4pfP2z8iA windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v589/Hydrus.Network.589.-.Linux.-.Executable.tar.zst I had a good couple of weeks mostly cleaning code and optimising things. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights If you have a lot of files, your database update may take a couple minutes this week. Some users have had unoptimised similar-files search for a while, and this fixes it. If you have a lot of tag siblings and parents (e.g. if you sync with the PTR), I have reduced the lag around sibling/parent processing significantly. There's a cache that previously had to be regenerated on every sibling/parent change, and for the PTR that could take 1.3 seconds, now it just updates in a couple milliseconds. If I have been working with you on lag issues, and we discovered that shutting off 'normal time' sibling/parent work fixed it, try turning it back on this week and let me know how it goes. If you do prefer manual control of this, there's also a new 'sync now' menu entry under tags->sibling/parent sync that basically slams all the 'work hard' buttons and functions as a catch-all 'do all outstanding work now'. A bunch of big lists like the 'file log' should now initialise and sort a little bit faster, particularly when you are pushing 10,000+ items. I fixed the 'page chooser' dialog's 3x3 layout, which was accidentally un-done in a recent layout overhaul. This thing is stupid, but I'm still fond of it. There's also some new checkboxes under options>pages that govern whether 'my files' and the more advanced 'local files' will show up in the 'files' choice. The safebooru parser should stop grabbing '?' tags, and catbox collection URLs are now parseable. next week I want to get some meaty work done, either chipping away at dynamic file storage or duplicate auto-resolution.

>>16134 my I had to copy my folder but some db file is missed. as a result all pics are gone. But they are still in db folder, I just can't see them when I open program. How could I restore them :(

Can I generate additional URLs to files already downloaded? For example so that files downloaded through site API would be marked in browser extension on corresponding site regular web UI. Data (IDs) required to construct web URLs is available within API response.

when the preview frame is completely collapsed, it'd be cool if it didn't load the preview into the frame. There are times where I quickly flick through many files, and the constant loading of the previews lags hydrus a bit, and it's especially annoying if they're videos with sound. I'm guessing that Hydrus knows when the frame is collapsed, because it snaps shut rather than sliding shut gradually. it's not a big deal, but this would be nice.

I accidentally closed a page of 52 4chan watchers, most of them dead. I have a database backup with the page. How do I restore the source data?

Been using Hydrus for years as an early adopter and supported on Patreon when I could afford to do so. You're a fucking saint for continuing development for as long as you have and as consistently as you have. (And a thank you to all the other people who have chipped in to help as Hydrus has grown beyond a single-developer project). Just wanted you to know that I managed to update from v237 without any issues although it did take a while going 20-30 versions at a time. Anyways I have a general question for anyone who might know the answer. What would be the easiest way to convert/optimize Hydrus files that are already in the db without fucking up my db? I want to make use of newer formats for a lot of my files without having to re-import and re-tag everything. Thanks.

>>16140 the dumbshit way would be to run an external script, re-import everything, and use the appropriately configured duplicate scanner to port everything over, it should all be either pixel duplicate or very close distance

I've been using the same path folder since version... I don't know, 420? and I noticed that when I try to run hydrus_server.exe I get this error (pic 1). Running that .exe from a fresh install seems to run fine (pic 2). Can I just backup my database and move it to a fresh install and move on with life?

>>16123 Hell yeah, should be even better in v589, let me know how things continue to go. >>16125 >>16126 Thanks, I will fix this! I'm not sure if it is even appropriate to ever apply that to paused subs, but I'll have a play around and see what makes best sense. >>16128 I am not a Linux guy, and I certainly don't have a Wayland machine to test with, so I very much cannot promise anything, outside of some deeply informed bug report that says 'oh I happen to know this Wayland-Qt lore, you need to change how this resize event is handled', but it working incidentally. There is another guy who I trust to make pulls to the hydrus codebase who has expressed interest in debugging different flavours of Linux, so it is possible he will figure out what is going on, but my general attitude is that my code is duct taped shit, and the python-Qt environment is pretty hurdy gurdy on its own, and then you add whatever new UI rules Wayland is trying, and I just can't guarantee anything clever like mpv is going to work well. That said, Qt is getting better, on average, with every new version, and its integrations into different OSes is improving too, so whatever the hell is going wrong on x or y system will, over the coming years, get less busted. It won't be much by my hand though! >>16133 >>16130 I hope to improve this in future btw, as this option doesn't cover all timestamps (and the system is broadly more complicated than one checbox can cover). I want to write a thing that'll nicely do a tooltip of the reverse, so you'll hover over '3 months ago', and it'll have the date as tooltip, and vice versa. Pain in the ass to do, but I want to improve how times render everywhere. >>16135 Sorry for the trouble. Are you absolutely, definitely sure you cannot recover that .db file? Or a backup, even an old one? Did you get the 'hey this looks like the first time you have run the program' when you booted up for the first time, or have you had a whole load of missing file/folder errors? If the missing file was client.db, I'm afraid your database is probable unrecoverable and you are starting back from square one. If you are ok with that, and you really do just want to recover the files in your client_files structure, then I think you probably want to look at 'install_dir/db/help my media files are broke.txt'. You'll basically be running the 'clear orphan files' command to get everything cleanly out, and then reimporting. BUT you really should try to recover your old database if you can. All your archive/inbox data and tags and things are probably lost as well, if you haven't got those files. 'help my db is broke.txt' in the same directory may have some useful background reading, since I think you may need to diagnose exactly what went wrong here more. Let me know if I can help any more. If you only missed one db file, then you may be operating on a mish-mash of old and new db files and there may be more work to do to clean things up before you recover, so let me know how you get on. >>16136 If you have URLs that you want to associate with files via the API, this is the command you want: https://hydrusnetwork.github.io/hydrus/developer_api.html#add_urls_associate_url

>>16137 Damn, this is not supposed to happen. I will examine how it tests this, I think the preview correctly knows it shouldn't load video if you hide it by double-clicking the slider resize thing (which removes it entirely rather than sizing it to 0px), but I wonder, if you manually size it to 0px, whether it still thinks it is loaded and should be doing things? I'll check it out. >>16138 If you have the page in a session backup (pages->sessions->append session backup): - Pause all network traffic under network->pause - Do append session backup and load the old session, it should appear in its own contained named page of pages - Drag and drop the watcher from the backup to your real session - Close the rest of the old session page of pages - Resume all network traffic. If you only have the page in a backup database file: - Open the backup client database by using the -d="path/to/db" launch parameter, as here: https://hydrusnetwork.github.io/hydrus/launch_arguments.html#-d_db_dir_--db_dir_db_dir - Go to the watcher page, and ctrl+a the list, then right-click->copy urls - Open a new watcher in your real client - Paste the URLs in For the second case, I'm afraid you can't simple move a page from one db to another, so just pasting the URLs back in should get you most of the way back. If you were not aware of the first case, have a poke around your session backups. The program makes a save on every client exit, so that may reach back a week or two if you check the timestamps. >>16140 Thank you for your support; I am glad you like it! It is cool the v237 update went ok. I think >16141 is the way for now. Export, convert, and reimport. I would recommend against it though just because it would be a pain in the ass, and say 'wait three to five years', because I expect hydrus to get much better auto-duplicate resolution tech, and with that it'll get internal file conversion tech. If and when, inshallah, we get Jpeg XL adoption, or some similar 'good' format, I foresee most of the internet doing a bunch of (possible AI-assisted) upscaling, HDR-ification, or just general 1-to-1 conversion for space-saving conversion from all the old sRGB jpegs and pngs, and hydrus will have a variety of ways of supporting that through an 'exe manager' system (search previous hydrus threads for more on this) that will call some external ffmpeg-like file converter, re-import the file output, and apply sensible metadata merge and de-dupe tech all in a native pipeline. When this will happen, I do not know, so if you want it done more promptly, maybe you might like to play around with 100 files now and see how much of a pain it is. I'd be interested to know the results if you do try it. My background rule on this, btw, is the hard drives are cheaper than man hours, so if your intention here is to save hard drive space, it is probably easier just to buy another 8TB drive than spend a hundred boring-ass human man hours on the conversion. >>16142 Sorry for the trouble. There should be a fairly simple fix here, no worries. Sometimes when I change the program, I need to change the database structure too. This is a very complicated thing, and I cannot guarantee that a change I make today will still make mathematical sense a year from now (this is called 'bit rot', and aggravated by me being a solo dev), so when your client wants to update from v520, I have a check first that says 'are we recent to the time this database was current, or is it used to code that is years old?'. If it was years ago, it dumps out and says 'please try an earlier version'. Thus, if you want to update from a very old version, like v300->v500, I ask that you not do it in one go, but in increments of 10-30, so you'd try doing v300->v330, with one full boot and then an exit, and then you'd try v330->v360, and so on, until you are up to the version you want to get to. That's what the error in the first screenshot is talking about. Please check here for more info: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#big_updates So, please try 'updating' to v535 or so and see if that works. Then try v558, and then v571. As that help document talks about, you may like to do clean installs too. Let me know how you get on!

>>16147 Thanks for the response and for what little it might be worth I find that .avif almost always outperforms .jxl and is my preferred format now it just sucks the support is still largely missing outside of browsers (and Hydrus! :D) > My background rule on this, btw, is the hard drives are cheaper than man hours, so if your intention here is to save hard drive space, it is probably easier just to buy another 8TB drive than spend a hundred boring-ass human man hours on the conversion. This is precisely why I haven't bothered but I'm hitting a point where I'm going to have to setup a NAS or something instead of using plug'n'play drives. Hardware solutions are still much easier than software ones - so if I need to keep throwing hardware at it I will.

>>16147 >>>16138 >If you have the page in a session backup (pages->sessions->append session backup): It disappeared earlier. >If you only have the page in a backup database file: >- Paste the URLs in I did that around the same time I posted the question, but I am pretty sure it only worked with desuarchive. Thankfully, desuarchive was still accessible despite being both protected by Cloudflare and blocked.

I really wish we had a button maybe next to the "paste" and "favourite" to just clear the current search. Right now I'm using the "empty page" option on the favourites drop-down menu but that feels like a click too many.

>>16150 I normally just highlight everything and double click.

>>16147 >>16142 (me) Thank you for the quick reply and thank you again for the amazing software! So, I managed to update my client to 589, and it's working great! One question though, since you mentioned clean installs: now that I have a working client + database backup, would it be better to just do a clean install and get that backup running? I noticed I have lots of old folders in my main hydrus path that maybe it's just "junk" that it's lying around from previous versions. What do you think?

>>16151 Yeah, this seems to be the only way with multiple "and"s. It would be nice to have a "clear all" button like on a calculator.

So, I started using Hydrus a few days ago, and I have LOTS of things to manually tag. I'm also going to get tons of tags from an autotagger (I'm using this model for now https://huggingface.co/SmilingWolf/wd-v1-4-swinv2-tagger-v2), but I also want to add certain tags that are unique enough for me to know them of the top of my head. How do you guys usually go around with this, do you have a set of tags that you choose from? Do you try to keep tags consistent with danbooru's tag styles?

>>16154 I do all manual tags except for some artist tagging, for which I have my own personal tags auto-applied to artist subscriptions. I stagger my activity between other tasks, usually chores, collecting new files from new non-subscription artists, or playing videogames. I import about 200-300 files at a time and then wait until I've finished tagging them before the next batch of non-subscription imports. This is my workflow. >Eros:non-erotic, semi-erotic, erotic Goes fairly quickly. Semi-erotic for things that have some erotic appeal but have something preventing me from enjoying wholly as an erotic work. This is usually applied to erotic jokes. >Lewdness: safe, lewd, explicit Explicit is a parent tag of whatever explicit body part is shown for the namespace exposure:nipples, cock, pussy, et cetera. During this I also tag if a pussy is a close slit, and if it's outline is visible through clothes, if the clitoris or urethra are visible, and if the clit hood is protruding. >Character count:solo, two-ten, 1-10girls/boys/futas/shemales, many girls/boys/futas/shemales, many, none >Character:name and Origin:anime, manga, videogame, cartoon, comic book, source material, etc Character: tags have IP: tags as parents, which is the equivalent of a series tag but makes more sense to me because not everything has multiple entries and is thus technically a series. IP:tags often have Genre: tags as parents. Once I went through and created most of these, I don't have to touch them very often. >Hair:long, very long. short, styles and Colors:black and white, redscale, bluescale, etc. and Hint of Color:hair, body, background, eyes, etc. This is the first really tedious step, and the next three are all fairly tedious as well. I group Colors: and Hint of Color: in this step since it goes quickly and is necessary to prepare for the next three steps >Eyes:color, number of eyes, kind of eyes, kind of pupils >Clothes color: and clothes under color: >Clothes:alternate outfit, nude >Body:trueflat, very small/small/large/very large breasts, small/large/very large cock, large nipples, loli >Frame:face, upper body, lower body, face cropped out, multipanel, character count:duplicates I define character count:duplicates as any instance of multiple copies of the same character that don't exist in the same physical timespace like clones, nor in any particular sequence of events. I put it here instead of with the normal character count because I kept forgetting it when it was originally there. >Creator: For anything not autotagged with a subscription. Usually goes very fast. >Folder specific tags / sets Anything miscellaneous that I easily mass tag large portions of the current batch of files with. Usually has a strong alignment with Character: tags. Set: tags are unique to a group of sequential images. >Individual tagging I set up three tabs each limited to the current import batch by tagging any subscription imports that came up in the meantime as "Set: temp #". One tab for all sets, one tab for all alternate groups not in a set, and one tab for remains after. I go through them in this order tagging thoroughly 5 files at a time. If less than 5 are left at the end, I apply a set: temp # tag to them until the next batch is ready for individual tagging. I've tagged about 31300 files so far. I have approximately 3200 to go. But I am also gathering new files from artists of interest until I am out of artists, then new files for characters of interest until that backlog is finished, then fetishes of interest until that backlog is finished. I only make subs for artists of interest, and once done I'll stop mass gathering files on the regular. I've been going for about two years I think. I applied quite nearly all my ideal tags with rare exceptions, and if I need to change a tag, I go through the effort of exchanging it for the new ideal tag instead of using siblings. I do not trust boorus nor AI to tag files to my standards and tastes.

>>16155 >>Character:name and Origin:anime, manga, videogame, cartoon, comic book, source material, etc <Character: tags have IP: tags as parents, which is the equivalent of a series tag but makes more sense to me because not everything has multiple entries and is thus technically a series. IP:tags often have Genre: tags as parents. Once I went through and created most of these, I don't have to touch them very often. Also, it may seem like it would be simple to make Origin: a parent of most IP: tags. But due to the existence of adaptations I find characters may often belong to both or simply just one of the origin: tags in a way that doesn't warrant creating a separate Character: or IP: tag to me.

>>16154 When I started using Hydrus I sat down and made a list of about 160~ types of tags and I ONLY use those tags. There's things that don't count under namespaces (eg: character names, IP, studio name, artist name, etc.) "hair color" is treated as a single tag even though it could be "white_hair|blue_hair|red_hair|purple_hair" etc. Stick with the colors of the rainbow you don't want to end up with tags like "chartreuse_hair" just use "green_hair". Like the other user I have a "lewdness" tag that I use for "safe|ecchi|hentai". I avoid using autotagger and the PTR because most people suck at tagging and you end up with many useless tags and a lot of tag clutter.

Any Linux users have a suggestion for backing up my hydrus client files (NOT the database)? I have a lot of stuff but I do have a server with plenty of storage. I'm just not very familiar with rsync or others to know what flags are good.

>>16154 I basically only tag rating, character, or series namespaces. I get most of my tags from the PTR or parse them from sites then upload them to the PTR. I follow the PTR's guidelines closely which is similar to gel/danbooru.

>>16157 >"hair color" is treated as a single tag even though it could be "white_hair|blue_hair|red_hair|purple_hair" etc. > Stick with the colors of the rainbow you don't want to end up with tags like "chartreuse_hair" just use "green_hair". I don't get what you mean. You're still naming more than one color, so it's more than one tag, right?

>>16155 >>16156 >>16157 >>16159 >>16161 Thank you for the detailed answers. It looks like it will take a while for me to get comfy with a tagging workflow, but now I see the path ahead.

>>16162 I am severely autistic. If you want to do anything else with your life, I am not an example to follow.

I'm doing a gallery download from pixiv for about the last 2 days, and I'm getting a lot of files downloading, that turn up "already in db". In other words, they should have been skipped, and not downloaded. I'm guessing this is after the file is hash checked. Instead of "URL recognized:imported...", they are marked with "File recognized:imported...". I think pixiv may have changing urls, and this is why this is happening. I suppose there is no way to get a file hash from pixiv before downloading? Files that I already have are downloading, and should have been skipped. I have seen a few skip with URL being recognized, but very few. Almost all have downloaded, then hash check has shown that they were already in the db. It really slows the downloading, and is a huge waste of bandwidth. Thanks!

>>16164 Also, I have already downloaded a lot of files from pixiv over the last 2 years or so, so I am assuming some of the "file recognized" is the same file already downloaded from pixiv, therefore I'm assuming they change their urls once in a while.

I recently borked something trying to update too many versions at once, yeah yeah I know. Luckily this was a backup drive so nothing was lost(other than my time) but was wondering if there's anyway to look into and undo this and make sure nothing went wrong with my database. Hydrus closed itself after failing to update so I haven't touched it since. Worst case, I'll just have to delete my database and copy over my back up again and I hate that it took me 3 days to copy everything over before. v559, 2024/09/17 19:39:47: hydrus client started v559, 2024/09/17 19:39:49: booting controller… v559, 2024/09/17 19:39:50: booting db… v559, 2024/09/17 19:40:11: checking database v559, 2024/09/17 19:40:13: updating db to v541 v559, 2024/09/17 19:40:13: updated db to v541 v559, 2024/09/17 19:40:15: updating db to v542 v559, 2024/09/17 19:40:16: updated db to v542 v559, 2024/09/17 19:40:18: updating db to v543 v559, 2024/09/17 19:40:19: updated db to v543 v559, 2024/09/17 19:40:21: updating db to v544 v559, 2024/09/17 19:40:22: updated db to v544 v559, 2024/09/17 19:40:24: updating db to v545 v559, 2024/09/17 19:40:29: updated db to v545 v559, 2024/09/17 19:40:43: updating db to v546 v559, 2024/09/17 19:40:44: updated db to v546 v559, 2024/09/17 19:40:46: updating db to v547 v559, 2024/09/17 19:40:46: updated db to v547 v559, 2024/09/17 19:40:48: updating db to v548 v559, 2024/09/17 19:40:48: [[[41, 3, [2, 4, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 46]], [[0, "simple_data"], [0, null]]]]]]], [[41, 3, [2, 4, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 46]], [[0, "simple_data"], [0, null]]]]]]], [[41, 3, [2, 3, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 46]], [[0, "simple_data"], [0, null]]]]]]], [[41, 3, [2, 3, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 46]], [[0, "simple_data"], [0, null]]]]]]], [[41, 3, [2, 28, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 47]], [[0, "simple_data"], [0, null]]]]]]], [[41, 3, [2, 13, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [0, 0]]]]]]]], [[41, 3, [2, 13, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [0, 0]]]]]]]], [[41, 3, [2, 13, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [0, 1]]]]]]]], [[41, 3, [2, 13, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [0, 1]]]]]]]], [[41, 3, [2, 14, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [1, 0]]]]]]]], [[41, 3, [2, 14, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [1, 0]]]]]]]], [[41, 3, [2, 14, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [1, 1]]]]]]]], [[41, 3, [2, 14, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [1, 1]]]]]]]], [[41, 3, [2, 11, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [2, 0]]]]]]]], [[41, 3, [2, 11, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [2, 0]]]]]]]], [[41, 3, [2, 11, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [2, 1]]]]]]]], [[41, 3, [2, 11, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [2, 1]]]]]]]], [[41, 3, [2, 12, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [3, 0]]]]]]]], [[41, 3, [2, 12, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [3, 0]]]]]]]], [[41, 3, [2, 12, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [3, 1]]]]]]]], [[41, 3, [2, 12, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [3, 1]]]]]]]], [[41, 3, [2, 9, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [4, 0]]]]]]]], [[41, 3, [2, 9, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [4, 0]]]]]]]], [[41, 3, [2, 9, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [4, 1]]]]]]]], [[41, 3, [2, 9, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [4, 1]]]]]]]], [[41, 3, [2, 10, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [5, 0]]]]]]]], [[41, 3, [2, 10, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [5, 0]]]]]]]], [[41, 3, [2, 10, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [5, 1]]]]]]]], [[41, 3, [2, 10, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [5, 1]]]]]]]], [[41, 3, [2, 15, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [6, 0]]]]]]]], [[41, 3, [2, 15, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [6, 0]]]]]]]], [[41, 3, [2, 15, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [6, 1]]]]]]]], [[41, 3, [2, 15, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [6, 1]]]]]]]], [[41, 3, [2, 16, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [7, 0]]]]]]]], [[41, 3, [2, 16, 0, [3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [7, 0]]]]]]]], [[41, 3, [2, 16, 0, [2]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [7, 1]]]]]]]], [[41, 3, [2, 16, 0, [2, 3]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 148]], [[0, "simple_data"], [0, [7, 1]]]]]]]], [[41, 3, [0, 97, 0, [0]]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 149]], [[0, "simple_data"], [2, [126, 1, [0, null]]]]]]]]], [[41, 3, [2, 6, 0, []]], [42, 5, [0, [21, 2, [[[0, "simple_action"], [0, 149]], [[0, "simple_data"], [2, [126, 1, [2, null]]]]]]]]]] v559, 2024/09/17 19:40:48: Had a problem saving a JSON object. The dump has been printed to the log. v559, 2024/09/17 19:40:48: Dump had length 4.66 KB! v559, 2024/09/17 19:40:48: If the db crashed, another error may be written just above ^. v559, 2024/09/17 19:40:48: A serious error occurred while trying to start the program. The error will be shown next in a window. More information may have been written to client.log. v559, 2024/09/17 19:40:48: Traceback (most recent call last): File "hydrus\core\HydrusDB.py", line 266, in __init__ File "hydrus\client\db\ClientDB.py", line 9830, in _UpdateDB File "hydrus\client\db\ClientDBSerialisable.py", line 747, in SetJSONDump File "hydrus\core\HydrusDBBase.py", line 289, in _Execute sqlite3.OperationalError: table json_dumps_named has no column named timestamp_ms During handling of the above exception, another exception occurred: Traceback (most recent call last): File "hydrus\client\ClientController.py", line 2212, in THREADBootEverything File "hydrus\client\ClientController.py", line 1003, in InitModel File "hydrus\core\HydrusController.py", line 588, in InitModel File "hydrus\client\ClientController.py", line 207, in _InitDB File "hydrus\client\db\ClientDB.py", line 239, in __init__ File "hydrus\core\HydrusDB.py", line 287, in __init__ Exception: Updating the client db to version 548 caused this error: Traceback (most recent call last): File "hydrus\core\HydrusDB.py", line 266, in __init__ File "hydrus\client\db\ClientDB.py", line 9830, in _UpdateDB File "hydrus\client\db\ClientDBSerialisable.py", line 747, in SetJSONDump

[Expand Post]

File "hydrus\core\HydrusDBBase.py", line 289, in _Execute sqlite3.OperationalError: table json_dumps_named has no column named timestamp_ms v559, 2024/09/17 19:40:48: boot error v559, 2024/09/17 19:40:48: A serious error occurred while trying to start the program. The error will be shown next in a window. More information may have been written to client.log. v559, 2024/09/17 19:40:52: boot error v559, 2024/09/17 19:40:52: Traceback (most recent call last): File "hydrus\core\HydrusDB.py", line 266, in __init__ File "hydrus\client\db\ClientDB.py", line 9830, in _UpdateDB File "hydrus\client\db\ClientDBSerialisable.py", line 747, in SetJSONDump File "hydrus\core\HydrusDBBase.py", line 289, in _Execute sqlite3.OperationalError: table json_dumps_named has no column named timestamp_ms During handling of the above exception, another exception occurred: Traceback (most recent call last): File "hydrus\client\ClientController.py", line 2212, in THREADBootEverything File "hydrus\client\ClientController.py", line 1003, in InitModel File "hydrus\core\HydrusController.py", line 588, in InitModel File "hydrus\client\ClientController.py", line 207, in _InitDB File "hydrus\client\db\ClientDB.py", line 239, in __init__ File "hydrus\core\HydrusDB.py", line 287, in __init__ Exception: Updating the client db to version 548 caused this error: Traceback (most recent call last): File "hydrus\core\HydrusDB.py", line 266, in __init__ File "hydrus\client\db\ClientDB.py", line 9830, in _UpdateDB File "hydrus\client\db\ClientDBSerialisable.py", line 747, in SetJSONDump File "hydrus\core\HydrusDBBase.py", line 289, in _Execute sqlite3.OperationalError: table json_dumps_named has no column named timestamp_ms v559, 2024/09/17 19:40:54: doing fast shutdown… v559, 2024/09/17 19:40:54: shutting down gui… v559, 2024/09/17 19:40:54: shutting down db… v559, 2024/09/17 19:40:54: saving and exiting objects v559, 2024/09/17 19:40:54: cleaning up… v559, 2024/09/17 19:40:54: shutting down controller… v559, 2024/09/17 19:40:54: hydrus client shut down

I had a good week. I did the background work I wanted to do, and for the release I've got a variety of quality of life bells and whistles to roll out. Nothing huge, but a bunch of little UI improvements. The release should be as normal tomorrow. >>16166 A quick look suggests your update code has hit some bitrot around the switch that moved from second-based timestamps to millisecond ones. That object that couldn't save looks like a shortcut set. Maybe I save a change to a shortcut in the v548 update step, and it is happening in that case before I do the timestamp update. I'm sorry to say this looks like roughly 35% bad fuckin' news. You can try booting into v550 or so--maybe it'll reset your shortcuts back to default or something. Maybe everything would be fine but for that. The sort of things that might be a nightmare, which you should check if it does boot, are your subscriptions, obviously your normal search page session, and your downloader list. Fingers crossed, it just fucked with your shortcuts though. If you aren't sure things are good, go back to the backup. If it is all fucked up, then yeah roll back your backup and try again. Master list as of here https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#big_updates appears to be: 521 (maybe clean install) > 527 (special clean install) > 535 > 558 > 571 (clean install). So I guess try v558 first? If the backup restore takes a very long time, I think for this situation you only need to restore the four 'client.*.db' files. A db update like this doesn't touch your client_files folder at all, so no need to delete and restore that--it should be the same for both folders. You can use a program like FreeFileSync to mirror your install back to the backup folder's state with minimum hassle just by having it check for different file sizes/modified dates. Hopefully that isn't more than a hundred gigs even for a PTR-syncing client. Let me know how you get on!

>>16167 Back, I went and did a clean install of v550 like you said and everything seems to be back to normal. Here's the log file. But yeah, I normally don't do updates from that far head, my fault for feeling ballsy jumping from v540 -->v559. Right now I'm still checking everything and so far everything is still there. My last session loaded up fine, Galley Downloads, tabs, tags, searches aren't giving me any errors, animated gifs and webms aren't giving me any errors, etc. I don't know how deep to look but I'm still willing to do a full back up again and start over if necessary v550, 2024/09/18 01:52:21: hydrus client started v550, 2024/09/18 01:52:23: booting controller… v550, 2024/09/18 01:52:23: booting db… v550, 2024/09/18 01:52:23: Found and deleted the durable temporary database on boot. The last exit was probably not clean. v550, 2024/09/18 01:52:31: checking database v550, 2024/09/18 01:52:34: updating db to v548 v550, 2024/09/18 01:53:00: updated db to v548 v550, 2024/09/18 01:53:02: updating db to v549 v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:09: An object of type String Splitter was created in a client/server that uses an updated version of that object! We support versions up to 1, but the object was version 2. For now, we will try to continue work, but things may break. If you know why this has occured, please correct it. If you do not, please let hydrus dev know. v550, 2024/09/18 01:53:10: updated db to v549 v550, 2024/09/18 01:53:13: updating db to v550 v550, 2024/09/18 01:53:13: updated db to v550 v550, 2024/09/18 01:53:13: initialising managers v550, 2024/09/18 01:53:27: booting gui… v550, 2024/09/18 01:53:28: The client has updated to version 550!

>>16168 Great, sounds good! Keep updating, and those 'String Splitter' errors will be fixed. That's some downloaders that were updated from the >v550 install that v550 doesn't understand--presumably a future update will overwrite them one more time and fix everything. Don't try to download too much until you are updated to at least where you originally wanted to get to.

https://www.youtube.com/watch?v=eYN9WivpQ6M windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v590/Hydrus.Network.590.-.Linux.-.Executable.tar.zst I had a pretty good week and have a bunch of quality of life improvements to roll out. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights The 'check now' button in 'edit subscriptions' and 'edit subscription' is now more intelligent and handles pause status. It'll ask you about paused subscriptions and queries, and it better examines and navigates what you can and will be resurrecting. If you shrink the search page's 'preview' window down to 0 pixel size, it now recognises it is hidden and will no longer load videos and stuff in the background! In the parsing system, the 'character set' choice for a 'String Match' now includes the quick-select options for hexadecimal and base64 chars. When hitting share->copy hashes on a thumbnail, the program now loads up the md5, sha1, and sha512 hashes into the menu labels, so you can now verify md5s or whatever at a glance. I wrote a thing for API devs and any other advanced users interested in hydrus content updates about how the Current Deleted Pending Petitioned model works in hydrus: https://hydrusnetwork.github.io/hydrus/developer_api.html#CDPP . I went into very specific detail because when talking about this with a user the other day, I couldn't remember it perfectly myself. Lastly, I am happy to say that I succeeded in doing, as I planned, some meaty background work. The 'Metadata Conditional' object, which does 'does this file have x property, yes/no?', and which I have been thinking about for a couple years, got its first version, and it went a lot better and simpler than I expected. The whole thing plugs into the existing system predicate system and will share much edit UI with it. The MC will be a key lego brick in the development of the duplicate auto-resolution system, and most of the comparison logic for the first version is essentially done now. I pretty much just have to write the database tables and maintenance daemon, and then some UI, so fingers crossed it won't be too long before I am rolling it out for advanced users to play with. next week I want to add local media file path fetching to the API and do a bit of API permissions cleanup.

>>16170 >The 'check now' button in 'edit subscriptions' and 'edit subscription' is now more intelligent and handles pause status. It'll ask you about paused subscriptions and queries, and it better examines and navigates what you can and will be resurrecting. Noice.

i'm an idiot who keeps adding the ptr thinking it would be nice to have more tags and then removing it when those tags are useless and only on files i already had tags on from boorus. anyways, my database files are fucking huge even after vacuuming them all. is there any way to debloat them?

>>16173 If you want tags, especially for files that are not on boorus, look around for Ai taggers. You'll find a discussion of them a handful of posts above. I've been using an old one that does a pretty good job. It just doesn't do characters (people or anime names and such). However, if you set the threshold to low, it will tell you whether a pic is loli or not (it will tag it loli). This is the link to the tagger I am using. https://github.com/Garbevoir/wd-e621-hydrus-tagger/tree/main

>>16178 AI taggers are the worst at the tags I actually search on so there's no point. I've tried them and it's just the same cycle as me adding and removing PTR. I like having other tags too, but I'm nearly always searching on character, gender, creator, or series. Also, it's a lie that that one doesn't do character tags, it just doesn't namespace them as character tags. Which is honestly worse because not only are they inaccurate, but they're spamming up the wrong namespace too. Circling back to my previous post, I think I just vacuumed too early after the last time I removed it, because I managed to knock off a good 10GB+ by vacuuming again. My DB files are at 19GB now, which still feels too big for the amount of files and how sparsely tagged they feel, but I can tolerate it.

>>16179 >My DB files are at 19GB now Damn. And I was surprised because my DBs were 230 MB.....

>>16180 >>16179 Haha how could anyone tolerate having such a large db, that's crazy...

>>16181 is that including files? it looks like it is. files+db for me is 203GB

Where do I look up Sankaku's http header?

>>16183 I'm sorry, I've no idea what I'm doing. I wasn't on beta page. So, now what?

>>16184 Okay, I'm even more of a retard, it logged me out. Finally found the bugger.

>>16182 It doesn't include files but it does include thumbs. i totally forgot about the thumbnails, thank you anon. I'm moving them to a different drive right now until I can get more storage for my PC.

Imageboard parsers should replace HTML entities in thread subjects for the watcher titles.

>>16188 Oof, it doesn't actually contain the thumbnails, it's over for me.

I think my db is at about 60gb now.

>>16179 I don't think my tagger does characters. I'll have to take a good look. It might.

>>16154 remember: tags are for searching, not describing. if you're never going to search it, don't bother spending the time to add it as a tag. also, if you're importing files from boorus or using autotaggers, your files are going to have, well, booru-style tags. might as well keep it consistent and use the tags that already exist; why reinvent the wheel? so in conclusion: don't listen to this dude >>16155 also, since you say you're new to hydrus, make sure you read up on tag siblings and parents. those are useful. also take a look at tag display and search, which lets you hide tags. also, under the "tag suggestions" section of the options, you can add tags to the "most used" tab which helps with tagging quickly.

>>16170 >The 'Metadata Conditional' object, which does 'does this file have x property, yes/no?', and which I have been thinking about for a couple years, got its first version, and it went a lot better and simpler than I expected. Oooh, exciting. Thanks for your hard work.

>>15721 Bro your site is so unreadable, it's crazy, I don't know how you do. Why isn't there just some simple indications on how to download the main sites, like download a 4chan thread (conserving the title of the files), 8chan, Danbooru, Safebooru, Gelbooru, Pixiv, a Sankaku Complex thread, a nhentai comic, like open menu > download > paste the adress you want to download the content of > end of story. No, instead we have >network I guess >why not download? >network > downloaders >import downloaders (that's not it) >export downloaders (that's not it) >manage downloaders default options (that's not it) >manage downloaders and url display (why repeat manage downloaders?) (that's not it) (it's just other options) >Let's try import downloader >Lain. Ok, cool? >It asks some pngs >I'll fetch them I guess >find them on Github >they're not needed, they're default in Hydrus now >If those PNGs were needed in previous versions, but aren't needed anymore, why isn't precised in Hydrus like "hey those PNGs are already available, but you may need some custom PNGs to download some sites <3"? WHY. WHY IS YOUR SOFTWARE SUCH A MESS? WHY DOWNLOADING A PAGE BECOMES A FUCKING EXPLORATION RPG? WHY CAN'T WE JUST PASTE THE ADDRESS IN A DEFAULT WINDOW ALREADY PRESENT ON THE LEFT AND DOWNLOAD THE FUCKING PAGE? >muh read the documentationnnnnnnnnnnnnn I'm literally here: >https://hydrusnetwork.github.io/hydrus/getting_started_downloading.html All I see is bullshit >blabla API >a screenshot but you can't click on it >you have to open it in another tab >Once you've opened it in a new tab, the screenshot shows nothing >just a bunch of images >and? >by reading carefully you can see, in small "gallery downloader" >how do we make this option appear? >we don't know >we have to ask on 8chan.moe so that assholes (not you Hydradmin) tell us to read the unreadable documentation Why can't we just paste a url and download its content? Why isn't there on the site a big visible tab >DOCUMENTATION or MANUAL with the big steps explained ? Like >Customize your User Interface >>To change the size of your window, do this >>To make the tag managing window there, do this >Download the content of a page >>To download the content of a page just paste your URL there and click enter WHY WHY CAN'T WE DO THIS? Have you read Audacity's documentation ?

[Expand Post]

>https://manual.audacityteam.org/ >Everything is clear, simple and readable Tell me if you need help for I don't know organizing the documentation and the interface of the site. It's absolutely not normal we can't do such a simple action as downloading the pics of a thread in10sec. How do Jdownloader, 4Kvideodownloader work ? You just paste your url, end of story, the download starts. With yt-dlp you hit add then start, and that's it. It's scandalous.

>>16195 Yeah, that sounds cool!

>>16196 LOL!! Need some help? Yeah, this program took me quite a while to figure out how to use, and there are still some dark recesses I have yet to explore. BUT IT WORKS!! Just ask for help here. Devanon is pretty nice, and answers questions on the weekends. Also, sites change, so you might have to find new downloaders, or like I do, tweak old ones to make them work again. Look up hydrus cuddlebear to find downloader scripts. >Page Do you mean an actual gallery page or a single pic? For gallery pages, you want the gallery downloader. Pick your tags, and your site, and set options. Then start downloading. For a single pic, you want the URL downloader. You can also put multiple URL addresses in at once to batch download. To get the downloader options up, hit F9. Under File, Options, you will find a LOT of options to set. Database and Network also have options under them, especially Network. You just have to explore. If you have problems, come back here and people will help you. Just ask.

>>16196 >>16198 Also, this is the manual you are looking for. It's under Help / Help and getting started guide. I think some stuff still isn't in it, you really have to explore and try out, especially since devanon is constantly upgrading and improving everything.

>>16154 Namespaces help control tag suggestion (related tags) and presentation (what tags appear on the thumbnail), but you cannot nest namespaces. Colons (:) in the middle of a tag can interfere with autocompletion, and you may need an extra action to search in all namespaces.

>>16198 >Do you mean an actual gallery page or a single pic? For gallery pages, you want the gallery downloader. Pick your tags, and your site, and set options. Then start downloading. >For a single pic, you want the URL downloader. You can also put multiple URL addresses in at once to batch download. I think the gallery downloader is for converting a search query into a URL using a gallery url generator. The resulting URL is still processed with a page parser and can used with the URL downloader, but the gallery downloader is a specialized tracking UI.

>>16196 >Have you read Audacity's documentation ? <https://manual.audacityteam.org/man/credits.html >The previous Manual for Audacity 1.2 was written by Tony Oetzmann, with major contributions by Dominic Mazzoni. The current Manual builds on that work and has significant contributions by the following: >Gale Andrews - RIP >Richard Ash >David Bailes >Christian Brochec >Matt Brubeck >John Colket >James Crook >Steve Daulton >Scott Granneman >Greg Kozikowski >Leland Lucius >Dominic Mazzoni >Edgar Musgrove >Tony Oetzmann >Alexandre Prokoudine >Peter Sampson >Martyn Shaw >Vidyashankar Vella >Bill Wharrie Nigger, this program is made by one guy, not a team.

>>16150 I think I agree. I'll add an 'empty page' bitmap button or something. >>16152 Great, I am glad you are working again. I think a regular clean install is generally a good idea. I'm not totally sure what you mean by 'get that backup running', but no worries, you know your situation better than me. If you have a backup that has a bunch of spammy folders in it, yeah, delete everything except the db folder I think and do a re-extract. If nothing else, keeping things clean is a good default philosophy. I personally like to run FreeFileSync and then my backup is just a perfect weekly mirror of my main install. >>16158 For a simple user-friendly solution, FreeFileSync has a Linux version, although I have no idea how good it is: https://freefilesync.org/ >>16162 I think I say it in the docs with a fuller explanation, but for my part, my general rules for not going crazy are: 1. Don't try to be perfect. 2. Only tag what you actually search for. Tags are for searching, not describing. IRL for me this mostly means I only manually add creator/series/character tags and a handful of specifics on my personal interests like 'meme:ogey rrat'. >>16164 >>16165 Damn, I'm sorry for the trouble. Sometimes sites just do this, and we don't have excellent solutions. Pixiv is double-difficult because they have multiple files per post, which causes additional uncertainty in our 'have we seen a file at this URL before?' checks (i.e. we may have seen one file, but we don't know if it is the one the downloader wants to get). If pixiv is completely redownloading, that suggests they have changed their actual raw file URLs. In some tests here with some old files in my dev machine, it looks like not only have we gone from https://pixiv.net/member_illust.php?illust_id=XXX to https://www.pixiv.net/en/artworks/XXX (I think this happened a couple years ago), there's a CDN direct file URL https://i.pximg.net/img-original/img/2015/11/24/04/58/13/XXX_p0.jpg turning up that wasn't being recorded previously. That direct file URL looks fairly stable, and while they may have changed it, perhaps hydrus is at fault for not properly recording this previously. Maybe this situation will not be so bad in future, if they change URL format yet again, because the direct file URL will still be able to give a pre-download 'already in db' result. Unfortunately, Pixiv offers no hashes. I'm sorry to say (and probably shouldn't, I think, given how bad my own code is!) their tech has, for a long time, been old/private/weird. I was horrified recently to discover that their main file metadata API suddenly started giving different answers for the 'translated tag' section depending on your language request header, leaving that JSON row absent if you don't send one--giving different API responses to different callers is not typical or helpful. But in recent years they have been updating to modern standard phone-friendly tech, although that has its own plusses and minuses. I'm hoping their API will get some updates, but I also fear they'll roll out some OAuth garbage that blocks us off entirely. We'll see. Anyway, I'm sorry for my part. I hope some better URL tech in future that will better navigate full renames (I have a similar issue with twitter.com to x.com URLs I want to solve). All I can say for now is to be careful, if you can, with super large download pages on content that is mostly old. But this is also, a little, the cost of doing business. Shouldn't be so bad in future, but let me know how you get on.

>>16173 Double check with database->db maintenance->review deferred delete table data. If that is empty and you have vacuumed, then I think you are currently as small as I can currently get you. I believe this means you should have fairly small client.db, client.caches.db, and client.mappings.db but upwards of a 15GB client.master.db. I am very slowly developing recycle tech on the whole database that will be able to recover orphaned definitions, which will be able to cut the master back down too. >>16179 Ah, looks like the deferred delete might have still been doing stuff in the background. >>16189 Thanks, is this with the default hydrus downloaders, and is there an imageboard that does this a lot, like everything have <br> or <p> or something? Can you point me to an example (longer-lived) thread? >>16180 >>16190 >>16191 Yeah if you sync with the PTR, your mappings will blow up to about 65GB and your master to 15GB, although my numbers may be a bit off there. Your client.caches.db will get pretty chunky depending on how many files you have in your actual client (it cross-references the new tag info). The PTR has like 2.3 billion mappings or something now, which your client will download and process into database tables and indices. The ~40bytes per row is a pretty cool ratio overall, but it obviously adds up. I generally think that the PTR is worth it if you have a decent computer and an SSD and hundreds of thousands or millions of files, along with a general tolerance for messy tags. If you want a curated db or need to preserve system resources one way or another, the PTR is not great! >>16196 Sorry it was so frustrating. I'm an unusual guy doing an unusual thing, and a lot of my work is unpolished and not user friendly. Most of hydrus is a sprawling mess of ideas rather than a tight product. For some users, it all clicks, including the documentation, and some others bounce off completely. I am ok with the idea that hydrus is not for everyone, and there are some other programs like Tag Studio https://github.com/TagStudioDev/TagStudio that particularly focus on being prettier than hydrus, which you may wish to check out, but if you would still like to keep with hydrus, I would be interested in hearing more about how you came to use hydrus so I can make the onboarding and help documentation better for the next person in your shoes. Keeping the help updated and useful for new users is always a battle. How did you find about about hydrus? Were you recommended in a thread etc..? What do you think hydrus does, generally speaking? Why do you want to use it? If you were mostly interested in the downloader, did you jump, in the help, straight to the downloading page? Have you tried, say, importing 100 files to the program and exploring how the search works? I generally see hydrus as a file management program before being a downloader, and since the program is complicated I generally recommend people take it all slow. While the downloader is important, I generally suggest that people see if they enjoy the general workflows of managing files before they commit too hard to the program. I suggest, I think in the help, just importing 100 or 1,000 files to the program, when you are starting out, and learn the basics of archiving and searching things like system:filesize. Once users get a feel for managing files and simple local tag editing, then I think it is a good time to explore things like downloaders and subscriptions. If you jumped to downloading first because that was what you were interested in most of all, it sounds like I should add more to that help page about the basics of opening pages, things like the F9 menu. and perhaps I can add a warning at the top of the page to say 'hey, if you skipped to this page, please go back to getting started with files'. By your feedback, I should add more concrete examples of downloading things--sounds good.

>>16196 >entitled brat has a meltdown >really thinks Customer Service owes him something Ahem. Dude, did you know this circus is ran by a single anon, for free?

>>16196 Cringe.

>>16205 >>>16189 > Thanks, is this with the default hydrus downloaders, and is there an imageboard that does this a lot, like everything have <br> or <p> or something? Can you point me to an example (longer-lived) thread? My customized 4chan downloader titles at least with apostrophes and ampersands, not ">" https://boards.4chan.org/mlp/thread/41431332 not on desuarchive

>>16205 >I generally see hydrus as a file management program before being a downloader, This. I use Hydrus exclusively off-line and is a blessed piece of software. Thanks.

>>16194 >>16201 >>16204 >Tags are for searching, not describing Perfect. Somehow along the way I got lost in the mindset of getting every single descriptor tag down, but it doesn't make any sense in this context. Ty!

>>16210 Yeah, tags are definitely far more for the search. You don't really care usually if some pic has a black bowtie. But you might be searching for a pic that you remember the character was wearing a black bowtie and holding a chalice. And when you have over 5 million pics... This is a program for organization autists. If you want to just download a pic, I recommend something like gallery-dl.

>>16210 >>16211 And the other thing I use this program for is mass importing of files for tagging (I use an auto-tagger), and mass downloading from boorus with tags. And after I'm done, everything is all nicely organized in it's own viewer. Very Nice!!

>>16204 > FreeFileSync has a Linux version, although I have no idea how good it is as far as i can tell its as good as the windows version, and donation edition transfers. my only complaint is that it doesnt make a sound when its done anymore, but i may have missed that option >>16205 > I am very slowly developing recycle tech on the whole database that will be able to recover orphaned definitions, which will be able to cut the master back down too. awesome, thanks! I've been using hydrus for something like 4 years now and I've been lugging around ghost ptr bloat for most of them, so I can wait until then to get my 10gb or whatever back. thank you for your hard work as always

I can never find a straight answer for this. Does Hydrus supports ugoria from Pixiv or not? I see its listed as supported but whenever I try to download ugoria off Pixiv, my gallery downloader says "vetoed" and skips it. That place is such a hellish site for scrapping off of. That site can make any scrapper software or script a complete mess to work with.

>>16214 Ugiora's aren't supported, but Hydownloader and gallery-dl can convert them into mp4s which hydrus can use. I think ugoira support is in the cards but it's not in hydrus yet.

>>16170 >590 "share" in the context menu is missing "export".

>>16170 Maybe "files" should be called "file service". It is no more about files than "1 png, 127 KB", "open" or "share".

>>16133 ty >>16146 i would really love the tooltip option, i prefer having ISO but having both is very nice

I had a mixed week, but I got a couple of neat things done. I fixed a handful of bugs, wrote a 'tag actually exists here' filter for tag siblings/parents export, and added local file/thumbnail path fetching to the Client API. The release should be as normal tomorrow. >>16216 Thank you, fixed for tomorrow!

>>16196 >typical modern faget blaming others for his own shortcomings >millions of them everywhere expecting the world to cater their whims Positively this is the result of a home without a father. You have to go back to Reddit.

How do I lock tags for a file once I'm sure it's definitive so that new imports from boorus don't fuck with it? Is creating a separate tag service the only solution?

>>16222 Afaik you go to network -> downloaders -> manage default import options. Then either you set one of the two defaults for watchable/post url on top right of the window, or you double click on an entry in the list and set custom settings for this entry alone, let's say "danbooru file page". There you activate or deactivate the checkboxes that you need like in the image of following post: >>15842

https://www.youtube.com/watch?v=1ySAkB-H33E windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.macOS.-.App.dmg linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v591/Hydrus.Network.591.-.Linux.-.Executable.tar.zst I had a bit of a mixed week, but I got a couple of neat things done. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I fixed a stupid mistake in last week's 'move left/right after page close' thing, which was doing it even if the page being closed was not the current focus! Also fixed is the share->export menu not showing if you only right-clicked one file. The tags->migrate tags dialog can now filter parent and sibling tags based on tag count. You can say 'only include pairs where the A/B actually exists on service x', where for siblings that B is the ideal tag at the end of the chain. I hope this makes it easier to filter giant multi-hundred-thousand pools of pairs, like from the PTR, down to only what matters for your 'my tags' etc.. The Client API has a new simple permissions mode for access keys, a convenience state just called 'permits everything', which does exactly what you think and is intended to be an easy catch-all for people who just want to turn on future-proofed access for an app they trust. On update, any access key that looks like it was told to permit everything in the old system will be updated to this mode (you will get a popup saying this happened, too). If you do need finer control, you can still tweak individual Client API permissions under services->review services. Relatedly, the Client API gets a new permission this week, 'see local files', and commands to fetch local file and thumbnail paths. If you want to access files directly on the local machine for metadata analysis or whatever, this is what you want! next week I'm sorry to say I have been unproductive recently, so I'll keep it simple. I think I want to try some duplicate auto-resolution database stuff. I still need to think about a couple things, so I'll play around a bit and see what makes the most sense. Fingers crossed, I'll have the skeleton of the maintenance daemon sketched out.

>>16221 >a home without a father Is that the meme of the week or something?

>>16225 "fatherless behavior" has been a normalfag meme for over a year or more now. And also a minor nig meme. This is just a variant.

>>16225 Do not take it lightly, single and divorced mothers are a civilizational problem and revoking the emancipation of women is long due.

>>16224 Wow! I've been waiting for that relationship filter feature for over 3 years! I didn't expect it to just be added out of no where thanks a bunch! If I could ask for a small (I assume it's small) option though, I'd like it if the "only" checkboxes could have an "OR" option or something like that. Basically what I mean if that I'd like the siblings or parents to be added if either one side or the other has counts, rather than it needing to be both when I check both boxes. Or is that already how it works? I just wanna make sure before I pull the trigger. but anyway, I'm really glad this is finally here!

>>16224 I appreciate the "for convenience, I moved stuff around in the access permissions" on upgrade to 591, both the action and the warning so I'm aware without reading the full changelog. Thanks for treating us so well!

>>16228 >Do not take it lightly, single and divorced mothers are a civilizational problem Who is picrel for? > In all OECD countries, most single-parent households were headed by a mother. The proportion headed by a father varied between 9% and 25%. It was lowest in Estonia (9%), Costa Rica (10%), Cyprus (10%), Japan (10%), Ireland (10%) and the United Kingdom (12%), while it was highest in Norway (22%), Spain (23%), Sweden (24%), Romania (25%) and the United States (25%). These numbers were not provided for Canada, Australia or New Zealand. https://www.oecd.org/content/dam/oecd/en/data/datasets/family-database/sf_1_1_family_size_and_composition.pdf > and revoking the emancipation of women is long due. In Russia, fatherless families are more likely to be ruled by the mother's parents, and old women are often sadomasochistic slavers who love strong men, illusory rules and hierarchy, and know that they will not be imprisoned or sent to war. If you are talking about the right to vote, guess what.

>>15916 >>>15901 >>The derpibooru downloader downloads descriptions without the links in them. :( >I will check it out. I don't know how this thing works, but if it is just pulling the visible 'text' of the html, and the URL you want is in <a href="xxx">, it may be tricky to get that in a neat way. The hydrus note tech is only plaintext for now, so no proper rich text or links or anything yet. The example urls in the "default" one have no interesting descriptions. Here is one with a link and a quote (the ">" becomes an entity): https://derpibooru.org/images/2680312 The link is preserved. The quote is not. I may have been looking at a description fetched from a mirror that had lost the link. If there is still a reason, there is a hidden form that contains a textarea with the Markdown source of the description.

>>16236 >with the Markdown source of the description. with entities for ">", at least

>>16236 >The link is preserved. The quote is not. Sorry, I meant the leading ">".

tangentially related to Hydrus but Mozillia recently announced renewed interest in adding Jpeg-XL (JXL) support to Firefox, thanks to a new rust-based library for it that a Google research team is developing for them (crazy, I know). also the JXL website got an overhaul and looks really nice, even including a direct jab at AVIF.

>>16236 Sorry for rushing and not reading or thinking. It is indeed about things linkified with href. Another issue is that images included in the description leave no useful trace. An image included from any site directly like in https://derpibooru.org/images/3448782 is ignored completely. An image included from derpibooru itself like in https://derpibooru.org/images/3450178 is replaced by "This image is blocked by your current filter - click here to display it anyway your current filter.", although there is no filter that should block it.

>>16240 >An image included from any site directly like in https://derpibooru.org/images/3448782 is ignored completely. and note that the image is included using derpibooru's proxy, and original url appears in the invisible Markdown, but not in the HTML.

>>16239 >google developing a rust library for mozilla to use >same google that has been stonewalling jxl >same google that made jxl ???? I know google is big and all but I would have thought that the teams would at least appear to be in agreement in public.

>>16242 This is how I understand the situation: The "research" team is very much a fan of JXL and wants to see it adopted. The chrome team doesn't want JXL because (I think) they don't want the maintenance burden and don't care about the benefits of a new format. The problem is that the chrome team has much more clout with the heads of Google than the research team does, so they usually get their way, and they did with Chrome removing JXL support. The research team still believed in JXL even after Google as a whole was no longer backing it, so they asked if they could write a new library for JXL in a memory safe language (Rust) to hopefully reduce the maintenance burden and security risks of supporting a new image format. The Google heads said that they could, but that Chrome still won't support it even if they do, and they're only allowed to work on it if some other meaningfully significant project would use the library. The team knew that Mozilla was somewhat interested in adding JXL support to Firefox, but that Mozilla was apprehensive about fully committing to it, because of the security risks of merging a large library written in a memory-unsafe language (C++) into their flagship browser's stable release. So the research team contacted Mozilla to see if they were interested in Firefox being the project that the research team needed to get the go-ahead from their higher-ups, and Mozilla agreed. So here we are. source: a bunch of reading on different websites about the topic that I didn't save the sources for because I'm not a journalist. I wish I did save the sources though. It's an interesting situation.

>>16241 It is very simple and the ">" appears as ">". [30, 7, ["description from Markdown", 18, [27, 7, [[26, 3, [[2, [62, 3, [0, "textarea", {"id": "description"}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]]], 1, "href", [84, 1, [26, 3, [[2, [51, 1, [2, "^((?!No description provided\\.).)*$", null, null, "pony"]]]]]]]], "description-md@derpibooru"]]

>>16245 ponerpics uses a different syntax https://ponerpics.org/images/7063333?q=blood+cell the twibooru page does not have any source code in it https://twibooru.org/3344908