Just to update from the last thread, I have some personal business going on right now, so I am working on hydrus stuff at odd times for a bit. I will be back to normal schedule from the 24th and will have v500 out with a full changelog on the 28th. Thanks everyone!

>>18265 Question about file import: does Hydrus still copy the file when importing if "delete original files after successful import" is ticked? or does it simply move the file?

>>>/hydrus/18257 I noticed you referred to your mom as "Mum." Are you in the UK?

>>18259 >>81213 here. I've been running from source for a bit now, and it seems to be going well. I haven't had any more crashes. I'm using PySlide2 and LibMPV 1.0.1 with mpv-dev-x86_64-20220501-git-9ffaa6b, and it seems to be doing really well. >>18261 I use ZeroTier One to access my shit when I'm away. I wouldn't call it an easy set up, but it's definitely easier than all the other shit. The only drawback I've seen is that, on Android, you have to run it as a VPN, which means no real VPN at the same time.

>>18268 Damn, copied and pasted from the old thread without thinking when I saw that it was ended. For fixed reference, >>/hydrus/18259 >>/hydrus/18213 (not even sure how the 81 happened) >>/hydrus/18261

>>18269 Fucking hell. >>>/hydrus/18259 >>>/hydrus/18213 (not even sure how the 81 happened) >>>/hydrus/18261

>>>/hydrus/18215 : >>Upgraded from v497 QT5 -> v499 QT5. >>Upgrade process is to copy files from the zip, overwriting all files in the hydrus destination folder. >>Video files do not play in the client now, they all say to open externally. I get a message saying 'MPV is not available' >> >>Rolled back to v497 QT5, all appears well again. >>>/hydrus/18260 : >>Thank you for this report. I am sorry for the trouble. It looks like the new mpv is causing a variety of issues. If it is convenient, could you >>try extracting the Qt6 version on your desktop? Just make a fresh install of Qt6 499 and try importing a video to it. Does it play? If it does, it could be Qt5 doesn't work well with the new mpv. If it doesn't, there's a deeper mpv problem going on, and I'll ask you to hit help->about, which should make a popup with more detailed information about why mpv could not load. >> >>If nothing works, you may be able to update to 499 but delete the new mpv-2.dll that comes with it. The old mpv-1.dll that is probably still in your install folder should be fine, but if it is not, that is also useful information to know. OK, so I tried putting 499 QT6 into its own folder, and the client would never start. I see it show up in task manager for about 10 seconds, then it closes out. No visible GUI. No logs in the DB folder. Then tried 499 QT5 in a new folder, and this would start without issue. I'm running on Windows Server 2016, no idea why the QT6 client doesn't like it. When attempting to import a file in the 499 QT5 client, it crashed out.picrel

I swear I've seen this option before, but how to make the image viewer not going back to the first image after reaching the last file? In other word, how to make it not loop back?

>>/hydrus/18262 >>Try extracting a fresh version of v499 to your desktop, boot it up, and only import some jpegs. Is that stable? If you then import some videos and load them up, does it suddenly become unstable? For me, totally fresh v499QT6 under windows 10 19403 still crashes immediately when attempting to open any explorer dialog, including clicking the add file and add folder button under import. Including when using the QT file selection option. Files imported by dragging to the client from explorer (pictures and videos) open and play fine. Files imported via the url importer also work fine. Files, both pictures and videos, downloaded by a gallery downloader immediately crash a fresh 499 client.

>Files, both pictures and videos, downloaded by a gallery downloader immediately crash a fresh 499 client. I'm experiencing the same thing.

>>>/hydrus/18258 >>The idea of going back to check for new metadata has long been a request, and I've been hesitant to do a quick bad job because, as said, I don't want to hammer sites or let someone who doesn't know what they are doing redownload 2 million html pages every month just to capture 5 new tags. Wouldn't this be a good application of the PTR? Have a single server/service be responsible for occasionally checking metadata from the sites that support tag export, and merge that into the PTR, so clients pull in new tags from there.

>>18275 That would kind of redefine what the PTR is as a whole, make it a sort of active participant, whereas right now it's basically just storage and distribution.

>>18276 Not necessarily, the service could just submit PTR petitions for newly imported tags like a regular user would.

>>/hydrus/18257 >In general, false positives tend to be fairly rare. Are you finding you are getting a lot of them? Can you say what search distance you have searched? Could these be being found at, say, hamming distance of 10 or more? Distance of 4, actually. This has mostly been an issue for sequential shoots in photo sets, where many images have more or less static backgrounds and the subject of the shoot's pose doesn't change drastically from image to image. Personally, it's not a very big issue; I was just wondering if there was a better way for me to do things in the dupe filter. Although, I guess it would be a use case for you to consider, perhaps.

>>18266 It doesn't delete the file until it knows the import is successful, so I think the order of operations is: - copy file to your temp dir - do import calculations and checks - copy file from temp dir to your hydrus file structure - delete temp dir file - do post-import cleanup jobs (including 'delete source file') I can't remember if it does copy/delete for the temp dir->hydrus step or if it does a move. I forget if there was ever some pain in the ass permissions thing that made me do a copy/delete instead. If you are interested, I copy to temp dir to start with to ensure your file has a nice filename and a non-unicode directory path, since some file-scanning tools I use like ffmpeg and opencv have a history of not being able to deal with, let's say, a file on your desktop that has a path that includes a scandanavian-charactered username. >>18267 I was born in the UK and moved to the US with my family when younger and became a citizen. You might notice me writing 'favourite' and other UK spellings in the hydrus UI. >>18268 >>18270 Thank you for letting me know. I have had several reports of mpv causing crashes or just not loading, and I regret it all happened now. I've had no trouble myself apart from a weird thing where my private build I make every week with a bat file caused swift mpv crashes with mpv-2.dll while the github build works great for me. I have some ideas from you and others, so I'm going to see what I can do in code. Some of this may be PyInstaller related though. I might roll back the dll delivered in the release, I might just roll back Qt5. I still need to think and gather my evidence together to find the common factor. I'm going to keep my eyes open for the potential easy-virtual local network. I guess that's really what 'vpn' is supposed to stand for and be, whereas we really use it to mean 'proxy' most of the time these days. I thought the days of LAN parties were over, but if this tech becomes simple and you can bring friends into your local network on a particular subnet, then hey you could run a quake or private MMO server for your friends, whatever you wanted, and have all the technical bullshit handled for you. >>18271 Damn, I am sorry again. I'm getting confident now that the latest version of mpv uses new OS tech in the same way Qt6 does. I know Win 7 users can't run Qt6 applications, and now it seems like Windows Server 2016 can't either. I am not an expert in Windows Server, but it may be that an optional Service Pack may get you working with Qt6 applications. (Even now, Win 7 users need to install "Visual C++ Redistributable for Visual Studio 2015" just to run the Qt5 version of the program.) Several others have been moving recently too, like OBS and qBittorrent, so if this seems to happen again, you know what to look for. Ok, what I'm going to try is to roll back the python-mpv and the mpv version on v500 for Qt5. That will restore you to the tech that we know works in <499. mpv 'version 2' is probably just going to be something that needs a newer OS. I am going to be updating my 'running from source' help and the help in general as we transition to Qt6 to talk about all this, so this is helpful information to roll into that. I regret the inconvenience, and that it happened now.

>>18272 No, I'm afraid I don't remember ever adding an option for that. The archive/delete filter works that way, if that is somewhat what you are thinking of. If it helps at all for quick navigation, home/end should get you to the first and last position when you are in the normal browsing media viewer. >>18273 and >>18274 Damn, this sucks. Just to be precise, is that version 19043? Not 19403? My dev machine here says 19044.blah. I was wondering if you might be in one of those Windows preview betas, but it doesn't sound like it. I am going to put out a Qt5 version of v500 on the 28th, and I will roll back the mpv version to something more stable. Please do a clean install to that. If you have not rolled back to a pre-v499 client, I am confident you can clean install roll-back to 498 as well, Qt5 again I think. The client will moan at you about the version difference when you boot, but I don't think it will give you any errors. https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs It seems like there are going to be some users who just cannot run Qt6 for some local OS reason. I regret and am surprised that some Win 10 users are even being caught. I want to see if I can stamp it all out, but if the errors are as 'core' as clicking on thumbnails or loading simple dialogs, then I think the actual problem must be related to PyInstaller (which bundles the executable out of my code), Qt, or Windows. I am going to be writing more in my 'running from source' help section, and I'd love to make some .bat files that do all the technical stuff for you and make it very easy to set up. You seem like a candidate for this. Please keep me updated on how you get on. >>18275 >>18276 >>18277 I like my servers dumb, so I'll keep the PTR as-is. However, if a handful of hydrus users set up re-parsing of their known urls and then upload what they get to the PTR, then all users who sync with the PTR benefit from that too. We actually already have this essential system due to new users building up their collections. When new users start/backfill their collections, they put 'incase' and 'samus_aran' into their downloaders and download lots of files and tags from the latest snapshot on the site. Any gaps in the PTR's knowledge are filled in then, and of course all users who sync with the PTR are appraised of those updates automatically. So you get the latest snapshot of tags without having to hit the site yourself. This is another reason I am less hurried to get this tech done. If you are a super tag enthusiast, the PTR solves a bunch of problems fairly efficiently. It is messy though, and tastes differ, so if you want your own private walled garden, you'll need the background routine. >>18278 Thanks. This is something to think about. I guess I might personally set those as alternates, and I'd set them as some sort of related group later, but I know people see all this differently and it sounds like we need an efficient answer here. Also, now I think of it, large alternate groups add about the same amount of database cruft as large 'not related' ones do. I will make a note and see what I can do. I feel fairly confident I can just add a new action (and shortcut action) and an efficient database scan like I talked about to stop us going bonkers with ten million database records clogging things up.

I have a question about the PTR. Is it stored in one of the main db files or in its own db? I often make file-based backups, and since the dbs change every time I have to back them up each time. If the PTR is in a different db, I can just set my backup software to ignore changes to ptr.db or whatever since it's not as important to keep backed up as the others. If it's inside another db, I would have to basically sync the entire PTR to my backup machine every time I made a backup.

Might be a stupid question, but when i was using a chromium browser, i could click and drag images into Hydrus and they would import without issue. I recently switched to firefox and the drag and drop method no longer works. anyone have a reason for why this is the case?

IF YOU HAVE v499 MPV PROBLEMS >>18271 Please try replacing the mpv-2.dll in your client install directory with this one: https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20220501-git-9ffaa6b.7z/download There's some other bullshit in that archive, but you only need the mpv-2.dll . Another user said that one loads whereas the newer one doesn't. This may reduce some crashes too. I'm going to roll back to it for all clients next week. >>18268 Just noticed that I picked the same 'old and safe' mpv you chose when I scanned down the sourceforge list. >>18281 Unfortunately for your case, the PTR data is all mixed in with everything else in your .db files. Since it will blow up to several dozens of gigabytes when you sync, you will be looking at backing up that much data every time you run a backup. Not so much a problem on a USB drive backup, but yeah if you can only back up to a slow network location, you may want to be careful with the PTR. >>18282 I thought I had fixed this. iirc, when firefox exports an image via drag-and-drop, it actually exports a bitmap and/or a bmp file to your temp directory, rather than the file itself, but I thought it also attached other metadata types to the drag object, and I thought I was able to grab one of those instead. I will look at this again. Can't promise anything, since it depends on what firefox gives me.

My PTR account just got a 30-day ban for "uploading Sankaku url tags". I don't really know how this happened, but it is possible that I forgot to remove such a tag when I was copy/pasting tags from variations. Is there a log or similar somewhere where I can see, for example, what tags got uploaded and when?

>>18284 I also got this.

could you add a way to give files in collections a different sort than the collections themselves and files outside of collections, so that (for example) you could have a search sorted randomly, but still have any files that are in collections properly ordered by page, instead of also being sorted randomly? maybe It could be another set of menus next to the collection menus on the file screen. An option like this would make collections a lot better for me.

Is there a way I can list all creator tags by the least number in my database? I want to go through some images by artists that I only have a few images of.

>>18279 >I was born in the UK and moved to the US with my family when younger and became a citizen. Interesting. Someone add it to the lore! 'Merica!

>>18283 >>>18271 >>Please try replacing the mpv-2.dll in your client install directory with this one: https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20220501-git-9ffaa6b.7z/download >>There's some other bullshit in that archive, but you only need the mpv-2.dll . Another user said that one loads whereas the newer one doesn't. This may reduce some crashes too. I'm going to roll back to it for all clients next week. Replaced mpv-2.dll in my 599 qt5 install test dir, and so far no crashes. Imported about a 100 images, all good.

Small but useful suggestion: add buttons to the connection part of the options page near proxy settings to test them. Maybe it could scrape ipleak.net/json/ or similar and show the results to the user, one button for each http/https/no_proxy.

Would keeping memes and personal photographs separate be a good use of separate file services?

>>18291 Yes, that kind of separation is exactly why multiple file domains were introduced.

Is there a way to export files with filenames that will sort in the same way I have the files sorted in Hydrus? Using file ID won't work for me and I've got mixed filetypes so the system isn't able to take care of it for me. In the same vein, is there an easy way to add page tags to images?

>>18286 not exactly, but there is a kind of workaround. in options under sort/collect, make the primary sort put titles together and the secondary sort random and set collect to leave unmatched. first pic related. the downside is that all collections will be at the end of the page, but at least they're in order. >>18287 search for "creator:*" and then change the sorting of the tag box at the side to sort by count, fewest first, group namespace. then scroll down until the creator tags. second pic related. >>18293 >Is there a way to export files with filenames that will sort in the same way I have the files sorted in Hydrus? that would depend on how you have the files sorted in hydrus... if you imported them in order, i think you can use {file_id} in the export window. pretty sure it mostly just counts up. if you're asking about using some other metadata like import time in the filenames, i don't think you can do that. for just arbitrary files placed next to each in a page manually, perhaps if you export in order, your file manager could sort by file creation date or something like that? i don't know if that would work. >In the same vein, is there an easy way to add page tags to images? unfortunately, there isn't a shortcut that's like "tag these files with page numbers in the order they're in" or anything like that. would be nice.

Does anyone keep their screenshots (anime etc) in Hydrus? If so, how do you organize them?

>>18293 >>18294 >Is there a way to export files with filenames that will sort in the same way I have the files sorted in Hydrus? Turns out it was easy to implement. I've opened a pull request on the hydrus GitHub, if you want to check it out.

Since it would have been interesting to know when I was considering adding the PTR, I thought I'd give a little update on my PTR sync progress. 10 days (and 5GB of download) in I'm apparently 75% of the way through. So I should be all synced up by the weekend hopefully! One thing that I did notice is that when the PTR update happens creeps forward every day, maybe 45 minutes to an hour. Presumably this is because either hydrus or the PTR starts counting down the 24 hours until I can download again from when it finishes, rather than when it starts. It's not a huge issues, but it does mean that what started as a midnight/bedtime job became 9am job that I occasionally interrupt with CPU usage.

>>18297 It's possible to download the entire PTR in one shot. Increase the daily bandwidth limit for services. Processing still takes some time. From experience this isn't necessarily a great idea though. I had my db on a poorly cooled NVMe, which the sustained activity of the PTR merge for hours on end pushed the temps to 85C for extended periods, eventually causing a disconnect and the loss of a thousand or so cells of data across all 4 db files. It was recoverable with an sqlite clone, but ended up needing manual deletion of a couple dozen rows of bad data using sqlitestudio to recreate a unique index and then regenerating the tag storage mappings cache.

I don't know if pixiv ugoira zips can be a vector for malware, but after downloading a lot of them with hydownloader and starting an import into Hydus I'll occasionally get zip files flagged as either Wacatac.H!ml or Wacatac.B!ml, both in the hydownloader DB directory and a second time as a hydrus temp file.

mkdocs-material won't build for me using an AUR manager. Says I need to use pip to install any later versions. Will this affect Hydrus when I update to the next version?

can I access the Hydrus database of another computer/server from a different network from within the client?

I have a lot of pages open, so when I open the command palette with Ctrl + P, my client always lags for a second. Then as soon as I type a single letter, it lags again when it searches for that single letter in all the pages. Can there be an option to not search until at least 2 or 3 characters are typed? It would make it way less of an annoyance to use.

Version 499, running from source with faulthandler module enabled. Getting either of these tracebacks sometimes when I petition to delete some tag siblings: a) object of type 'NoneType' has no len() File "<hydrus dir>\hydrus\client\gui\widgets\ClientGUICommon.py", line 249, in EventButton self._func( *self._args, **self._kwargs ) File "<hydrus dir>\hydrus\client\gui\ClientGUITags.py", line 4506, in _DeleteSelectedRows self._UpdateListCtrlData() File "<hydrus dir>\hydrus\client\gui\ClientGUITags.py", line 4685, in _UpdateListCtrlData self._tag_siblings.AddDatas( all_pairs ) File "<hydrus dir>\hydrus\client\gui\lists\ClientGUIListCtrl.py", line 456, in AddDatas self._AddDataInfo( ( data, display_tuple, sort_tuple ) ) File "<hydrus dir>\hydrus\client\gui\lists\ClientGUIListCtrl.py", line 192, in _AddDataInfo if len( text ) > 0: b) object of type 'NoneType' has no len() File "<hydrus dir>\hydrus\client\gui\ClientGUITags.py", line 4685, in _UpdateListCtrlData self._tag_siblings.AddDatas( all_pairs ) File "<hydrus dir>\hydrus\client\gui\lists\ClientGUIListCtrl.py", line 456, in AddDatas self._AddDataInfo( ( data, display_tuple, sort_tuple ) ) File "<hydrus dir>\hydrus\client\gui\lists\ClientGUIListCtrl.py", line 192, in _AddDataInfo if len( text ) > 0: Also getting this one when I click the 'browse' button for 'folder path' in the 'edit import folder' dialogue, and no file/folder browsing dialogue opens at all: type object 'QFileDialog' has no attribute 'Options' File "<hydrus dir>\hydrus\client\gui\QtPorting.py", line 175, in _Browse options = QW.QFileDialog.Options()

Is there a way to prevent my subscriptions from downloading files unless they're a certain age (e.g. 7 days old)? The site in question is e621, and I want to make sure that the tags on the posts have been thoroughly finished before it's grabbed, e.g. I don't want to grab a post that was just posted and barely filled out with proper tags. In order of preference (top is more preferable): 1. No additions to the query, but client-side rejection of posts below a certain age 2. Universal way to add "age:>7days" to the e621 downloader, or to a group of subscriptions etc, without having to do it on an individual level 3. Adding "age:>7days" to every single subscription I have (I've already done this, was easy with export/import and some find and replace. I think this isn't very nice to the server though, since they almost certainly don't have this query cached?)

Sorry to hear about your loss dev, take the time you need. Since the old thread is gone, I want to quickly mention the need for incremental ratings/counters again. No need to rush, but I hope it has a realistic chance of getting implemented.

>>18284 >>18285 I also got a 30-day ban for "uploading Sankaku url tags". I don't use Sankaku so the only way I could get Sankaku url tags would be if I got them from the PTR to begin with! I assume that means the url tags got copied from one file to another while going the duplicate files. I don't see why this would be a bannable offense when PTR still hasn't banned the dumbass that keeps tagging everything as "guro". I've petitioned the removal of the guro tag from over 50k files at this point and still have to go thru a few hundred files a week to petition removal of that stupid fucking tag. What's the point in contributing to the PTR if it's just gonna ban me for tags that PTR gave me in the first place but PTR won't deal with someone intentionally tagging tens of thousands of images incorrectly?

>>18306 got the ban as well, don't even download from sankaku

>>18284 >>18285 >>18306 >>18307 The head janitor was trying to get the guy who originally parsed all this stuff, but it seems it caught up a bunch of other users who uploaded some via the duplicate filter merging tags across. I hope to have a nicer fix for this whole problem before the end of the year with a serverside tag filter. Please hit up services->manage services, go into your PTR entry, and then click 'check for automatic account creation'. It will generate you a new account, and if you then hit 'refresh account' under review services, you should all be set up with your fresh account. The guy also banned another janitor doing this :^) >>18286 Thanks. Yeah, I would like this. I'm planning a rework of the sort/collect controls on pages, exposing the secondary sort to whatever you like, and in that same brushstroke I could add a 'inside collections' sort override. I can't promise when I will have this done, but the basic overhaul is on my mind. >>18289 Thanks. Several users have been fixed with this now. Sorry again for the trouble. I broke my golden rule about never rolling out bleeding-edge library updates, and I paid for it. >>18290 Great idea, I will queue that up.

>>18296 Thanks. I have an unusual dev situation, both technically and socially, so I don't normally take pull requests, but I appreciate this work and it is simple, so I will fold it in one way or another. >>18293 >>18294 Yeah I keep meaning to write a dialog to do quick # tagging. It'll happen on a medium size job week one of these days, please keep reminding me. >>18295 I import my vidya screenshots and video capture to hydrus. I tag the vidya series they came from, the fact it is a vidya memory, and if I took it myself. I regularly go over old memories by searching for a broad term with a low system limit to sample from a wider pool randomly, like this: vidya memory system:limit=16 So I don't need much additional organisational metadata. If you want to present your screenshots with more metadata, I guess you'll need more tags, maybe something like episode: or chapter: namespace. If you need to do time based sorting, then sorting by 'modified time' usually works really well for files you imported from your hard drive. I can load up every screenshot I took playing a particular game, sort by modified date, and get the whole game sorted from beginning to end. >>18299 Damn. I'm not sure how Pixiv ugoiras are actually constructed. If you make them by uploading a list of jpegs and timing data to pixiv.net, then I'd imagine they are all trustworthy, but if you actually upload a zip and some timing json to the site, then I guess the user could potentially poison the upload somehow. I feel like the former is true though. Right now, hydrus does not inspect any archives it imports. It doesn't consult file header or anything. So at least on the hydrus end, there's no actual worry. But the anti-virus sounds like a pain. What happens when your anti-virus flags the file? Does it fuck up the hydrus import? Can I do something, or present a nicer error message, to handle that nicer on my side of things?

>>18300 I don't make the AUR package so I'm not sure, but mkdocs is not strictly needed. It compiles some templated help files into the pretty html. I suspect the AUR package script will fail though if and when you try to update. This breaks the seal on your warranty, since there might be a good reason your OS has an older version of mkdocs, and I don't know how the AUR virtual environment works, but if you feel brave, you could try opening a terminal and going: pip3 install --upgrade mkdocs-material >>18301 Not perfectly, but there is limited support for pulling a client's data using the Client API. https://hydrusnetwork.github.io/hydrus/client_api.html In the future, especially as the Client API fleshes out, I will write a module for the client itself so it can browse another client's data in the normal search interface. Obviously there will be complicated permission and network stuff to set this up, but my ideal will be you'll be able to just select 'my nsfw client's files' instead of 'my files', and it'll all be seamless. >>18302 I didn't write the command palette, but I will see what I can do! >>18303 Sorry for the sibling trouble. I messed up, and I have it fixed for v500. For the file dialog thing, I know this sounds weird, but can you do the mpv dll thing I talk about here: >>18283 I know some other users who were unable to open file dialogs, and it seems it might be that dodgy mpv dll was breaking it. I guess it was messing with Qt initialisation somehow. Just download the archive and extract the mpv-2.dll on top of the one in your hydrus install directory. Please let me know if that fixes the file dialog issue, too, as I am not certain if that is it yet. >>18304 Thanks, this is a really interesting idea. I like your different ideas in solving it. There's some small technical issues with detecting how old a file is without actually visiting it, but I can probably bodge a decent solution simply by not touching an import row until its creation time is x days old. >>18305 Thanks! I want it too now, having had the conversation again. I do a whole bunch of visiting old memories, and I like seeing total view count history and things on ancient files. I have the idea written in my 'medium size job' list, which I try to hit up every four weeks. About my Dad, I'm back to schedule starting today and feeling ok. I'm not as upset as I expected, certainly not debilitated beyond the first day, but man sorting all the life and family stuff out is simply stressful and exhausting. Just a bunch of paperwork now, and that'll be spread out over weeks and months.

>>18310 >I don't make the AUR package so I'm not sure, but mkdocs is not strictly needed. I'm not the person you're responding to but oh man the amount of trouble that mkdocs-material has caused with the AUR version over time has been annoying, and the amount of child dependencies it pulls in is a bit excessive as well (especially dependencies that also need to be built from the AUR). I fully support whatever you need to do to keep the program working, but It's one of the main reasons I want to switch to the hydrus flatpak (which I've tested and is not working very well, unfortunately) or the hydrus docker (which I've actually tested on my NAS, but it's not clear what client I should be using to browse remotely besides VNC) >Not perfectly, but there is limited support for pulling a client's data using the Client API. https://hydrusnetwork.github.io/hydrus/client_api.html >In the future, especially as the Client API fleshes out, I will write a module for the client itself so it can browse another client's data in the normal search interface. Obviously there will be complicated permission and network stuff to set this up, but my ideal will be you'll be able to just select 'my nsfw client's files' instead of 'my files', and it'll all be seamless. I'm also not this person but this answers my docker client question from above. I don't stop into these threads much but I've been using hydrus for some years now and I really love it. thanks dev.

>>18310 >For the file dialog thing, I know this sounds weird, but can you do the mpv dll thing I talk about here: >>18283 Still getting the same traceback for the file dialogue, I'm afraid. Just for the record, I'm running Windows 10 21H2, and have 'QT_API' set to 'PyQt6'. Also see the attached Python package summary.

>>18308 >It will generate you a new account, and if you then hit 'refresh account' under review services, you should all be set up with your fresh account. PTR doesn't do IP bans?

Two questions: 1. Up until now I used an "artist:" namespace, before running into the problem that some files don't really have artists, and I ought to use the "creator:" namespace I had since learned about. Is there a way to directly convert one namespace into another? 2. At first I didn't tag many of my files with artists, but most of them have "filename:" tags with filenames from Pixiv and similar sites, and I already had a script written to get artist info given such filenames. If I generated a file (say, a csv or something) with a list of filenames and corresponding artists, is there a way to feed it to Hydrus and say "for each file that has a tag from column A, tag it with the corresponding entry in column B"? I mean as a one-time, "real" tag rather than parenting. I'm guessing not, since that's probably way too specific and niche. I know there's the txt file option when actually importing, but not to update existing files.

What's your workflow for tagging personally curated collections of files, anons? I never use any mass downloaders. All my files going into Hydrus were at some point manually selected by me. My tagging process is, >Choose a folder to start tagging, preferably under 500 files >Go through every image that could exist on a booru and verify it via SauceNao >Use url downloader every time there's a hit >Import the folder into Hydrus while mass tagging all files with at least 1 tag based on the folder it came from >Delete inferior dupes and make sure the booru tags are moved to superior ones if needed >Archive all url imports that need no further tagging as their dupes are deleted >Add a sparse few personal tags not from boorus to url imports if needed, then archive them >Hide deleted files in folder import >Choose a trait common to many files and start multi-tagging groups of files based on this trait. >Repeat until there are no common traits >Go through every file individually and add tags that are less common, archiving each file that's done being tagged This is the most efficient process I've come up with so far. I can mostly skip tagging for anything that exists in a booru. Picking a trait common to many files and going through the whole set just looking for the presence of that trait makes sure I don't forget it for chunks of files, and allows me to semi-mass tag, which really speeds things up. I was originally doing all the tags for a single file at once, but then I was going through files one at a time, and I'd quite often forget one of the things I was looking for, or discover a new common trait to look out for and tag. This meant a lot of backtracking through images I thought I had finished tagging, and had already archived.

>>18314 >Is there a way to directly convert one namespace into another? I asked a similar question, and I think the answer was "not yet". Simplest solution, go through all images with the artist namespace, search for a particular artist, add the artist tag under the creator namespace for all images, delete the artist tag under the artist namespace for all images, repeat the last 3 steps until the artist namespace is eliminated. >before running into the problem that some files don't really have artists You don't have to add an artist namespace tag. You could just keep using artist: for artists, and creator: for anything with a "non-artist" creator. For similar such namespaces, I recommend going into File -> Options -> Tag Presentation and setting them to the same color.

>>18316 >You don't have to add an artist namespace tag. You could just keep using artist: for artists, and creator: for anything with a "non-artist" creator. I know that's an option, but I'd prefer consistency, and there's no reason not to put artists under creator: too if I can. >For similar such namespaces, I recommend going into File -> Options -> Tag Presentation and setting them to the same color. I have already done that, since leaving the artist tags with the default colour made it harder to see them.

Reminder that since you fixed hydrus to follow windows' global scaling setting, there has been some visual bugs in the zooming of images. I use 125% scaling and I've mostly noticed it showing up as white lines in the middle of images.

>>18295 Same anon here. I now add the series name in the series namespace manually, and when I import I add the filename under the filename namespace. My media player saves screenshots with the series, episode, and timestamp in the filename. I don't add any other tags unless it makes a good reaction image, which I usually would crop anyway so I end up exporting it and doing that.

Since everyone is talking about bulk tagging, is there a way to go back and erase all the tags under My Tags, if for example I wanted to rerun hydrus-dd with a new model version.

>>18320 I think the tag migration utility is what you want, tags->migrate tags. I personally made a hydrus-dd tag service to hold my hydrus-dd tags, to keep it separate.

>>18321 Thanks, good advice. I don't suppose you've found any scripts to export your own or PTR tags along with images into a dataset structure that deepdanbooru can train with?

I had a good week-and-a-bit getting back to schedule. I have fixed some critical bugs, including the crashy mpv problem on Windows, cleaned a bunch of code, and improved regular tag searching with new tools, cleaner logic, and automatic sibling substitution. The release should be as normal tomorrow, probably a little early. Windows Qt5 users will have special install instructions.

>>18322 I just use the default dataset, you'd probably have to make your own. I suppose you could ask the guy behind hydrus-dd how he did it, I doubt he exported everything by hand.

new release is missing installer and linux executable files.

>>18325 Usually only takes twenty minutes, but Github was taking ages to build today, I had to restart one of the macOS runs.

https://www.youtube.com/watch?v=UGZKKXTNcE8 The first Windows build was broken, if you got that please check again--the links have been updated to a hotfix. windows Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt5.-.Extract.only.zip Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt6.-.Extract.only.zip Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v500a/Hydrus.Network.500a.-.Windows.Qt6.-.Installer.exe macOS Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.macOS.Qt5.-.App.dmg Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.macOS.Qt6.-.App.dmg linux Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.Linux.Qt5.-.Executable.tar.gz Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v500/Hydrus.Network.500.-.Linux.Qt6.-.Executable.tar.gz I had a good week-and-a-bit returning to normal hydrus schedule after my personal issues. There are some important bug fixes, particularly for windows crashes, and some neat updates to tag search logic. Those who use the Windows Qt5 release will want to perform a 'Clean Install' this week: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs crashes and bugs I messed up the new mpv version in v499. My golden rule is to never put out bleeding-edge library updates, but without thinking I gave Windows users a dll from late August. This caused instability for a variety of installs, but thanks to some great reports and user testing, we were able to figure out the problem and solution. I regret I wasn't able to roll out an official fix until now, but I will remember this issue for future--never fold in the latest build of anything. There are two fixes here. Windows Qt6 users simply get a more stable mpv-2.dll today. You don't have to do anything special, just install as normal and your hydrus should be more stable. Windows Qt5 users will be rolling back to mpv-1.dll, so if you are a Qt5 user who updated to v499, you should perform a 'Clean Install', as here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs . Just follow the guide and you should be good again, but let me know if you have any trouble! I also fixed a critical issue that was affecting a couple of users with damaged similar file search trees. If you have had 'similar file tree rebalancing' maintenance that seemed to go on forever before locking up your client, this is now fixed. Some related simple errors when the maintenance routine ran into a damaged or looped tree are also fixed. The Client API now handles disconnects more gracefully. Some logspam is cleared up, and very slow file and tag searches via API now cancel on disconnect just like in the UI (e.g. when you type a new character in autocomplete tag search, it'll cancel the older slower search and start a newer faster one). If you run a busy Hydrus Companion or another Client API application that really hammers your client, let me know how you get on. tag search logic I have updated the tag search logic in two important ways: - First, if you give a file search a tag that currently has a better sibling (loading up old favourite searches can do this), let's say you enter 'shinji', which would now normally display as 'character:shinji', the database now recognises that this tag has a better sibling and will give you the results as if you had entered 'character:shinji'. Previously, it would give you no results since that tag 'didn't exist' any more. This sounds like a small easy thing to change, but as I peeled this system apart this week, I recognised there were several logical problems and inefficiencies with edge-case sibling search, so I rebuilt the bad parts and cleared several issues up. In any case, sibling tag search = better. - Second, searching files for the tag 'asuka' will no longer give files that have 'character:asuka'. This 'unnamespaced search tags give all namespaced variants too' rule has been in place since the start of the program, but it has always been awkward to implement, sometimes confusing, and it makes it difficult to search for an unnamespaced tag explicitly. If you would like to search for all namespaced variants of a subtag, please enter '*:asuka', which is now acceptable input. Note that this does not affect tag lookups. If you type 'rei', you will see 'character:rei' in the result list to choose from. But entering 'rei' will not find files with any namespaced version of the tag. Wildcards follow the new unnamespaced rule too, now. If you enter 'm*na', you will not get files with 'character:mana'. '*:m*na' will, though, and these 'any namespace' wildcard rules are now supplied as suggestions whenever you enter an unnamespaced wildcard. other highlights You can now fully edit tag, namespace, and wildcard search predicates. Either shift+double-click some active tag search predicates, or right-click and select 'edit', and you can now change their text. You can also convert between one or another just by typing. Thanks to a user, the 'pattern' you use when declaring export filenames now supports '{#}', which will give you the same as the incrementing number '#' column in the manual export dialog. You can now export files and give them a filename based on their current thumbnail order.

full list - crashes: - I messed the mpv update up in v499. my golden rule is never to put out bleeding-edge library updates, but without thinking I gave everyone a dll from late august. it turns out this thing was pretty crashy, and many users were getting other unusual behaviour as well. it seems like people on very new versions of Windows were mostly ok, but a little instability, whereas some older-Windows users were unable to start the client or could boot but couldn't load mpv at all. these latter cases were plagued with other problems. thanks to user help, we discovered it was the newer mpv dll causing all the problems, and an older one, from early May, seems to be fine - so, I am rolling back the mpv in the windows releases. the 'v3' 2022-08-29 I bundled in 499 was causing several users serious problems, possibly because of the advanced 'v3' chipset instructions or related advanced compiler tech. for the Qt6 release, we are going back to 2022-05-01, which several users report as stable, and for the Qt5 we are rolling back to the 498 version, 2021-02-28, which is back to mpv-1.dll. Since Qt5 users are increasingly going to be Win 7, we'll go super safe. THEREFORE, Qt5 extract users will want to perform a clean install this week: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#clean_installs - (you can alternately just delete the now-surplus mpv-2.dll in your install directory, but a full clean install is good to do from time to time, so may as well) - updated the sqlite dll in the windows release to 2022-05, and the exe in the db directory to 2022-09 - rewrote how some internal MPV events are signalled to Qt. they now have their own clean custom event types rather than piggy-backing on some bad old hydrus pubsub code - I either fixed a rare boot crash related to the popup messaging system, maybe exclusively on macOS, or I improved it and we'll get a richer error now - . - tag sibling search: - if you search explicitly for a tag that has a better sibling (one way this can happen is when loading up an old favourite search), the client will now auto-convert that tag to the ideal in the search code and give you results for the siblinged tag - this started off as a predicted five minute thing and spilled out into a multi-hour saga of me realising some tag sibling search code was A) wrong in edge cases and B) slow in edge cases. I have subtly reshaped how core file-tag search works in the client so that it consults each tag service in turn based on its siblings and its mappings, rather than mixing them together. this does not matter for 99.98% of cases, but if you have some weird overlapping siblings across different services, you should now get the correct results. also, some optimisations are more effective, so any instance of searching for tags on small tag services on 'all known tags' is now a bit quicker - big brain: please note the logic here is complex, and I have not yet updated autocomplete counting to handle this situation. if you type 'cat' and get 'cat (3)' from the three 'cat' tags on 'my tags', but 'cat' is siblinged to 'species:feline' on a big service like the PTR, it will still say (3), rather than (403) or whatever from the auto-corrected PTR results. I have a plan to fix this in a future cleanup round - . - tag subtags and namespace wildcards: - searching for 'samus aran' no longer delivers files that have 'character:samus aran'. the subtag->namespace logic no longer applies. this was a fun idea from the very start of the program, but it was never all that useful as default behaviour and added several headaches, now eliminated. if you wish to perform this search going forward, please enter '*:samus aran', which is now an acceptable wildcard input - tag lookup is unaffected. typing 'samus aran' will still provide 'character:samus aran' as a tag to choose from - a heap of rinky-dink counting logic went along with this, such as providing tag search results like ('character:samus aran (100)', 'samus aran (100-105)'), where it tried to predict how many results would come with the unnamespaced search. this no longer exists, and a decent bit of CPU is now saved in any large tag search - wildcard searching works on similar rules now, so if you enter 'sa*s ar', you will see 'character:samus aran' as a result in the tag list, but searching for it will not give results with 'character:samus aran'. again, enter '*:sa*s ar*' to search for all namespaces (which is now provided as a quick suggestion any time you enter an unnamespaced wildcard), or enter 'character:sa*s ar*' explicitly - 'system:tag as number' also now follows similar rules, so if you leave the namespace field blank, it will search unnamespaced numbers. it now supports namespace wildcards, so you can enter '*' to get the old behaviour. the placeholder text on the namespace input now states this - 'system:number of tags' now uses the same UI as 'system:tag as number', where you enter '*' as the namespace to mean all namespaces, rather than checking a box

- misc: - all tag, namespace, and wildcard search predicates are now properly editable from the active search box. shift+double-click or select from the right-click menu, and you now get a simple text input alongside any system predicate panels. previously, this would only offer you a button to invert the tag to -tag and _vice versa_. now, you can add or remove the '-' and '*' characters yourself info to freely convert between tags, namespace:anything, and wildcard search predicates (issue #1235) - thanks to a user, you can now add '{#}' to an export filename pattern to get the '#' column in your filename (useful if you want to export files in the order they are currently in on the page) - furthermore, if you delete items from the manual file export window, the '#' column now recalculates itself to stay contiguous and in order (previously, it left gaps) - fixed a bug when deleting siblings on a local tags service. sorry for the trouble! - on manage siblings, when you remove, add, or replace a pair on a local tags service, you will now get a simple 'note' reason informing you more on what is going on. the 'REPLACEMENT:' thing recently added to tag repositories should now work for you too - when a downloader or similar adds files to a page, and you have at least one existing file selected, the status bar now updates correctly - fixed a critical issue that was affecting some users with damaged similar file search trees. when starting similar file search tree rebalancing maintenence, their client would go into an infinite loop and spool the cyclic branch into an ever-growing journal file in their temp directory until their system drive briefly ran out of space. sorry for the trouble, and thank you for the excellent reports that helped to figure this out (issue #1239) - the similar files search tree rebalance maintenance now detects more sorts of damaged trees and handles them gracefully, and the full tree regeneration clears out any damaged maintenance information too - fixed another problem with the tree branch maintenance system when the root was accidentally queued for branch rebalance - when you right-click->copy a wildcard search tag, it now copies the actual wildcard text, not the display text with (wildcard search) over the top - I added ',' to the list of non-decodable characters in the hacky URL Class encoding/decoding routine. sites that use an encoded comma (%_2C) for regular path components or query parameters should now work - a user has fixed a regex parsing problem in the predicate parser for system:hash - OR search predicates now sort their sub-predicates on construction/editing, meaning the label is always of set order, and they can now compare with and hence reliably nullify each other - the manage logins dialog now boots a little taller - the main gui tab bar may look a bit nicer/more appropriate in macOS - updated the help text on gui pages where it talks about overflowing rows of tabs, which auto-scroll even worse in Qt6, hooray - . - client api: - the client api now handles request disconnects better. the hydrus server code benefits from the same engine improvements - the 'twisted.internet.defer.CancelledError' logspam is cleaned up! - if a client disconnects before a client api autocomplete tag search or a file search is complete, that database job is now cancelled quickly just like when you type new characters in the client UI or stop a slow search - if you are a client api dev, please let me know how this works out IRL. I'm not 100% sure what a 'disconnect' means in this context, but if you want to develope autocomplete quick lookup as the user types, and you have a way clientside to cancel/kill an ongoing request before it is complete, please give it a go and let me know if this all works. cancelled requests don't make a log record right now, but you should see the client's db lock free up instantly. at the very least, I have the proper infrastructure for this now, so I can add more/better 'cancel' hooks as we need them - . - uninteresting code cleanup: - refactored the file note mapping db code to a new module - refactored the file service pathing db code (this does directory structures and multihashes for ipfs) to a new module - refactored some tag display, tag filtering, and tag autocomplete calls down to appropriate db modules - refactored and extended some tag sibling database methods and names to clarify whether they were working with ids or strings next week I was not able to get to many things I wanted to this week. Things have piled up, so I'm just a bit buried right now. I will just continue working on urgent issues and smaller issues and see how we are on the crashing. I'm stressed about my Dad, more than anything because there has just been a ton of energy-draining stuff to do, but not as upset as I thought. As I said before, we had a great relationship, so there are no huge regrets. I'm sure it will kick in more in a couple months. Since the hydrus userbase trends young and my Dad died a bit early, I don't expect many of you have organised a funeral. My serious advice is A) talk to your parents now about what they want in a funeral, and B) make sure they have a will. We were good on both fronts, and it has made the whole thing so much easier. Almost all children bury their parents, so get it done now, while it is easy. As for hydrus, getting out 500 versions is pretty cool. I've been at this for ten years, and the codebase is now 10 MB over almost 300 files. I still enjoy working on it, and I want to keep at it as long as there is interesting stuff to do. The hydrus userbase has grown significantly this year, and my todo list is overflowing more than ever, so running short on work is not a worry. Handling stress and burnout has been tricky at times, but assuming I stay healthy on that front, I can comfortably see 750 in the distance. Let's see what machine learning does to us all over the next five years. Thanks everyone!

Sorry for your loss. We appreciate the hard work you've been putting in but definitely don't push yourself too hard. Just wanted to let you know I'm having some trouble starting my client after updating from v499 to v500 (both Linux QT6). I get the following: v500, 2022/09/28 16:44:56: hydrus client started v500, 2022/09/28 16:44:56: hydrus client failed v500, 2022/09/28 16:44:56: Traceback (most recent call last): File "hydrus/hydrus_client.py", line 230, in boot from hydrus.client import ClientController File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/ClientController.py", line 38, in <module> from hydrus.client.db import ClientDB File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/db/ClientDB.py", line 16, in <module> from hydrus.core import HydrusDB File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/core/HydrusDB.py", line 2, in <module> import distutils.version File "_distutils_hack/init.py", line 88, in create_module File "importlib/init.py", line 127, in import_module File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "setuptools/init.py", line 10, in <module> File "_distutils_hack/init.py", line 88, in create_module File "importlib/init.py", line 127, in import_module ModuleNotFoundError: No module named 'setuptools._distutils' v500, 2022/09/28 16:44:56: hydrus client shut down I tried performing a clean install but that didn't change anything. Reverting back to v499 got it working again.

>>18330 Same here. Client crashed trying to startup on 500. Reverted back.

What determines whether something shows up under Network -> Downloaders -> Manage Default Import Options? I imported a new version of a site's downloaders, but now it's disappeared from this section.

Is mpv preview bugged for anyone else? Rather embedding mpv into to the preview area, I'm getting a new popup whenever I select anything. Switching to native previews works as intended, but doesn't support sound

Can anyone else download with Hydrus from Deviant Art? I can access the site still, but when I try to use Hydrus, I get a "nothing found in the document" error. Putting the URL into Firefox, I get something like "invalid page" or such. It looks like they've changed the API.

>>18334 Ok, just tried again with a fresh install of 500a. "The Parser found nothing in the Document" When I stick the URL into Firefox, I get this: error "invalid_request" errorDescription "Invalid or expired form submission" errorDetails csrf "missing" status "error" So yeah, something has changed with their API.

>>18321 >I personally made a hydrus-dd tag service to hold my hydrus-dd tags, to keep it separate. Oh snap, is this how people normally use tag services? I don't connect to the PTR or anyone else's instances so I've never used tag services before, but part of my workflow is trying to use tags from different sites (with very different quality standards) and eventually they all just smoosh together into a mess. I just made a tag service for one of my sites, and I see that I can stuff the tags into that service only, and not my main service - is making one service for every site a good idea/are there drawbacks?

>>18336 If you have tag services X, Z and Y, then I don't think you can select to search tags from only tag services X and Z, for instance, right now. Only one service or all services.

>>18337 not a big deal for me thankfully. I also decided to set up another tag service dedicated to holding parenting/sibling logic rules (mass-exported 40k pairings from a booru, not sure if it's recommended but I've been using these rules for a couple years now with no issues) and apply it to my other tag services, so I can still use my own parents/siblings without them getting instantly lost in the clutter. I wish I could resync the tags from my sources to their new tag services easily, but after years of duplicate merging I usually don't have the exact files that came from the boorus, and from what I can tell, the "re-fetch page/merge tags with local" functionality of downloading only works if it detects it's the same file.

>>18333 >Rather embedding mpv into to the preview area, I'm getting a new popup whenever I select anything It is not a bug, but a feature. There is a full screen switch to windowed mode. Check the screenshot out.

>>18336 The way I have mine setup is I have 4 services, hydrus-dd, my tags, parsed tags, and the PTR. I want my tags to be "my tags" stuff that wouldn't work/be found on a booru, like a specific series of meme or whatever (e.g 'this is bait'), actual text from the image, etc. Parsed tags is relatively new, I used to just use my tags for all tags. Parsed tags is only for booru tags, and I basically want it to be like a localized ptr with my own opinions. 1 tag service per site seems burdensome, but you could make a "dirty" tag service and a "clean" tag service for easy separation. Download tags to the dirty service, and migrate tags that you want to the clean service. Then you can keep tags you're unsure about around in case you change your mind on them later.

>>18339 The issue I have is that both the embedded thumb in the bottom left and the popout player don't show any video in mpv mode, but instead open a separate window they are linked to, like x11 was unable to draw the video to the correct window and has fallen back to a new one. These extra windows are buggy and a new one is opened every time the video changes, cannot be closed. I've tried building from source and using the Flatpak. No idea what's causing this. Worked fine on my old Manjaro desktop. Anyone else seen this or have any ideas?

>>18341 This is a Wayland issue. Running Hydrus with xorg fixed this. Perhaps unrelated to Hydrus entirely.

>>18338 >mass-exported 40k pairings from a booru How did you do that? >>18342 Truly the future of display protocols.

>>18343 >>mass-exported 40k pairings from a booru >How did you do that? I imagine it depends on the booru, but I'll attach the one I have for e621. I forget if I wrote this from scratch or copied the base from somewhere but you just run these in python (input API creds), then import the resultant .txt files into hydrus: https://pastebin.com/bxDC9Znm https://pastebin.com/c70bJzKn https://pastebin.com/e9MwrRG7 At least for e621, generating the pairings is quick, and importing them into hydrus via .txt is quick. Doing the tag sync takes like half an hour but not too bad. Removing the pairs via show all -> highlight ~10k pairs -> delete is slow (10k at a time works fine for me), and at least in the latest version of hydrus it seems to cause some sort of UI error that hides all the pairs when it does this, but clicking apply works as intended anyway. Then reopen the window and do it again.

>>18344 The third link is for Siblings, not Parents - forgot to change the title. It's just a copy of the 2nd link with the mode changed. API documentation is here: https://e621.net/help/api and you can do interactive searches e.g. here: https://e621.net/tag_aliases Idk if this sort of functionality is normal for other boorus but it's pretty easy for e621.

>>18311 I ended up downloading and editing the PKGBUILD myself to remove mkdocs-material from the make dependencies, as well as the other stuff related to it, It was only a couple of lines to edit. after that its just two commands to make and install the package.

>>18334 I'm seeing the same issue. where do we raise a report?

>>18347 Here is the best place. Hopefully the author sees it. I'm really not sure what has happened. I was getting 403's once in a while during search, but I figured it was because I was searching too fast and the site was blocking me. I raised the time between searches, and that seemed to help a lot. Then suddenly, I got nothing but empty pages upon searching. I think they changed something in the API.

>>18348 Sounds like the API requires a CSRF token for every request now. https://github.com/mikf/gallery-dl/issues/2981#issuecomment-1261951885 I'm not sure Hydrus is capable to sending this token via a parser, currently. In any case, the file page parser still works, but I suppose it *might* produce files that are not the best ones available. To make the file page parser work, you can go into network>downloader components>manage url classes, open 'deviant art file page', go into the options tab and delete all steps in "optional api url converter".

>>18344 >>18345 Thanks mate. I'm not huge on furshit but I know furfags are uber autistic when it comes to tagging porn. Hopefully it should also clear up shit like pokemon species being in different namespaces in every booru.

>>18350 you might be able to use e621 tag logic on non-furry artwork, as it's pretty comprehensive for general tagging and includes furry-specific tagging on top - I can't think of any downsides at the moment, but with 40-50k pairings being misused, I wouldn't be surprised if something gets weird. The reason the block of logic exists, and why I find it so handy, is it quickly expands simple tags into very robust and comprehensive tags. e.g. "ear ring" implies "ear piercing" and "ring piercing" and "piercing", and it renames nonstandard tags to accepted tags, e.g. "cops" to "police", which can then be fed back into the cycle of adding implied tags, and so on. One of the weirder parts of having this system both on the booru and on hydrus is that grabbing posts from e621 already pre-bakes all the implications and aliases, so working backwards isn't as intelligent as it could be if you did it all through hydrus to begin with. I'm not sure if there could be a logical way to work backward in this case or if it's impossible to determine which tags were original.

Is anyone else having issues with the image viewer having scaling artifacts recently? For reference I'm on Linux+KDE+Xorg with 125% global zoom on a 1440p screen, and I've been using "QT_ENABLE_HIGHDPI_SCALING=1 QT_SCALE_FACTOR=0.8 QT_AUTO_SCREEN_SCALE_FACTOR=1" environment variables for a long time to fix the atrocious scaling artifacts that hydrus had until a few months ago or so. Some sort of scaling change must have happened, because all the text in hydrus got smaller. I removed the enviromental variables and everything is the size it used to be (only had to tweak thumbnail sizes), but now I get these artifacts in the image viewer. It's most apparent when duplicates processing with different file dimensions, as each file will alias to different lines/create different artifacts, and the amount of shifting makes it hard to see small differences between the files. If I put the environment variables back, the text becomes small again, but the artifacts go away.

>>18311 >>18346 Thanks. I'm glad you like hydrus! Yeah, mkdocs is a funny one. I overall really like the package in terms of using it, but it does have a lot of bells and whistles and I'll agree it probably shouldn't be something the end user is generally confronted with. Some users helped me migrate, and if you can believe it, I was hand-coding the html help before, my knowledge based on a 'learn html in 24 hours' book probably from 1997. Anyway, let me know what works for you, and I will try to adjust hydrus to work with that. If you hit help->help and getting started guide without a 'help' directory, it now makes a little dialog saying 'hey, looks like you didn't build the help files, want to open the website?', which I think is the sort of solution I prefer. Maybe the AUR guys can offer a flag of some kind that skips building it? I sort of wish there was a way it was bundled into the release as a compiled zip, but now I think of it, that is probably pretty easy to set up. Like in the old days, when you could download the help of a program as a separate thing since 8MB was a lot to get back then. That zip would be something other package makers could potentially rely on to get a fast prebuilt help. I'll think about this. >>18312 Thank you, I get the same file dialog bug when I run with PyQt6! I will fix this up. >>18313 Not right now. Probably impossible to implement really, with almost totally anon accounts and all the vpn tech around these days. I was worried about eventual moderation tools when I first started hydrus, and if malicious behaviour ever does become a problem I'll put time into it, but that whole concern has all been massively overshadowed by the tidal wave of regular problems there are just trying to get mismatched auto-parsed tags to merge together, pruning out mistakes and trying to establish consensus on things like siblings. >>18314 As >>18316 says, quick answer to 1 is 'not yet', but I'd love it. It is very technically complicated, so it'll be a big job. For 2, I think your best answer, if you can do some scripting, is the Client API. You can do file searches and apply tags to files with that, so you should be able to apply whatever update logic you like based on your lists of tags. You could do an artist/creator swap in the same way if you wanted. https://hydrusnetwork.github.io/hydrus/client_api.html However if you have never done a scripting language like python or perl before, it may be too overwhelming. Sounds like you have some experience though. Let me know if I can help.

>>18315 I did all my original 'import all my shit' years ago, when I started the program, but I still do manual hard drive imports for my vidya screenshots and video captures. I always like to do the tag processing there at the import step. I keep it simple, just a 'series:' tag for the vidya name and something like 'vidya memory' and 'i took this'. I like to be surprised when I browse my collection, so I regularly search for things like 'vidya memory' with a small 'system:limit=16' on random sort. I've never been one to search for specific small tags, so I usually just stick to creator/series/character tags. I go crazy if I try to tag my files with too many unnamespaced tags. And I never really use them in searches, so they aren't as personally useful either. When there's a big wave of memes, I will put a bit of time into tagging characters, meme:, or similar. Much of the 'character:ebola-chan' and 'meme:ogey rrat' on the PTR was by me. >>18318 Thank you--I meant to ask you, can you specify the exact numbers here so I can reproduce the problem? I tried but could not find a resolution combination that causes this. I know exactly what the bug is though. Please say: UI Scale: (obviously 125%) Resolution of image: Exact resolution of media viewer: (if you get the artifact with borderless fullscreen, that'd be easier to measure) Image zoom at which the lines appear: Ideal tile width/height px under options->speed and memory: (default I think is 768) A screenshot may be able to give much of that info. If you are ok sharing the image too, that'd be helpful. >>18322 >>18324 Client API is the way to go here. It'll let you grab the image and its tags, which you can then munge into whatever format a training model will take. Let me know how it goes if you try this out. I know the current models take centuries of CPU time to put together, but I'm getting into ML now as a hobby, and I am interested in keeping up with the news here for when training becomes consumer-possible and user-friendly. If it is ever possible, or training-updating, hydrus fits into that process extremely well. https://hydrusnetwork.github.io/hydrus/client_api.html

>>18330 >>18331 Damn. I am sorry about this. This looks like the same problem as hit the Windows builds. Someone flipped a switch in PyInstaller or Setuptools, and suddenly a thing that is 56 by default had to be <50 or >=65 or any build with it in might not boot. It broke a bunch of python programs apparently. Stick with v499 now, I will ensure v501 Linux has the fix as well. >>18332 It should will be any 'post url' or 'watchable url' listed in network->downloader components->manage url class links, on the 'parser links' tab. That lists everything that is hooked up to work and not redirecting to a different url class (check the other tab of that dialog for some info on that). Normally the list increases when you add a new downloader, but perhaps this update collapsed your url classes down a bit, redirecting several things down to one master type? If you poke around manage url classes on that same submenu, do you see the url classes for that downloader you added? >>18333 >>18342 Let me know if I can help with this, maybe just adding a 'here's what to do to fix' section to the help. I don't know enough about the details here to comment cleverly. Basically though, for mpv to work, I instantiate an mpv embed window and then give it to a Qt QWidget container. There's some OS-level window handle stuff going on, which I guess is what is bugging out here. Everything is wrapped in python, so it is by the grace of God alone any of it works.

>>18354 >can you specify the exact numbers here so I can reproduce the problem? I'm not anon but I have this problem also. UI Scale: 125% Resolution of Image: 657x750 Exact resolution of media viewer: 1402px tall, 2560px wide Image zoom: 1000% (near the eyes) Ideal tile width: 768px I'm attaching two sizes of this picture, try putting them in a duplicate filter and switching between them, it's very obvious what's going on that way (attached what duplicate switching looks like for me). Almost every image can be forced to have these artifacts by zooming in and out until you hit a certain "bad zoom", and most images that I've been seeing in the duplicate filter have at least one of the copies of the image with almost every zoom as a "bad zoom", which means comparisons almost always have this tiling anomaly.

>>18334 >>18335 >>18347 >>18348 >>18349 A user posted a new downloader which apparently fixes things. I will test it and roll it into v501, so fingers crossed this is fixed soon. Please pause your DA subs for now. >>18337 I hope to have this soonish, probably in the new year. When I did multiple file services, I added a bunch of tech to support this. Should be doable. >>18352 Moving to Qt6 meant that a whole lot of UI scaling now works 'better'. Instead of everything being based off of font scaling as in Qt5, now all widget components are scaled, and any floating point scale value is allowed (previously it was sometimes 100% OK 200% OK, nothing else). In doing this, I then learned how all this shit actually works Qt side and updated my custom thumbnail grid and media viewer widgets to do zoom properly. They now have correct zoom numbers and things. All the math has been slightly adjusted, and I know there are bugs in my image tiling system, worse for some users. I am pissed off with the tiling problems, so I now have a master plan to just rewrite one part of it and ditch the tile-stitching entirely, spending some memory to just render super-monitor-sized tiles that you never see the edges of. I apologise for any inconvenience. If you can stick to 100% UI scale, you'll probably get the prettiest result. You might like to try boosting the tile size in options->speed and memory. I think default is 768, maybe higher reduces tiling artifacts. Please let me know how things go in future. Oh, and if you aren't on Qt6 yet, please try it out. All the UI scaling is much better in 6, they won me over and I now think they did a good job. Now my shit code has to catch up.

>>18356 Thank you, this is excellent!

>>18357 >Oh, and if you aren't on Qt6 yet, please try it out. I just posted all the raccoons, and I'm on Qt6 I believe, per the help -> "about Qt". I'm still on v499 because someone said that 500 is having issues on linux - not sure if that changes anything. I don't really understand how the tiling mechanism is supposed to work but I just set it to 2560px which I assumed would effectively mean "one tile" for me, but I still see artifacts

>>18359 4096px (which seems to be the max size) also has artifacts. I don't get the "4 tiles shifting in different directions" effect as much anymore (though it does still happen sometimes), but the whole image still does shift in one direction which still makes it a bit annoying to do detailed comparisons. I'll just leave it on this setting for now because it seems like a small improvement.

>>18327 >You can now fully edit tag, namespace, and wildcard search predicates. Either shift+double-click some active tag search predicates, or right-click and select 'edit', and you can now change their text. You can also convert between one or another just by typing. Could the old "inversion" trick be added back as another right click menu option? I mean how using edit on "tag" would turn it into "-tag", and doing that again would turn it back into "tag". I found myself using it a lot.

>>18341 >>18342 >>18355 I posted about the mpv issue in the previous thread. Essentially mpv has the command line option "wid" that specifies which window to attach it to, which is the method Hydrus uses. Your window manager is the system that handles these "wid"s. Most window managers support this, like Windows, MacOS, X11. For Wayland to support this you'd need Qt, mpv, and your compositor to all support the XDG foreign protocol, which is unstable at the moment, and even then I don't think you can just pass the wid to mpv. The issue has to do with this: https://github.com/jaseg/python-mpv/issues/196 The solution is to change how mpv is embedded in Hydrus, by using the render API: https://github.com/jaseg/python-mpv/issues/222#issuecomment-1179721210

>>18353 More mkdocs-material nonsense on the AUR in pic related, now that arch linux has dropped python2 from its official repos. Is omitting mkdocs-material handled properly when building (e.g. only skipping the help file functionality)? If so can we make an official recommendation for it to be set to optional?

>>18363 I was wondering what was pulling in python2. It takes a long time to build and then fails at the end so I've just been ignoring it.

>>18354 >I'm getting into ML now as a hobby, and I am interested in keeping up with the news here for when training becomes consumer-possible and user-friendly. If it is ever possible, or training-updating, hydrus fits into that process extremely well. Tagging is great but automatic classifier tags will hopefully be the future someday. Something like: system:has (person OR face) (https://github.com/weiaicunzai/awesome-image-classification) system:mood <90% happy (https://github.com/topics/emotion-detection) system:not explicit (https://github.com/notAI-tech/NudeNet) File relationships could also be extended: Automatic artist grouping or image style grouping (https://github.com/tdody/StyleYourArt) Grouping images that contain the same face(s) (https://github.com/topics/face-clustering) There are already more than a few models capable of processing images in such ways in bulk, but not always efficiently Also inline upscaling for high-dpi displays (ESRGAN or waifu2x)

>>18357 >Please pause your DA subs for now. Thanks! Will do!

>>18365 AI and ML are going to be a TOTAL game changer for Everything.

hello, i am having issues with importing images tagged as contentious_content on sankakucomplex. i have tried both the default downloader and the beta downloader. i was logged in with the default downloader and also tested it with the testing feature where it said testing / login successful. if i set a downloader for exclusively contentious_content and limit the amount by "date:>2022-10-01 for example", i get this error in the sarch logs: 400: {"success":false,"code":"invalid-parameters","errorId":"error_d1628487e11b33f5"}… (Copy note to see full error) Traceback (most recent call last): File "hydrus\client\importing\ClientImportGallerySeeds.py", line 366, in WorkOnURL network_job.WaitUntilDone() File "hydrus\client\networking\ClientNetworkingJobs.py", line 1918, in WaitUntilDone raise self._error_exception hydrus.core.HydrusExceptions.BadRequestException: 400: {"success":false,"code":"invalid-parameters","errorId":"error_d1628487e11b33f5"} this issue hasnt been a thing until a few weeks ago. my latest imported file with contentious_content has from 23 days ago (september 9th 2022). does anyone know how to fix this? is there a different / new login script needed for the website? does anyone have a login script for sankaku beta? i am on hydrus version 499.

This duplicate pair acts strangely in the duplicate filter. Switching between A/B causes the image to shift slightly left and have different perceived zoom levels on my client.

>>18369 because they are cropped differently, baka.

>>18369 >>18370 also they have a different filesize and resolution

I've been quality trumping a lot of duplicates lately, and something I noticed that's been getting on my nerves is that sometimes it will make me battle worse copies against each other before it has them battle the clear winner. e.g. let's say I have 6 copies of a file: A. 100x100 B. 112x112 C. 90x90 D. 128x128 E. 140x140 F. 1000x1000 sometimes (always?) it will make me battle A vs B, then B vs C, then B vs D, then D vs E, then finally E vs F, where F obviously ends up winning. If it instead had me battle F vs A, F vs B, F vs C, and so on, it would be much faster to determine the better copy and I would take much less time trying to pixel-hunt differences between copies when it ultimately doesn't even matter. I know there's a way to arbitrarily score duplicates against each other at runtime (pic related), but I'm wondering if this could be pre-emptively calculated and then e.g. show the files with the biggest difference against each other first? (note in this example only the resolutions are different for simplicity, but in practical experience I'm often battling the same resolutions against each other with different JPEG artifacts and trying to figure out which is better)

Can you sort collections by average file size, or limit collection size in search? I have a bunch of manga in my inbox but I want to go through the ones with larger files rather than the ones with 300 normal-sized pages.

I have a question about Hydrus client_files folder. I lost my database recently and my backups are no good (lesson learned: test your backups and keep more than one). >Will hydrus import the old files from the client_files folder? I don't think it does from my testing. I know the tags will be gone but I'm not too worried about that.

I'm willing to make a downloader for a site that isn't just porn. A mix of lewds and non-lewds is okay as long as there is some non-lewds. Any requests?

>>18365 >Also inline upscaling for high-dpi displays (ESRGAN or waifu2x) upscale generation as a feature would be killer, as someone with a ton of old shitty art that looks great when upscaled. Upscaling data and replacing the original is a bad idea, but just making a new copy to live alongside it would be fine. probably handle it like thumbnails, with (u00)-(uff) folders or something.

What's the best way to merge 2 db's? Or rather, export one and import it to the other? Ideally I'd like to transfer tags, URL's, subscriptions, and parents/siblings.

I had an ok week. I cleared out some bugs, upgraded the popup toaster, and improved some UI quality of life. The release should be as normal tomorrow.

Say I want to import something in the order of 30000 images from my Twitter archive. What's the best way to mass import URLs? Can I just put them into a big text file or in a bunch of smaller ones and feed them to Hydrus somehow, or do I have to use the API and make 30000 requests serially?

https://www.youtube.com/watch?v=4UeFP0al1IY windows Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt5.-.Extract.only.zip Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt6.-.Extract.only.zip Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Windows.Qt6.-.Installer.exe macOS Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.macOS.Qt5.-.App.dmg Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.macOS.Qt6.-.App.dmg linux Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Linux.Qt5.-.Executable.tar.gz Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v501/Hydrus.Network.501.-.Linux.Qt6.-.Executable.tar.gz I had a good week working on a mix of important bug fixes and UI quality of life. highlighted fixes First off, I have rewritten the guts of the popup toaster. It looks the same, but now it is more 'embedded' into the main gui window and I completely overhauled its layout system. I strongly hope that this will eliminate a variety of jank that has occured over the years, from odd activation/focus issues to weird width sizing to unusual popup locations. If you have had trouble with popup messages, let me know how things work today! Next, due to a change behind the scenes in another library, the Windows build wouldn't boot last week and needed a hotfix. The Linux build seems to have suffered from a similar issue, so I have fixed both. If you had trouble booting v500 Linux, I am sorry--please try again today. The macOS release uses a different system and was unaffected. I fixed a bug (and sometimes crash) when opening file selection dialogs for users running from source with PyQt6. Thank you for the reports! I fixed a crash that was affecting many users (including myself a couple times, I now realise) when performing some options-saving events such as pausing/resuming subscriptions, repositories, or import/export folders. I was finally able to reproduce this reliably as I was putting the release together today. Thank you very much to the users for the reports here, I would not have figured it out but for what you discovered. It turns out it was the collect-by control doing it, so its ability to update itself without a client restart is disabled for now. I fixed a bad bug in the manage ratings dialog, where any numerical ratings that started and ended on dialog ok in the 'mixed' (dark grey colour) state were not leaving them alone but resetting those files' ratings to the minimum allowed value (0 or 1 stars). This was an unusual bug, and I regret it a lot. I think it came in in recent weeks, when I rewrote how some rating states are handled internally. The Deviant Art downloader broke last week. It seems like they locked away the API we were using. I have reset us to the original html parser, which still seems to work ok. Please check your DA subs and tell them to retry ignored, and let me know if there are any glaring problems. full list - misc: - the Linux build gets the same 'cannot boot' setuptools version hotfix as last week's Windows build. sorry if you could not boot v500 on Linux! macOS never got the problem, I think because it uses pyoxidizer instead of pyinstaller - fixed the error/crash when clients running with PyQt6 (rather than the default Qt6, PySide6) tried to open file or directory selection dialogs. there was a slight method name discrepancy between the two libraries in Qt6 that we had missed, and it was sufficiently core that it was causing errors and best, crashes at worst - fixed a common crash caused after several options-saving events such as pausing/resuming subscriptions, repositories, import/export folders. thank you very much to the users who reported this, I was finally able to reproduce it an hour before the release was due. the collect control was causing the crash--its ability to update itself without a client restart is disabled for now - unfortunately, it seems Deviant Art have locked off the API we were using to get nice data, so I am reverting the DA downloader this week to the old html parser, which nonetheless still sems to work well. I expect we'll have to revisit this when we rediscover bad nsfw support or similar--let me know how things go, and you might like to hit your DA subs and 'retry ignored' - fixed a bad bug where manage rating dialogs that were launched on multiple files with disagreeing numerical ratings (where it shows the stars in dark grey), if okayed on that 'mixed' rating, rather than leaving them untouched, were resetting all those files back to the minimum allowed star value. I do not know when this bug came in, it is unusual, but I did do some rating state work a few weeks ago, so I am hoping it was then. I regret this and the inconvenience it has caused - if you manually navigate while the media viewer slideshow is running, the slideshow timer now resets (e.g. if you go 'back' on an image 7 seconds into a 10 second slideshow, it will show the previous image for 10 seconds, not 3, before moving on again) - fixed a type bug in PyQt hydrus when you tried to seek an mpv video when no file was loaded (usually happens when a seek event arrives late) - when you drop a hydrus serialised png of assorted objects onto a multi-column list, the little error where it says 'this list does not take objects of type x' now only shows once! previously, if your png was a list of objects, it could make a separate type error for each in turn. it should now all be merged properly - this import function also now presents a summary of how many objects were successfully imported - updated all ui-level ipfs multihash fetching across the program. this is now a little less laggy and uses no extra db in most cases - misc code and linter warning cleanup

- tag right-click: - the 'edit x' entry in the tag right-click menu is now moved to the 'search' submenu with the other search-changing 'exclude'/'remove' etc.. actions - the 'edit x' entry no longer appears when you only select invertible, non-editable predicates - if you right-click on a -negated tag, the 'search' menu's action label now says 'require samus aran' instead of the awkward 'exclude -samus aran'. it will also say the neutral 'invert selection' if things get complicated - . - notes logic improvements: - if you set notes to append on conflict and the existing note already contains the new note, now no changes will be made (repeatedly parsing the same conflcting note now won't append it multiple times) - if you set notes to rename on conflict and the note already exists on another name, now no changes will be made (i.e. repeatedly parsing the same conflicting note won't create (1), (2), (3)... rename dupes) - . - client api: - /add_tags/search_tags gets a new parameter, 'tag_display_type', which lets you either keep searching the raw 'storage' tags (as you see in edit contexts like the 'manage tags' dialog), or the prettier sibling-processed 'display' tags (as you see in read contexts like a normal file search page) - /get_files/file_metadata now returns 'ipfs_multihashes' structure, which gives ipfs service key(s) and multihashes - if you run /get_files/search_files with no search predicates, or with only tags that do not parse correctly so you end up with no tags, the search now returns nothing, rather than system:everything. I will likely make this call raise errors on bad tags in future - the client api help is updated to talk about these - there's also unit tests for them - client api version is now 33 - . - popup messages: - the background workings of the popup toaster are rewritten. it looks the same, but instead of technically being its own window, it is now embedded into the main gui as a raised widget. this should clear up a whole heap of jank this window has caused over the years. for instance, in some OSes/Window Managers, when a new subscription popup appeared, the main window would activate and steal focus. this annoying thing should, fingers crossed, no longer happen - I have significantly rewritten the layout routine of the popup toaster. beyond a general iteration of code cleanup, popup messages should size their width more sensibly, expand to available space, and retract better after needing to grow wide - unfortunately, some layout jank does remain, mostly in popup messages that change height significantly, like error tracebacks. they can sometimes take two frames to resize correctly, which can look flickery. I am still doing something 'bad' here, in Qt terms, and have to hack part of the layout update routine. let me know what else breaks for you, and I will revisit this in future - the 'BUGFIX: Hide the popup toaster when the main gui is minimised/loses focus' checkboxes under _options->popups_ are retired. since the toaster is now embedded into the main gui just like any search page, these issues no longer apply. I am leaving the two 'freeze the popup toaster' checkboxes in place, just so we can play around with some virtual desktop issues I know some users are having, but they may soon go too - the popup toaster components are updated to use Qt signals rather than borked object callables - as a side thing, the popup toaster can no longer grow taller than the main window size next week More small job catchup, and I'll try and clear some github bug reports. Let me know how the crashes are going, but my fingers crossed.

>Hear that some artists are trying to get their work removed from Danbooru because they don't want their art used for AI training >get the bright idea to just download it all >create new Hydrus database just for Danbooru, start a gallery download for * >downloader stops working after 1000 pages (18359 files) >the most common tag on Danbooru has over million files >6 hours, 8000 files, and 16GB in, the newest file's modified times are only 4 days ago Never mind then.

Dev, please read.

>>18382 You hit a page limit. Obviously, it's 1000 pages. Look on their forums to see if there is a way around it, like if you can start from a certain post count or date. Gelbooru and Sankaku have this same problem, but there is a code you can put in to get around it. You use it like a tag. On Sankaku, it's id_range:xxxxx to xxxxx, or something like that. That way, it starts deeper into the database, and the starting point is page 1. Look in the forums on Danbooru to see if they have something similar.

>>18384 You might have to use the tag many times to get all you want. Downloading Sankaku Chan, I had to do this about 100 times, with different ranges.

>>18384 Yeah, but even then I would have to manually do that several hundred times assuming it got 18,000 files with each search and a lowball estimate of 5 million total files. >>18385 Wait, you downloaded all of fucking Sankaku? How? They give me the shitty triangle PNG every third or fourth image and IP ban me for a day after downloading a hundred files or so. Even ignoring that, don't you have to manually input your access token every 48 hours?

MPV can't be loaded. Using "Hydrus.Network.501.-.Linux.Qt6.-.Executable.tar.gz" on Linux MX (Debian 11 stable) I just upgraded from v499 to v501 and MPV won't load. The related error messages are shown in the screenshots.

>>18386 >They give me the shitty triangle PNG every third or fourth image and IP ban me for a day after downloading a hundred files or so. I don't have a problem with them. Try lowering your search and file download times in the settings. You may be going to fast. Also, make sure you are logged in with Hydrus. Best way to solve problems is download Hydrus Companion, and then use it to sync cookies with the site.

>>18388 >Also, make sure you are logged in with Hydrus. I mean logged into Sankaku with Hydrus. It's in Network -> Logins -> Manage Logins. This will tell you if Hydrus sees you as logged in to the site. Also, Sankaku Channel has a 500 file limit, and is really slow. Once past 500 posts, you have to start using the id_range: tag to go further. And if you want a couple of months in, your probably going to have to do that at least 20 or 30 times to get there. It's a lot of work. Pixiv is the easiest to download from. They have a 1000 page limit, so you can easily get 200 - 300k files.

>>18382 >Hear that some artists are trying to get their work removed from Danbooru because they don't want their art used for AI training It is also happening in Derpibooru.

A different problem with Danbooru here. >start downloader for certain tag >stops finding new files prematurely >check search log >last checked page shows 0 new files found >open last checked page in browser <19 images on this page (all of them) require a Gold account >many more pages after it >Hydrus doesn't try to download the next page despite there being a next gallery page, presumably since 0 files were found Is this a downloader limitation that I can fix myself or is this a Hydrus problem? I remember in the past that Hydrus would check one page after hitting 0 new files, was that removed?

>>18380 On v501 in Linux, the shortcuts under "the actual media ... (mouse only)" have stopped working, or at least become very inconsistent. I have mouse bindings to pause/play and seek through video but they don't have any effect most of the time. Occasionally certain files will let me use the bindings but which files work often changes when I close and open the media viewer again.

>>18391 You can manually continue the page search by right clicking on the last line of the search log, and then click "Try again, and continue with search" or something like that. I just discovered this myself. >Many more pages after it Not sure if it's going to work in this case, maybe try and re-search the page right after the gold account line.

HELP i accidentally opened hydrus twice. the first window loaded my previous session as normal, but the second window is completely empty: no session, no files, and it says "it looks like this is your first time using hydrus". i'm scared that the second window is going to overwrite my db. i opened the options menu in the second window in the hopes that while it's open it won't save anything to disk. what do???

>>18394 Kill Hydrus with the Task Manager. Problem solved.

Is there a way to clear these? This is getting a bit ridiculous

>>18395 i just closed it as normal and nothing happened. still, i'm curious what would have happened if i had added images to the second window.

To whatever janitor sent me the message "Read the difference between siblings and parents," get fucked. I know the god damned difference because I am basically the only fucking person translating the Japanese tags dumped from Pixiv into the PTR, and I'm doing so as fucking siblings as should be. And not even just the "oh pixiv or danbooru says it's this," tags, but the random bullshit, too. My own translations, and always with custom notes about it, often times mentioning how I came to a decision on the more dubious translations. If something was placed incorrectly, it was because of a mis-click, but if you want to be passive-aggressive fuckers about it, I can just keep all my shit off the PTR entirely.

>>18393 >10000 You won, anon.

>>18399 Jannies don't care. They don't have any accountability and do it all for free :^) Enjoy your images having tons of irrelevant PTR tags because they're part of an artist collection or whatever on E-hentai.

>>18401 Well, when I translate tags, I do it painstakingly. I cross-reference with multiple sites, multiple machine translators, I consider the context, I even sometimes take the time to understand the roots of words, because there's sometimes slang and portmanteaus that don't actually have meaning of themselves. I do this, and I do it publicly, because I want to help make everyone's experience with Hydrus better, but if I'm just going to catch shit from some janitor over a mis-click, clearly none of that matters, so I'll just keep it to myself.

>>18387 same problem on Fedora 36.

>>18399 >because I am basically the only fucking person translating the Japanese tags dumped from Pixiv into the PTR, and I'm doing so as fucking siblings as should be. Well, we appreciate it dude. Those tags are a bitch.

If I were to delete my Hydrus backup folder, then press update database backup in Hydrus, what would happen? Would it let me know the folder was missing? Would it rebuild the entire folder without question?

>>18392 >>18393

>>18359 >>18360 I've decided to rewrite my tiling system to make one supersize tile that you never see the edge of instead of trying to stitch a bunch of little ones together. It has caused too many headaches and isn't worth the ~50% saving in memory. I think setting the tile size to 2,000+ option is bugging too, it uses some fuzzy math to get a 'nicer' size and I think it is scaling down even then, forcing small stitched-together tiles. I am not sure when I will get this done, but it is on my mind and written in my top priority list. >>18361 I saw this earlier and tried to hack it better for 501. Whole thing is now in the 'search' submenu when you right-click. Try the new 'invert selection' or 'exclude', whatever it gives you that sounds right. >>18362 Thank you, this is really useful. If it is not a huge ton of work, I would like that render API as a different embed option for users who want to try it out. It might fix macOS too, which has been another persistent mpv problem. >>18363 >>18364 Jesus, that's crazy. I don't know how the AUR build/deploy script works, so they would be the ones to remove/skip an actual build from the AUR package, but yeah hydrus will be fine if mkdocs isn't there. It isn't in the main requirements.txt and it doesn't import it in the code, it is only needed to convert the 'docs' directory in the source package into the html 'help' directory. If that 'help' directory is missing when you boot hydrus, it doesn't care, and if you try to open help->help, it'll just say, 'looks like you don't have local help, do you want to open the website?'. I feel like there must be a version of mkdocs that doesn't need all that shit, the idea it needs python2 seems crazy. I just looked up how we do it for the official github builds, attached as screenshot. There's nothing special about the install command, but seems like it doesn't need the stuff in yours? Maybe mkdocs-minify is something from an old version, or Arch specific?

>>18365 >>18376 Thanks. These links are great, and I agree that we should expect to plug this tech in in the future. I don't think we need new system tags (especially as that will be slow, since I'd have to write them and their database tech), instead this can all be handled by tags. You could pipe one of those classifiers into the Client API to scan a certain set of files and write back 'has face', 'emotion:happy' or whatever you wanted on a 'ml tags' local tags service, and then search them as normal. My part I think will be in providing the most helpful and flexible platform for that. Lots more I can do with the Client API. I am also thinking of writing a background maintenance task that will do the reverse of this, pulling instead of pushing, where it will send files to an external executable call and take the tags it returns and do stuff with them. We've had similar ideas for slow-burn waifu2x style conversion work. Might be that is wasted though and what we really need is an external uniform bridge, just some python library or daemon, that makes it easy to plug any classifier into a tag in hydrus, and a comfortable way to run these things in the background without overheating your GPU, we'll see. Obviously this tech is kicking off right now, and I am interested in pretty much all of it. I'm talking with a guy right now who is going to be using hydrus to train forked iterations of the stable diffusion model, so I'm keen to see what happens and how I can help. >>18367 Pretty sure it is going to be the internet of the next generation. Not sure which careers survive unscathed tbh. Normies clearly aren't ready, some artists (particularly Japanese, I've seen) are already going neo-luddite. >>18366 You can now unpause them! Tell your DA subs to retry ignored. >>18368 I'm afraid I don't know. I haven't worked on sank in a long time, and I don't use the site much IRL, so I'm not familiar with its quirks. Are you logging in with Hydrus Companion? That tends to be the best solution for any sort of spicy nsfw login issues. It feels by that error that there is a new thing requests have to include to fetch that content though. 'invalid-parameters' when presumably any other request is fine sounds like you need to add a new token, or maybe spicy content is just prohibited on that API now. >>18369 >>18370 >>18371 Beyond the annoyance of slightly different resolution ratios, if you have noticed tile warping like in >9950, I have a plan to fix this.

This is probably a dumb question but how do we see when our backup was last updated? If i right click on the client_files folder it says Date Modified: Oct 2021 but if I go and click on a specific, new image within that backup it will show Oct 2022. Don't know if it matters but I'm using Dolphin file manager.

>>18372 Thank you, this is useful. It currently always battles the kings of each group against each other, but the order at which it presents you things is really bad math and it is one of my next big priorities. I want to add options to this, and a great option, default on, would be 'sort the duplicate processing filter by filesize descending'. Then you'd establish your F early on as the king and it would make all the other decisions easy. In fact, as you say, really I should try to sort by the predicted score. And yeah I could have options to maximise score difference in the pair or minimise. This must be roughly doable. I updated the code behind the filter recently and am more happy with it, so I now have the breathing room to improve the db fetch code. >>18373 I've had another request like this recently, and I like the idea. The sort/collect controls are getting super cramped, but if I compress and tier them down a bit, I think adding some more search options to differ collection sort vs internal sort would be neat, and then I can do average filesize as well as total filesize. >>18374 Sorry to hear about your problems. Yes, you can import your old files easy. Two things: 1) You want the 'fxx' subfolders in there, not 'txx'. Should be 256 of them. Import them in batches. 2) You may get some 'could not parse/understand 50 files' when you import. If you synced with the PTR, don't worry about those warnings. That's extensionless update files internal to hydrus and you can ignore them. If you still have an old client.db and client.master.db, let me know, we may be able to recover your inbox statuses. >>18377 No great way yet, but I want to have a really nice solution in future, basically a wizard and then one-click and it just does it. For now, if you cannot wait, you'll want to investigate tags->migrate tags to export some HTA files that you can then import into the other client. It can do parents and siblings too. URLs and Subs have no easy import/export yet. URLs will soon. Subs you should probably just copy/paste the lists of query texts across. You can try and 'export' the subs from the dialog into png files that import the other side, but I think this is buggy due to how complicated subs are. Most important: make a backup before you start, just in case anything goes wrong. >>18379 10k is a good estimate for when things just start to go laggy, although I know some people with 100k or more. I know some gigabrains have had 2 million+ URLs in their subscription across many pages. There's about a 400k limit per page due to database stuff. Best way to do it is open a new 'download->urls' page, and then copy/paste 5k at a time using the paste button. Make the URLs newline separated. See if things get mega laggy after 15k, let them run for a bit, and once a bunch are finished, click the arrow by 'file log' and it should have 'delete x successful from the queue', which will clean things up, and then paste some more.

>>18383 Thanks, as here >>18407 , I don't make the AUR package, so I'm afraid I cannot fix this. I think if AUR package could remove that requirement, or figure out a way to get mkdocs that doesn't need that python2 bullshit, that's a good way forward. >>18382 >>18390 When it comes to people losing work because of ML, I hope for the best. I don't think it will be possible to resist. The tech is too good and seems set only to get better and better. Where we will be in five years, I really do not know. Maybe there will be hardware constraints, or training models to do real work like designing bridge supports will be too much work, but I dunno man, I feel like we are going to see a lot of depreciated earning power. The seethe is from a few online artists right now, but I can't help thinking it'll be a neo-luddite political movement in the end. Banning self-driving cars and virtual girlfriends and so on. As for doing full site archives, yeah hydrus isn't a fast downloader. Building a collection is a marathon, not a sprint. Try and focus on the artists you want to save. Also the bandwidth rules under network->data can be helpful, but don't be so rude to a server that you get banned. >>18387 >>18403 Damn. Thank you for this report. Did you have to install your own libmpv1 with 'apt' to get mpv to work to start with, or did this always work out of the box? A Linux source user was telling me they had an mpv problem when they were playing with the 500 build, they solved it by deleting some .so files in the main install dir. I've asked them what their routine is, if they remember it, but if you have any ideas, you might like to try the same thing. If I can pin down a process on what .so files are superfluous, if that is the problem here, I can remove them in the build script.

Is someone else having trouble downloading from Newgrounds and Artstation with hydrus? The url import works fine for both of them although i can’t download Newgrounds videos, only images. However the gallery downloader and subscriptions doesn't seem to work for both, in the case of Newgrounds when downloading using the gallery I get the error "The parser found nothing in the document!" and in Artstation the error is "403"

>>18391 The 'gallery page seemed to stop because no real results' has no great solution, I'm afraid. Twitter has done the same things in certain versions of our downloader, when an artist goes on a 20+ text tweet thread and makes the a page of results have no media. A future version of the downloader may be able to detect and gracefully pass over such pages, but for now you have to hack a 'resume' answer as people have said. If there's any way to say '-gold_account_posts' or anything, that's a neat solution. >>18392 Thank you for this report. Can you try doing your shortcuts while you have help->debug->report modes->shortcut report mode on? What popups do you get? Are the clicks being caught? Which window seems to capture the event, and which wants to handle it? All the stuff is dumped to your log file in your database directory, so if you would rather just send me the lot to look through, you can just cut and paste from that into pastebin or email it to me. Don't put your whole log online in public, as it may contain sensitive info. >>18394 >>18397 I think you are fine. Sounds like you had a really unlucky start and hit a lock at an odd time. If hydrus can't get a write lock on its database directory, it reverts to (your user directory)/Hydrus as database location, which is most likely what your second client instance did here. Check that location, there's probably a tiny database there. This is a protection if you install to Program Files or similar (macOS also uses it). I will have a look at the boot code to see how this could happen, since it is supposed to be atomic and not do this. If your client is safe and you do not have a backup, take this event as a message from God to set up a backup, lest one day it be real and you have no recourse. >>18396 I agree. I'll figure something out. >>18399 >>18402 Thanks for your work, I appreciate it in my own hydrus use. I pissed off the jannies this week when a cached number serverside started counting wrong and gave them a ton of 404s any time they tried to load up new petitions.

>>18405 Yeah I think it'll update without question. The basic routine is: If file exists in db but not in backup, copy it If file exists in backup but not in db, delete it If file exists in both, if filesize or modified date changed, overwrite it It might moan if the whole folder is missing. Try it out! If you want to do anything at all clever with your backups, I strongly recommend you move to your own backup solution with FreeFileSync or similar. My internal backup is ugly and I don't like it. >>18409 I don't think I store one. Good idea though, I should remember it and say 'last backup xxx ago' in the menu. >>18412 I heard Newgrounds was broken, but I think a fixed parser was floating around, I will make sure to chase it up. Artstation went heavy on the Cloudflare protections, you get 403 unless (I think) you use Hydrus Companion to copy your cookies across now, like logging in.

Deleting a Sibling shows an error. See pic. Also, the sibling is deleted fine. ----- Using "Hydrus.Network.499.-.Linux.Qt6.-.Executable.tar.gz" on Linux MX (Debian 11 stable) ----- How to reproduce. 1 - Open the "manage tag siblings" dialog 2 - Select 1 sibling 3 - Press the "delete button. 4 - A popup rises showing the error. 5 - Even though the error, the sibling is deleted fine.

>>18411 >Damn. Thank you for this report. Did you have to install your own libmpv1 with 'apt' to get mpv to work to start with, or did this always work out of the box? Yes, I installed libmpv1 with the package manager a year ago.

>>18413 > If there's any way to say '-gold_account_posts' or anything, that's a neat solution. Adding "-loli -shota" deals with the gold-only content (but doing so exhausts all two tags that non-gold members can use at once). Even then it's possible to come across entire pages of banned artists that cause it to stall. For some reason "-banned_artist" does not remove those posts from the results either.

>>18411 >they solved it by deleting some .so files in the main install dir. you're talking about libgmodule-2.0.so.0 right? I've been doing that for a while now since MPV stopped working unless I did that a while back, though it used to work without it. Now even having done that it stopped working after V499 for me. I have the same exact error as >>18387 which I think is a different error than the one I got before with the whole libgmodule thing. Since the error mentions libmpv.so.1 I tried replacing the one that comes with hydrus with my system one, but hydrus just didn't recognize it at all and said something like "not present when application was frozen" or something like that. So I don't know what to do.

How should creator tags be handled for ai generated artwork, such as by waifu-diffusion? Should the creator be the person who generated the image (giving the prompt, post generation edits and things like that) or the ai generator used to do so? I'm leaning towards the former in my db and having the generators be a meta tag, but I'm curious what others think.

>>18419 >How should creator tags be handled for ai generated artwork >but I'm curious what others think. It is an abomination, that said it might be tagged like author:machine generated IMHO.

>>18410 >URLs and Subs have no easy import/export yet. URLs will soon. By "soon" do you mean like within a couple of weeks, or by the end of the year?

>>18420 >It is an abomination I don't know man. years ago I would've agreed, but things are fast changing. pics related are all ai generated. You can still tell that they're ai generated for sure, but it's starting to get more difficult, and the ai models are only going to get better as they get improved and get fed more and better training data.

>>18419 Personally I'd tag it with the AI/model name/version like 'creator:stable diffusion' and/or 'creator:waifu diffusion v1.3' if there are any substantial edits after generation then also tag the editor. I'd also include a note with the prompt and parameters used to generate the image.

>>18422 >I don't know man. years ago I would've agreed, but things are fast changing. pics related are all ai generated. When art turns into an industrial commodity, it becomes worthless. Think about it. So you may think it is for the better, well, synthetic art might become so prevalent and its quality so good that it will become a given at any time and at any place. Then immediately the normies will jump to something more exciting: the bizarre, wicked, ugly, and horrifying; and that is a fact. It would be better if an EMP ends this satanic technological progression before is too late.

Can someone leave a comment on the AUR for the guy who maintains it to let him know that Dev said MkDocs should be optional by default? I'm stuck on v499 until it's no longer required.

https://danbooru.donmai.us/posts/1762437 (pic related) fails to import (either locally or via downloader) because >Could not render a thumbnail: float division by zero May have something to do with being apng?

>>18419 I would tag the person who generated it as the creator, and have a tag for the AI used to do so like "meta:waifu diffusion". >>18424 >When art turns into an industrial commodity, it becomes worthless. Think about it. No. I don't subscribe to pseudointellectual bullshit like that. Good art is good art whether it was made by an AI or a human, just as bad art is bad art whether it was made by an AI or a human. One piece of artwork isn't any more tainted than another just because it was made with a GPU or a stylus or a pencil.

So, I've been trying to add video downloading to the DA parser, and it's kinda going like shit. In the body of the page, there's a script that includes a big chunk of JSON, and it includes all the video urls, but they're mixed in with a block of other related deviations, so it needs to ability to grab variables (the deviation id) and use it as a selector. Parsing this actually seems to be faster, so if we had a way to select the correct deviation, it could make a good new parser over all, since it seems to present everything extended_fetch did. There's also video embedding, which doesn't seem to be available for every video, but seems to work for those it is. Here's the parser to nab the urls. I'm still making the url class and page parser. [30, 6, ["get video embed url", 7, [59, 2, [[26, 1, [[27, 7, [[26, 1, [[62, 2, [0, "video", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 2, "src", [84, 1, [26, 1, [[55, 1, [[[9, ["<video .+></video>", "https://backend.deviantart.com/embed/film/"]]], "<video controls=\"\" height=\"493\" src=\"https://wixmp-ed30a86b8c4ca887773594c2.wixmp.com/v/mp4/3d0e8d8f-ee8b-4307-bacd-50bf2c3f8c70/dei07r6-e66bc5ec-306c-4ec6-8be8-7f6052775b7a.VideoQualities.res_720p.64df71078d8e42848f110e955069d19c.mp4\" width=\"1027\"></video>"]]]]]]], [60, 2, ["url", [84, 1, [26, 1, [[55, 1, [[[9, [".+?(\\d{7,11})$", "\\1"]]], "https://www.deviantart.com/imbapovi/art/KonoSuba-Darkness-slime-body-inflation-mp4-video-876769602"]]]]]]]]], "\\1\\2/1/", [84, 1, [26, 1, [[51, 1, [2, "https://backend.deviantart.com/embed/film/\\d+/1/", null, null, "https://backend.deviantart.com/embed/film/876769602/1/"]]]]]]], [7, 10]]]

>>18428 Okay, here's the parser and URL class. Gave it some light testing and it seems to work. Enjoy downloading DA videos again.

Boys pls help me ;_; https://regex101.com/r/KGTL1H/2 how do I remove a generic file extension at the end? I want to basically match all the keywords between the '_' after the last occurence of '/' in the path but I don't want to include that '.opus' as it is in this example. https://regex101.com/r/KGTL1H/2 pretty-please help me get this done?

>>18430 I _think_ this does what you need? Adding the dot in your lookaround and in the negative character class at the end. https://regex101.com/r/IuL7dR/1

>>18431 you beautiful motherfucker, thanks anon Is there also any way to *not* have to use a namespace when importing into hydrus? I'm going now, but will check back tomorrow. Thank you very much, really.

>>18431 >>18432 Sorry for being dumdum, I simply didn't see the forest for the trees. There were of course two spaces to enter the regex into so now I have my stuff tagged.

>>18429 Thanks! Does it also download tags? I had tags downloading in the old one, but it doesn't work anymore, so...

I've been getting "Too much data: Was expecting {some amount of kb} but server continued responding" errors that is jamming my kemono.party subscriptions. Is there something I can do to skip the problematic query and stop having my subscription to be placed on delay?

I have been using the program grabber with the option to save tags to .txt then importing the files into hydrus with the import with .txt option. However some tags has been given the unknown: namespace so instead of for example high heels the tag is now unknown:high heels. Is it possible to convert all the tags with the unknown namespace to just the tag itself?

>>18436 what you're asking for is essentially "namespace relationships", specifically siblings. Nothing like that exists currently, but it seems to be requested often. you might be able to do a 1 time conversion with direct db manipulation but that's dangerous.

>>18437 >direct db manipulation For the sake of your precious data, don't.

I had a good week working on a variety of bug fixes and UI quality of life. The tag autocomplete dropdown gets an important de-janking overhaul, and the system:time search predicates are easier to work with. The release should be as normal tomorrow.

>>18434 Mine gets tags, though I guess it's possible I added that, though I don't remember doing it. If you don't already have it, import this content parser to your "deviantart file page parser" [30, 6, ["user tags", 0, [27, 7, [[26, 1, [[62, 2, [0, "main", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "div", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "div", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "div", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "div", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "div", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]], [62, 2, [0, "a", {}, null, null, false, [51, 1, [3, "", null, null, "example string"]]]]]], 0, "href", [84, 1, [26, 1, [[51, 1, [2, "^.*deviantart\\.com\\/tag\\/.+$", null, null, "https://www.deviantart.com/tag/tape"]], [55, 1, [[[9, ["^.+deviantart\\.com\\/tag\\/(.+)$", "\\1"]]], "https://www.deviantart.com/tag/tape"]]]]]]], ""]]

>>18438 Why not? Are you the kind of helpless dupe who doesn't maintain backups or already have a proxy set up so that you never make changes to the production copy of it until you've alpha tested the changes?

I'm new to this kind of stuff and I'm reading through the help documentation as I use this for the first time. Just to be sure, for downloading, I just go to pages -> new download page -> gallery, and then put in my search settings, right? I want to be sure I'm not saving things to the wrong spot or saving the files wrong as thumbnails or something. I'm taking it slow by testing it out with a few small artists on Danbooru, but will this method work if I want to eventually build up hundreds of thousands of images? Is there anything else a first timer should know because I am prone to fucking a lot of things up.

>>18442 Hit F9, download, gallery. Then pick your site you want to download from ( it defaults to Safebooru, I think ). Type in some keywords, and hit enter. You're good to go. All the files will be saved to the database in the folder where Hydrus was installed. I've downloaded over a million files so far.

Hey guys, can anyone download from Deviant Art yet? I'm on the newest 500, and I still can't download from Deviant Art. Search starts, but then goes to Done. Getting the same error as with 499. Looks like the API is still not working.

>>18443 That was fast and helpful. Thank you and thanks to whoever made Hydrus.

>>18442 >>18445 There's also a blacklist you might to look into. You can put tags in there you don't want to download. It's under import options, tags. Very useful when your downloading from someplace like Pixiv, where anything goes.

https://www.youtube.com/watch?v=SkOAc_eU9b0 windows Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt5.-.Extract.only.zip Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt6.-.Extract.only.zip Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Windows.Qt6.-.Installer.exe macOS (updated links, first version had boot trouble) Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v502a/Hydrus.Network.502a.-.macOS.Qt5.-.App.dmg Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v502a/Hydrus.Network.502a.-.macOS.Qt6.-.App.dmg linux Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Linux.Qt5.-.Executable.tar.gz Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v502/Hydrus.Network.502.-.Linux.Qt6.-.Executable.tar.gz I had a good week fixing bugs and improving UI quality of life. highlights First off, the tag autocomplete dropdown has some important behind the scenes changes. The way it is stored and drawn is better, the same updates the popup toaster and media viewer hover windows have had recently. In general, it means a lot less flicker and jank. If you have set autocompletes to embed in normal search pages because of bugs, try letting them float under options->search this week! Unfortunately, as a consequence of this change, I am disabling autocompletes floating in dialogs. Outside of normal search pages, they now always embed. There are hard technical reasons for this, and design ones too--I never liked how they overlapped apply/cancel--so if you were floating before and liked it, please bear with it for now and let me know if and where it is most inconvenient. I put some work into the system:time edit predicate dialog. Rather than the confusing '<' and '~=' radiobox, it now says proper english 'since' and 'before' and so on. Some of the calculations behind this are also improved. 'sort files by import time' now works between files that were imported in the same second. If you had a some small-image manga you wanted to sort by import time because it lacked page tags, this now works properly. I fixed some stupid mistakes that broke a part of the Client API (and Hydrus Web) in v501. Sorry for the trouble! full list - autocomplete dropdown: - the floating version of the autocomplete dropdown gets the same backend treatment the media hovers and the popup toaster recently received--it is no longer its own window, but now a normal widget floating inside its parent. it should look pretty much the same, but a variety of bugs are eliminated. clients with many search pages open now only have one top level window, rather than potentially hundreds of hidden ones - if you have turned off floating a/c windows because of graphical bugs, please try turning them back on today. the checkbox is under _options->search_. - as an additional consequence, I have decided to no longer allow 'floating' autocomplete windows in dialogs. I never liked how this worked or looked, overlapping the apply/cancel buttons, and it is not technically possible to make this work with the new tech, so they are always embedded in dialogs now. the related checkbox in _options->search_ is gone as a result - if you ok or cancel on the 'OR' buttons, focus is now preserved back to the dropdown - a bunch of weird interwindow-focus-juggling and 'what happens if the user's window manager allows them to close a floating a/c dropdown'-style code is cleared out. with simpler logic, some flicker jank is simply eliminated - if you move the window around, any displaying floating a/c dropdowns now glide along with them; previously it updated at 10fps - the way the client swaps a new thumbnail grid in when results are loaded or dismissed is faster and more atomic. there is less focus-cludge, and as a result the autocomplete is better at retaining focus and staying displayed as changes to the search state occur - the way scroll events are caught is also improved, so the floating dropdown should fix its position on scroll more smoothly and capably - . - date system predicates: - _this affects system:import time; :modified time; and :last viewed_ - updated the system:time UI for time delta so you are choosing 'before', 'since', and '+/- 15% of' - updated the system:time UI for calendar date so you are choosing 'before', 'since', 'the day of', and '+/- a month of' rather than the ugly and awkward '<' stuff - updated the calendar calculations with calendar time-based system predicates, so '~=' operator now does plus or minus one month to the same calendar day, no matter how many days were in that month (previously it did +/- 30 days) - the system predicate parser now reassigns the '=' in a given 'system:time_type = time_delta' to '~='

- misc: - 'sort files by import time' now sorts files correctly even when two files were imported in the same second. thanks to the user who thought of the solution here! - the 'recent' system predicates you see listed in the 'flesh out system pred' dialogs now have a 'X' button that lets you remove them from the recent/favourites - fixed the crash that I disabled some code for last week and reactivated the code. the collect-by dropdown is back to refreshing itself whenever you change the settings in _options->sort/collect_. furthermore, this guy now spams less behind the scenes, only reinitialising if there are actual changes to the sort/collect settings - brushed up some network content-range checking logic. this data is tracked better, and now any time a given 206 range response has insufficient data for what its header said, this is noted in the log. it doesn't raise an error, and the network job will still try to resume from the truncated point, but let's see how widespread this is. if a server delivers _more_ data than specified, this now does raise an error - fixed a tiny bit of logic in how the server calculates changes in sibling and parent petition counts. I am not sure if I fixed the miscount the janitors have seen - if a janitor asks for a petition and the current petition count for that type is miscounted, leading to a 404, the server now quickly recalculates that number for the next request - updated the system predicate parser to replace all underscores with whitespace, so it can accept system predicates that use_underscores_instead_of_whilespace. I don't _think_ this messes up any of the parsing except in an odd case where a file service might have an underscore'd name, but we'll cross that bridge if and when we get to it - added information about 'PRAGMA quick_check;' to 'help my db is broke.txt' - patched a unit test that would rarely fail because of random data (issue #1217) - . - client api: - /get_files/search_files: - fixed the recent bug where an empty tag input with 'search all' permission would raise an error. entering no search predicates now returns an empty list in all cases, no matter your permissions (issue #1250) - entering invalid tags now raises a 400 error - improved the tag permissions check. only non-wildcard tags are now tested against the filter - updated my unit tests to catch these cases - /add_tags/search_tags: - a unit test now explicitly tests that empty autocomplete input results in no tags - the Client API now responds with Access-Control-Max-Age=86400 on OPTIONS checks, which should reduce some CORS pre-flight spam - client api version is now 34 - . - misc cleanup: - cleaned up the signalling code in the 'recent system predicate' buttons - shuffled some page widget and layout code to make the embedded a/c dropdown work - deleted a bunch of a/c event handling and forced layout and other garbage code - worked on some linter warnings next week I think I have caught up on the biggest Qt6 crashes and other emergencies. I regret the bumps that happened over the past couple of months, but we got there in the end. Next week will probably be the last with a Qt5 release, so there will be some updates to the help for that. I am due a 'medium size' job week, so I also want to get my teeth into something. I recently overhauled how tags in .txt files were imported/exported, and I would like to do more work there. It would be nice to have URLs and/or more than .txt file formats.

>>18444 I think 501 has the rollback fix I did, so try the 502 I just put out now. It should fix you as soon as you update, but the old script I am rolling you back to isn't perfect. Let me know where it fails, I think probably on some login-required nsfw that has artist-disabled download buttons. >>18445 I make hydrus, I'm just a normal Anon who is luckily in a position to work on it. I am glad you like it. I'm bad at dealing with messages, so I force myself to catch up with everything on Saturdays. Let me know if you have any trouble as you learn the program--feedback from new users is always helpful. Just going through the help is the best way. Have a play with the program and see what works for you. Everyone usually gets overwhelmed by a million files (me too), so don't worry about it too much. Most important--set up a backup, so if anything goes wrong, you haven't lost everything. https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up

>>18449 >>1088 Thanks! Just installed Qt6 502, and Deviant Art is still not working. Looks like they might have changed or locked us out of their API. Can someone else try also to confirm?

>>18450 After search, it instantly goes to DONE. Search log shows "The parser found nothing in the document!" Going to the URL in Firefox shows: {"error":"invalid_request","errorDescription":"Invalid or expired form submission","errorDetails":{"csrf":"missing"},"status":"error"}

>>18450 >>18451 Tried using Hydrus Companion to import cookies from DA, but that didn't work either.

guys, how do i get it to put the tags of the files it downloads with the gallery downloader as the filename? trying to make an image reference set for stable diffusion model training with a specific tag

I made a quick AIBooru tag search downloader by copying the default Danbooru one and changing a few lines. I didn't bother with login stuff because they don't make any content exclusive to logged-in users as far as I know. Works on my machine. Now you too can archive masterpieces such as pics related.

>>18454 can i ask you to make something for that that would output filenames ready to be inputted into stable diffusions webui trainer?

>>18454 Lol, this should get interesting.

>>18447 >v502 Reporting on bugs. I just installed "Hydrus.Network.502.-.Linux.Qt6.-.Executable.tar.gz" and this bug >>18415 appears to be fixed. Thanks OP. However, >>18387 is still a problem as shown in the screenshots. I'm going back to v499.

>>18455 >training stable diffusion off of stable diffusion outputs But why?

>>18453 I think about as close as you can do that, is to get it to add a tag with the name of the gallery. Import options -> tags.

>>18452 >>18380 >The Deviant Art downloader broke last week. It seems like they locked away the API we were using. I have reset us to the original html parser, which still seems to work ok. Please check your DA subs and tell them to retry ignored, and let me know if there are any glaring problems. If you're still getting these errors, remove the API converter from the deviant art file page url class.

>>18264 Has anyone tried to install Hydrus on their android via linuxdeploy? Or something similar?

Are av1-encoded videos supposed to be supported by hydrus? I made an issue for its problems with the flatpak here: https://github.com/flathub/io.github.hydrusnetwork.hydrus/issues/5 but I'm pretty sure the flatpak owner disappeared. AV1-encoded videos work fine with the AUR build of hydrus as far as I can tell, but I'm wondering if the normally-released builds support this functionality or if this is just a quirk of compiling via AUR (and if this quirk could potentially be unsupported/dangerous).

>>18458 not gp (do people still call it that anymore?) but I could think of 2. >less of a problem regarding artists feeling like their work is being stolen, since it's an additional step removed from their art >if the situation continue at this rate (but that is a big if) it won't be long before it becomes easier to find or generate ai drawings, than find good human drawings to use as training data. Then you just take the good drawings to use as additional training data so it makes more good art instead of bad art. that being said, I notice that all of these ai models completely fall apart when it comes to any sequential art like comics, and anything with dialog. if it's gonna get any better at that, which will be very difficult, It'll need lots of human art. It can't make comics and text good enough to dogfood yet.

>>18455 What do you mean by "filename"? Are you trying to get the prompt? I know some posts there have the prompt info in the description, but as far as I know there isn't a way to make Hydrus add the post description as a note. That would be a nice feature though.

>>18464 trying to be as clear as i can. i want the tags of a downloaded image to simply be part of the filename. some files have way too many tags ofc, so it would be cool if you could also manage a list of tags you actually want to be part of the filenames. you know, the ones you deem important. its especially relevant for me as it would be absolutely perfect for use with stablediffusion

anyway i can get Hydrus to read the prompt from Stable Diffusion png's and have it autotag the images?

Does the duplicate filter handle notes yet?

>>18425 Anyone?

searching for similar files using the "all known files" option doesn't seem to search any files that aren't in inbox or archive, like previously deleted files. Is this intended behavior. If it is, it should probably be made more clear that the similar file search won't work on trashed or nonlocal files. Maybe just not allowing you to select those domains on that page or something like that? Is there any way to get it do this anyway? I was trying to find a way to get some similar alts to a file that I accidentally deleted in the archive/delete filter. There's no relationship set between them so I can't just search there.

>>18468 I don't use arch, but according to comments from a few people, it looks like mpv is broken on linux since v500 so you might wanna just stay on v499 anyway until that gets fixed.

In the meantime, he said in the comments >To work around this, you can build without the documentation by removing the packages from the makedepends list and removing the mkdocs invocation in build() as well as the help argument to cp in package() Can someone explain what he means by the help argument? Does he just mean remove the "-r help" and leave the rest? Sorry, I don't understand programming. Link to full code: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=hydrus

>>18470 Thanks. I'll hold off for now then.

>mpv "breaks on linux" >hey anon why dont you use linux? gee i dunno lol

>>18473 >a single developer-induced bug in an application is a good argument against an entire operating system Seems like a pretty desperate argument to me. What exactly do you have to gain from this?

>>18382 Try gallery-dl. Hydrus is great for personal collections, but for downloading an entire booru you probably want a CLI tool that can grab notes and commentary as well (read the config documentation to do this). >>18470 >it looks like mpv is broken on linux since v500 Works on my machine. I use artix btw

>>18475 >gallery-dl nta but i did see this. do you know if i can prioritize certain tags to be part of the filename when it's downloaded? if so, i wouldn't need hydrus.

protip guys. Don't startup a new version of hydrus before merging it with your current client. It'll create the db files which will then overwrite yours. ;_; I'm lucky that I made a backup yesterday night (after not backing up for a month and a half btw) so I should be okay, but restoring the backup will take a while.

>>18470 I haven't noticed any problems, been on v500 since the 3rd

>>18478 odd. Well it looked like a bunch of linux users suddenly couldn't use mpv in hydrus at the same time. since dev only tests on ubuntu afaik and he doesn't use one of those cross-distro solutions like flatpak or appimage for linux it probably has to do with differences from ubuntu's version of mpv or something. that's my guess anyway.

>>18479 >flatpak or appimage Nope. The breakage is in my case with a Debian 11 (Stable) based distro and using a .DEB libmpv1 package, plus the official TAR.GZ executable from Github. There is nothing fancy on my side.

>>18473 Better question: Hey anon, why are you using an OS tailored to give zogbots an absolute control on your property? Do you consent?

Is there a way to show Exif data of imported files?

Can I import files without having them go through the temp folder?

Around a week ago, I can't download from Danbooru. I can access danbooru from the browser fine, it is just hydrus. I get the following error: "This looks like an unsolvable CloudFlare captcha! Best solution we know of is to copy cookies and User-Agent header from your web browser to hydrus!… (Copy note to see full error) cloudscraper.exceptions.CloudflareChallengeError: Detected a Cloudflare version 2 Captcha challenge, This feature is not available in the opensource (free) version. During handling of the above exception, another exception occurred: hydrus.core.HydrusExceptions.CloudFlareException: This looks like an unsolvable CloudFlare captcha! Best solution we know of is to copy cookies and User-Agent header from your web browser to hydrus!" I've never messed with any of this stuff, but did look around the site without finding a clear solution. I installed the hydrus companion to see if transfering cookies using that would do the trick. I get notification that cookies transfer, but it doesn't solve the issue. Is there a solution to this problem?

>>18471 "help" is one of the folders that cp is copying with the -r recursive flag. Just remove the "help"

Fuck. Updated Arch and Hydrus and it won't start. Traceback (most recent call last): File "/opt/hydrus/hydrus/hydrus_client.py", line 24, in <module> from hydrus.client.gui import QtInit File "/opt/hydrus/hydrus/client/gui/QtInit.py", line 33, in <module> from qtpy import QtCore as QC File "/usr/lib/python3.10/site-packages/qtpy/QtCore.py", line 90, in <module> Qt.MidButton = Qt.MouseButton.MiddleButton = Qt.MiddleButton File "/usr/lib/python3.10/enum.py", line 480, in __setattr__ raise AttributeError('Cannot reassign members.') AttributeError: Cannot reassign members. Downgrading Hydrus back to 499 (last good) doesn't work so one of the Python/Qt libs has broken it.

Is hydrus supposed to grab a page and file even if the hash from the parser matches a deleted file's hash? I have the "force page fetch" options disabled for the site, and the parser properly returns a matching hash. Tested by: 1. Grabbing a post 2. Deleting the file and saving a record 3. Grabbing the post again 4. Hydrus downloads the full file again before skipping it because it's already in the database as "deleted" 5. File log shows that the hash was recognized properly

>>18487 actually now that I'm testing it also doesn't skip downloading files that are still in the database

>>18486 yeah, same crash here also on Arch.

feels like there might be something wrong with the total bandwidth used...

Do you ever post in the GG thread on /v/, Dev?

If you had trouble booting the v502 macOS build, please try this hotfix: https://github.com/hydrusnetwork/hydrus/releases/tag/v502a The Docker package is updated too, and I have updated the release post links above. Sorry for the trouble! >>18415 Thanks, as you saw, should be fixed in newer versions! >>18416 >>18418 >>18457 >you're talking about libgmodule-2.0.so.0 right? As I understand it, this guy did the same basic idea but on other .so files and it seemed to fix things. He never got back to me on what he did specifically, and he had an odd fix for the distutils problem in v500 too and said he was mixed up in what exactly fixed what, so his situation is not so clear -_- This issue is frustrating, it suggests somehow that the nice clean new version of python-mpv is unable to load libmpv1, when all other tests I have seen it is great at loading mpv and supports the new mpv 2 api. Can I ask you guys to try checking for a new version of libmpv1 with apt? This sounds stupid, but can you check if there is a libmpv2? The Windows mpv went from mpv-1.dll to mpv-2.dll, and while Linux is not my expertise in any way, is there any chance a similar library name changed happened? I know we had trouble with the older version of python-mpv in which versions of libmpv1 it would work with, and I had hoped this new compatibility release would only make that better, and while it has for most, for you it has not. If we can pin down safe vs bad versions of libmpv that do or do not work, I would love to roll that info into the help. I think this is the basic issue we are dealing with here: https://github.com/jaseg/python-mpv/issues/212 although his patch clearly didn't give you a nicer error message. In any case, I think the answer is to update your libmpv1 if possible. Any specific version info you learn would help, and if you can give the exact apt line that gives a working version, that'd be great and I can roll it into the install instructions.

>>18419 >>18420 >>18422 >>18423 >>18424 >>18427 I'm not involved in running the PTR any more, but my vote would go towards a general and generic 'creator:machine generated', 'creator:machine learning' or similar for now. It feels like things are moving too fast to label things specifically 'creator:stable-diffusion-1.3' or something, especially since people are now hacking together merged checkpoints of models. Maybe in the future there become like four popular ways of doing this work and we have better 'creators' to refer to, but for now it is all ML, at least in my head. I expect the big boorus will be making decisions, if they haven't already. I presume most are banning ML work from upload, doubly so as eastern artists are on the verge of the Butlerian Jihad, but maybe we'll get an ML booru appearing as a containment board for it all. Again, maybe that'll only be appropriate once this tech settles down and we get 10/10 aesthetic works out of it. Maybe then the culture will have fallen back to using 'creator:some guy', and it just happens that most of the work was ML and he just cleaned it up afterwards, maybe we'll be doing something else. For my part, I am still loving this tech and playing around with it. I learned how to do inpainting this week, and I generated some better hands and faces and things, but in that pursuit I lost the bigger aesthetics of the image, like trying to connect five excellent but oddly shaped puzzle pieces together. It seems like people are now figuring out easy consumer-level training and updating of models too, and new models are being posted every day. NSFW is the next barrier being broken. There is no way the corpos or cultural scolds will be able to hold this tech hostage, we are getting bananas stuff coming no matter what anyone wants. This is the first lapping of a decades-long tidal wave, and the normies are not ready. >>18421 This week, fingers crossed. We'll see how this sidecar work works out. >>18426 Thanks, I get the same error. I'll check this out this week and see if I can fix it.

>>18486 >>18489 Just upgraded Arch to see if it would break Hydrus, it did. Downgraded pyside6 to the previous version (6.3.2-1) and Hydrus started right up, so it looks like that was the culprit

>>18428 >>18429 Thanks, this is great! I will try and fold this into the defaults for the program, and I'll update my default html downloader to get tags like the API one did. >>18435 Can you please post/send me some of these bad URLs, so I can try my end? I've been updating the network engine's byte-counting checks recently, and have never encountered this specific situation before. >>18436 I'm upgrading the .txt importing tech this week. I will try and get a string processor plugged into the pipeline, which I have been planning for it, if I can find time. If I get it in, you should be able to add a regex that detects 'unknown:' and replaces it with ''. And yeah unfortunately converting namespaces within the program is not possible yet. I want sibling namespaces tech, many users do too, but it will be technically difficult. Best answer is to catch tags before they come in, or to use the Client API to delete/add them manually. >>18450 >>18451 >>18452 Sorry for the trouble. It looks like my automatic updater did not work for you. Maybe you once updated your DA downloader to a custom thing? I think you best solution for now is to go to the dialogs under network->downloader components->gallery url generators/url classes/parsers, and delete anything 'deviant art' related, and then click 'add defaults' and add anything 'deviant art' related back in. Fingers crossed, that will reset you back to the clean defaults, which I know shouldn't have this problem. If you still get trouble, try downloading some DA stuff with help->debug->report modes->network report mode on, and let me know what you see. That report mode will talk about what URL Classes and Parsers it is using, which will let us figure out the crossed wire here.

>>18461 Never heard of that. A couple of raspberry pi installs. I don't know anything about linuxdeploy, but the usual problems here are OpenCV and Qt. If you can get them built, everything else is usually easier. >>18462 Yeah, they should be good. I have an AV1 vid in my test suite every week that's fine. Looking at the error in your issue there, that's actually my error. It happens when FFMPEG returns no line information about frame data when it tries to render some of the video. Can you send me/post/link to an example video that doesn't work, so I can check it my end? I don't know if that ffmpeg you have supports AV1, but if you go 'ffmpeg -i file.mp4', do you get a nice output, or an error? Should be like pic related. >>18465 Try playing around with the hydrus file export system (right-click->share->export->files). There's a 'filename pattern' in there that lets you name exported files by their tags. If you want blue_eyes,samus_aran,bodysuit.jpg kind of filenames, you may be able to munge it with that. >>18466 You would need to use the Client API along with a png info parser--hydrus can't do that yet. >>18467 No, but it is near the top of my list. Several users would like this.

>>18496 >e.g. AV1 There is a test video attached to the github issue that causes the error with the flatpak version, but it doesn't cause any errors with my normal AUR install. Since the flatpak version doesn't appear to be supported by the author anymore anyway, I'm guessing there's just some dependency that's not being met with the custom flatpak configuration. Because of how flatpak works, running ffmpeg outside of the sandbox doesn't really indicate anything, but I can confirm that the mp4 you attached works fine on my normal system as pic related. Note I'm still on v499 due to all the spooky reports of v500+ hydrus on linux.

>>18469 Note that hitting 'all known files' locks your tag domain to something other that 'all known tags', and it searches for files specifically to the set tag domain, i.e. only files that have tags. It should show deleted/trashed files. If you think it is still broken, can you try hitting help->advanced mode and setting the file domain to 'deleted from my files' or similar using 'multiple locations' (scroll down) advanced input? There's some system predicates that may lock you to just local files. I think system:inbox and system:archive do this, no matter your file domain, but if you are searching for raw tags, then 'all known files' should show any deleted files that have those tags on the set tag domain. >>18477 Sorry for the trouble, and well done for making a backup. I make one right before I update every week (about an hour or two before I post the release), just in case something goes wrong. >>18482 I have very basic support now, I want more. If you have a file you know has EXIF, hit the 'cog' icon in the media viewer and then 'check for EXIF' data. In future this will not need a check, it'll know or not, and be in some way searchable at the database level. >>18483 No, I need to rename things for some libraries which can't handle unicode filenames. Use this if you need to use a different temp dir: https://hydrusnetwork.github.io/hydrus/launch_arguments.html#--temp_dir_temp_dir

>>18497 also I forgot that you can run the flatpak of ffmpeg directly so this is the result of the flatpak's ffmpeg on your video.

>>18497 Damn, sorry, I missed the file. It imports ok for me too, I'm afraid! I think it must be the ffmpeg version in the flatpak somehow, it can't render it, so when I render the first few seconds of the video in order to literally count frames, it doesn't work. I bet it is falling back to that weird frame estimation method because the normal output isn't giving sensible data either. >>18499 Odd, it feels like it should be ok. At the least, I will brush up that error. >>18484 Sorry, looks like your ISP range is CF blocked. This often happens when the site is under attack from some region. CF or the site will apply a blocking rule to 'Indonesia' or whatever, and everyone there gets the 'I'm under attack' page and captcha challenge when they browse to the site. Hydrus can't get through that page, so you can either change IP to a different region with a VPN, use Hydrus Companion to copy your browser's User-Agent and captcha-solved cookies into hydrus, or wait until the IUAM rule is rescinded. >>18486 >>18489 >>18494 Sorry lads, I just encountered this same shit a couple hours ago in my dev IDE. We finally got the new version of Qt that I was interested in, and it breaks our monkey-patched PyQt-PySide compatibility code. Downgrade to the old version please with this: pip3 install PySide6==6.3.2 I will fix this this week, however it needs to happen.

>>18487 >>18488 It should do the sensible thing, skipping the file and saving you bandwidth, in almost all cases. Sometimes, if the logic is confused, like if a file has multiple URLs from the same site, or the URL it has is known for several separate files, it will come to distrust those URLs and discard their 'already in db/deleted' status, but normally hashes are pretty iron clad. Some sites like Pixiv always fetch page data since they can have multiple files per URL. One thing that does happen is if Cloudflare optimises the hash, so the hash that the site says is different to what you imported before. This hits imageboards most often. You said the hash was matching, is the one from the site actually the same as the file, do you know? I think you can right-click a line in the 'file log' and see any md5 etc... that was parsed. To explain this better, here's an example: Site has a file, A. Generates Ah hash. You ask for page, site says Ah. You don't have Ah. You download file, Cloudflare turned it into B. You save B, associate it with URL for A, and store Bh hash(es). ... You ask for page again later, site says Ah. You don't have Ah. Hash takes precedence over URL logic. You download file... BTW: You can force the program to redownload a file, like you can with the 'force page fetch', but it is just under 'file import options' instead of 'tag import options'. If it looks like the program is not obeying the correct rules, like it is redownloading a deleted danbooru file, can you give me a simple script to follow that should reproduce the issue? >>18490 Thank you, I will check this. Are any of your bandwidth values under network->data->review... messed up? >>18491 I am mostly a lurker, rarely posting outside of here, but yeah, I post in several places around 8chan and the related ecosystem. I use my normal browser for it though, the Board Volunteer hydev account here only posts in this thread. I'm just another Anon when I take my hydev mask off.

>>18501 >You said the hash was matching, is the one from the site actually the same as the file, do you know? Yeah, if I check the file log, the parsed hash is the same as if I copy the file's hash in the database directly. I don't know which default parser to test with - default danbooru parser doesn't seem to have working hash parsing and default e621 parser doesn't seem to have working hash parsing. I'm currently using the kemono.party parser with some modifications to grab the sha256 hash out of the file path, which is the usecase I'd most like to have this functionality for as kemono.party files are large and I don't want to hammer their (probably donation-run) servers for no reason.

>>18500 not any of those anons but downgrading pyside fixed it for me, thanks

>>18501 >>18502 It took a while to find a large SFW image on kemono party to test with. I've attached my slightly custom spin of the normal kemono.party downloader, which can be used with this post: https://kemono.party/patreon/user/4333124/post/64578626 1. Input the downloader 2. Input the URL into a url import page 3. Check the 22MB png image file's hash is "sha256:c7cc3a00d9a19c9ca47f65ff22f435ac23edfbf556f35f200a49ccb8e42d8b1e" 4. Delete the png image, saving a record of the deletion 5. Input the url into a new url import page 6. Observe that the file still downloads again, but then is skipped on completion of download. 7. Check filelog to see hash parsed is "sha256:c7cc3a00d9a19c9ca47f65ff22f435ac23edfbf556f35f200a49ccb8e42d8b1e"

>>18504 Also the parser gets pic related as a result of parsing the post

>>18484 >>18500 Danbooru is getting hammered by the stable diffusion craze that's happening right now. I'm guessing that's why they have the protection in place. Who knows if or when that will stop. I deleted both my browser and Hydrus login cookies, logged in to Danbooru again, and sent the new cookies to Hydrus, but to no avail. I suppose for now the only way around this is a VPN.

>>18495 Ah, unfortunately I've restarted the client since then, so the network bandwidth had been reset. I didn't think it messed anything up, but was a fun visual glitch

Is there a way to make the manage tags dialog in the media viewer work like the manage tags dialog in the main viewer? As in, not committing changes until you close the dialog.

>>18500 >PySide fix This eventually worked, but I had strange trash in my local pip install dir that crashed Hydrus: /usr/lib/python3.10/site-packages/qtpy/__init__.py:214: RuntimeWarning: Selected binding "pyside6" could not be found, using "pyqt5" warnings.warn('Selected binding "{}" could not be found, ' v502, 2022/10/16 10:42:49: hydrus client started QObject::moveToThread: Current thread (0x5584439b5590) is not the object's thread (0x5584440ebae0). Cannot move to target thread (0x5584439b5590) qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/anon/.local/lib/python3.10/site-packages/cv2/qt/plugins" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx. [1] 171299 IOT instruction (core dumped) hydrus-client Didn't show up in pip list so tried removing ~/.local/lib/python3.10/site-packages/cv2 which fixed it. Hope this helps anyone with a similar problem and also didn't break something else on my machine

>>18493 when I said that I was searching for similar files, I meant by sha256 hash using the hamming distance predicate. Is that somehow locked to only local files? any file I ever imported should have its hash known even if it's deleted.

>>18510 I don't have a full understanding of how it works, but I'm mildly sure you need to calculate a special hash of every file for it to be compared - you can't use sha256 directly. see https://apiumhub.com/tech-blog-barcelona/introduction-perceptual-hashes-measuring-similarity/#How_does_pHashes_work

Does anyone know how to make librewolf work with chan.sankakucomplex.com? All posts just display the placeholder logo image. Even in troubleshooting mode and with all the librewolf specific options that I could think of turned off it still doesn't work. The reason I'm asking here even though it's a bit off topic is because I'm using librewolf in the first place because hydrus companion doesn't work with firefox anymore. If there's a way to just get hydrus companion to work with firefox instead of getting librewolf to work I'd be fine with that instead, but firefox refuses to let you install unsigned extensions no matter what.

>>18512 >chan.sankakucomplex.com Interesting. Also can't view https://chan.sankakucomplex.com/ with LibreWolf. Not a problem for me as I consider Sankaku a huge garbage pile. Seems to work fine with Firefox Nightly though. You can install the Hydrus Companion add-on in Firefox Nightly and use that for Sankaku stuff. LibreWolf is 99% Firefox after all. It may be as simple as changing LibreWolf's user-agent to fix it, but I have not tried it to see if it works.

>>18512 >>18513 Hmm, I thought it was just an issue on their part, I didn't think LibreWolf was the reason why I can't see anything.

>>18495 >>18450 Thanks! At one point, I had modified the downloader to get tags, so I'll try reseting it like you describe.

>>18492 >Can I ask you guys to try checking for a new version of libmpv1 with apt? This sounds stupid, but can you check if there is a libmpv2? >apt I'm not a skillful Terminal anon but I understand that the Package Manager is a GUI for the command apt-get, so I checked and the result is negative. No libmpv2 in the Stable Repo, nor the Testing Repo, or Backports Repo. Sorry.

So, I'm running from source on windows, and can no longer launch after updating to 502. v502, 2022/10/17 01:32:26: hydrus client started v502, 2022/10/17 01:32:26: hydrus client failed v502, 2022/10/17 01:32:26: Traceback (most recent call last): File "C:\x\Hydrus Network\hydrus\hydrus_client.py", line 230, in boot from hydrus.client import ClientController File "C:\x\Hydrus Network\venv\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import return original_import(name, *args, **kwargs) File "C:\x\Hydrus Network\hydrus\client\ClientController.py", line 26, in <module> from hydrus.client import ClientAPI File "C:\x\Hydrus Network\venv\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import return original_import(name, *args, **kwargs) File "C:\x\Hydrus Network\hydrus\client\ClientAPI.py", line 10, in <module> from hydrus.client import ClientSearch File "C:\x\Hydrus Network\venv\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import return original_import(name, *args, **kwargs) File "C:\x\Hydrus Network\hydrus\client\ClientSearch.py", line 19, in <module> from hydrus.client import ClientTime File "C:\x\Hydrus Network\venv\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import return original_import(name, *args, **kwargs) File "C:\x\Hydrus Network\hydrus\client\ClientTime.py", line 2, in <module> from dateutil.relativedelta import relativedelta File "C:\x\Hydrus Network\venv\Lib\site-packages\shiboken2\files.dir\shibokensupport\__feature__.py", line 142, in _import return original_import(name, *args, **kwargs) ModuleNotFoundError: No module named 'dateutil' Did the same update procedure I've been doing, pull repo, activate venv, install requirements_windows, launch, except this time it's dead. Went ahead and installed python-dateutil myself, but you should know that something broke.

>>18495 >I'll update my default html downloader to get tags like the API one did. Damn, guess I did add that, lol. My tag parser is here if you just want to use it: >>18440

>>18506 Maybe use "donmai.us" instead of "danbooru.donmai.us" for User-Agent Header That small change did it for me. Also cleared headers and cookies for it beforehand ofc.

Hi. Just doing another semi-regular pop-ins to check on the status of a feature. Specifically, making arbitrary ordered sets of images for things like art packs, variant sets, and short, simple comics. Is this feature in yet?

>>18509 >and also didn't break something else on my machine It broke Stable Diffusion, restored it from backup but unsure how to get these pip packages to play nice

>>18519 >User-Agent Header Is that in Network -> Data -> Manage HTTP Headers? I re-sent the cookies from Firefox with Hydrus Companion, and then I added an http header setting for donmai.us and used the same user-agent as the one for Sankaku Complex, but I don't know if that's the right idea.

>>18519 >>18522 Never mind, I got it. I used a more recent user-agent header and now downloads are coming through again. Thanks for the tip!

>>18521 Probably gonna have to put one in a venv.

Any news about supporting JPEG XL?

I'm currently recovering from a corrupted database and noticed my logs are filled with Traceback (most recent call last): Failure: twisted.internet.defer.CancelledError: Is this something I can ignore? It started after upgrading to 499 but I haven't had any problems using hydrus. It seems to happen when I download something with hydrus companion.

>>18525 hydev said before that he's interested in adding support for it, but he can't until the image processing library he uses supports it, since that's how he handles images in hydrus. I think it's pillow and unfortunately it doesn't support jxl yet. I'd love to see supported get added asap though. https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html

>>18526 Updating to 502 fixed it.

@hyrus_dev The "regex shortcuts" button is useful in hydrus. But when you click something, why does it only copy to clipboard? Why not also add it to the end of the text box if something is written there already, or at least add it to the text box if it is empty already. Would save a step if you just want to add a favorite regex when you're importing files. I know it saves only a second but it adds up over time to become an annoyance, when a fix is so easy.

>>18529 for that matter, being able to save "namespace:regex" pairs and load them with a button click (or by default) will be very convenient as well

Hey guys. Is there a way to sort pics that have only one known URL? I'm trying to sort for pics that have ONLY the Pixiv URL. System: Known URL only has options for Has or Does Not Have, but not Only.

>>18531 I guess it would really be Only the Pixiv Domain (Pixiv.net), and are not on any other site, like Gelbooru.

e621 says waiting for a work slot in downloader page.

>>18533 forgot. items show fine but is stuck at "waiting for a work slot" and never downloads anything

I had a good week fixing some bugs and improving how the program works as a build and from source. It is now easy to run the program from source on Windows. The release should be as normal tomorrow. ---- Hey, I'm sorry for the problems with dateutil and PySide 6.4.0. Should be all fixed tomorrow, and I put time in to the build script and requirements.txts to make sure this stupid build/boot bullshit won't happen so much again in future.

Hi Everybody, I am exploring Hydrus.Network.502a.-.Windows.Qt6.-.Installer.exe on a fully updated Windows 11 Pro 64 bit . After installing it on C:\Hydrus Network the clinet doesn't launch. I install Visual C++ Redistributable for Visual Studio 2015 The crash report: Traceback (most recent call last): File "hydrus\hydrus_client.py", line 24, in <module> from hydrus.client.gui import QtInit File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller\loader\pyimod02_importers.py", line 499, in exec_module File "hydrus\client\gui\QtInit.py", line 33, in <module> from qtpy import QtCore as QC File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller\loader\pyimod02_importers.py", line 499, in exec_module File "qtpy\QtCore.py", line 116, in <module> File "shibokensupport/enum_310.py", line 529, in setattr AttributeError: Cannot reassign members. Any help to get that fixed.

>>18536 I installed version 501. This works.

when I use the advanced search (the one where you can use grouping and all the logic operators) and then I save it after as a saved search, I have no way to get back to that search if I wanna edit it later, because the search that actually gets saved is the simplified version. could you make is so that a search remembers the original advanced version you input so that you can edit them after you save them instead of having to start over every time you want to make a change? This would be useful since advanced searches are often gonna be a lot more complex than the ones you just do right in the search bar, so it makes sense that you'd wanna adjust them more.

>>18527 Thanks. Guess I'll keep using WEBP for awhile. At least where quality doesn't matter. I wonder if we can expect support of WEBP2 in nearest future.

>>18536 Fixed in the release I am putting out now, sorry for the trouble! Fingers crossed, this shit shouldn't happen so much in future.

https://www.youtube.com/watch?v=VFyu1qvub7A windows Qt5 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt5.-.Extract.only.zip Qt6 zip: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt6.-.Extract.only.zip Qt6 exe: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Windows.Qt6.-.Installer.exe macOS Qt5 app: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.macOS.Qt5.-.App.dmg Qt6 app: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.macOS.Qt6.-.App.dmg linux Qt5 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Linux.Qt5.-.Executable.tar.gz Qt6 tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v503/Hydrus.Network.503.-.Linux.Qt6.-.Executable.tar.gz I had a good week improving how the program works behind the scenes. The build is more reliable, and it is now easy to run the program from source in Windows. reliability tl;dr: The program breaks less now and in the future. Just as I was hotfixing a fresh boot problem last week, in the intervening time yet another boot problem arrived! We've been hit by several of these in the past couple of months, and I hate causing people this stress, so I decided to put some time into the way the official release is put together. I really hope this will not happen as much in future. This is mostly technical, but the requirements.txts that prep the program environment are simpler, laid out for easier reading and maintenance, and have frozen version numbers for important libraries. I also put time into the build script itself, doing some cleanup and anti-deprecation work. The issues with PySide6 and dateutil are fixed. I want the program to be more stable going forward when I click 'publish'. It has been duct-tape-and-a-prayer for too long. Thank you to those who report problems, and please keep them coming if and when this new system fails. I want to keep improving things here, especially too as more users run from source. running from source Absent any more issues, I expect this to be the last week I put out a Qt5 build. The Qt6 build is stable with no giant outstanding problems. Users on Windows 7 (and some similarly old Linux and macOS versions) cannot run Qt6, so they will need an alternate solution. It is now easy to run the program from source in Windows. I have written several batch files to make it simple, no technical knowledge required beyond the ability to open a zip file, and the help has a brand new full walkthrough. If you want to run Qt5, the way to do so is equally simple and clearly labelled. Updating, too, is very similar to how you already update with the Windows extract zip. https://hydrusnetwork.github.io/hydrus/running_from_source.html The process is also simpler and clearer for advanced users who want to use 'git', 'venv', and 'pip' manually. If you have been running hydrus from your system python, you might want to check out the help to learn how to set up a venv--it is really easy these days, and you can save yourself a real headache. I'm obviously interested in reports if any of this fails for you; or if there is anything you find confusing. There is more work to do here, and once we have a workflow people are happy with, I would appreciate if some advanced Linux and macOS users can translate my new .bat files roughly to .command and .sh (I just don't have the familiarity/expertise in these platforms to do this well), so we have easy 'running from source' support for all platforms.

full list - misc: - fixed show/hiding the main gui splitters after a regression in v502. also, keyboard focus after these events should now be less jank - thanks to a user, the Deviant Art parser we rolled back to recently now gets video support. I also added artist tag parsing like the api parser used to do - if you use the internal client database backup system, it now says in the menu when it was last run. this menu doesn't update often, so I put a bit of buffer in where it says 'did one recently'. let me know if the numbers here are ever confusing - fixed a bug where the database menu was not immediately updating the first time you set a backup location - if an apng has sub-millisecond frame durations (seems to be jitter-apngs that were created oddly), these are now each rounded up to 1ms. any apngs that previously appeared to have 0 duration now have borked-tiny but valid duration and will now import ok - the client now catches 529 error responses from servers (service is overloaded) and treats them like a 429/509 bandwidth problem, waiting for a bit before retrying. more work may be needed here - the new popup toaster should restore from minimised better - fixed a subtle bug where trashing and untrashing a file when searching the special 'all my files' domain would temporarily sort that file at the front/end of sorting by 'import time' - added 'dateutil present' to _help->about_ and reordered all the entries for readability - brushed up the network job response-bytes-size counting logic a little more - cleaned up the EVT_ICONIZE event processing wx/Qt patch - . - running from source is now easy on Windows: - as I expect to drop Qt5 support in the builds next week, we need an easy way for Windows 7 and other older-OS users to run from source. I am by no means an expert at this, but I have written some easy-setup scripts that can get you running the client in Windows from nothing in a few minutes with no python experience - the help is updated to reflect this, with more pointers to 'running from source', and that page now has a new guide that takes you through it all in simple steps - there's a client-user.bat you can edit to add your own launch parameters, and a setup_help.bat to build the help too - all the requirements.txts across the program have had a full pass. all are now similarly formatted for easy future editing. it is now simple to select whether you want Qt5 or Qt6, and seeing the various differences between the documents is now obvious - the .gitignore has been updated to not stomp over your venv, mpv/ffmpeg/sqlite, or client-user.bat - feedback on how this works and how to make it better would be appreciated, and once we are happy with the workflow, I will invite Linux and macOS users to generate equivalent .sh and .command scripts so we are multiplatform-easy - . - build stuff: - _this is all wizard nonsense, so you can ignore it. I am mostly just noting it here for my records. tl;dr: I fixed more boot problems, now and in the future_ - just when I was getting on top of the latest boot problems, we had another one last week, caused by yet another external library that updated unusually, this time just a day after the normal release. it struck some users who run from source (such as AUR), and the macOS hotfix I put out on saturday. it turns out PySide6 6.4.0 is not yet supported by qtpy. since these big libraries' bleeding edge versions are common problems, I have updated all the requirements.txts across the program to set specific versions for qtpy, PySide2/PySide6, opencv-python-headless, requests, python-mpv, and setuptools (issue #1254) - updated all the requirements.txts with 'python-dateutil', which has spotty default support and whose absence broke some/all of the macOS and Docker deployments last week - added failsafe code in case python-dateutil is not available - pylzma is no longer in the main requirements.txt. it doesn't have a wheel (and hence needs compiler tech to pip install), and it is only useful for some weird flash files. UPDATE: with the blessed assistance of stackexchange, I rewrote the 'decompress lzma-compressed flash file' routine to re-munge the flash header into a proper lzma header and use the python default 'lzma' library, so 'pylzma' is no longer needed and removed from all requirements.txts - updated most of the actions in the build script to use updated node16 versions. node12 just started getting deprecation warnings. there is more work to do - replaced the node12 pip installer action with a manual command on the reworked requirements.txts - replaced most of the build script's uses of 'set-output', which just started getting deprecation warnings. there is more work to do next week I did not have time to do the sidecar import/export improvement, so I'll try again next week. I'd love to have 'neighbouring .txt file' support for URLs and .json/.xml, but let's see what I can manage. Thanks everyone!

Hey dev anon. I upgraded to 503, and erased everything but the db folder before I did so. Still can't connect to Deviant Art. Just get a DONE. I think it's looking for a token of some kind. Can anyone else connect do Deviant Art? Using the Deviant Art Tag Search, and gallery downloader.

>>18543 Deviant Art Artist Lookup does work though.

>>18541 >Thank you to those who report problems, and please keep them coming Well, you asked for it. I'm testing "Hydrus.Network.503.-.Linux.Qt6.-.Executable.tar.gz" on MX Linux (Debian 11) and the bug mentioned at >>18387 and >>18457 is still there. I'm going back to v499.

I have next to 0 understanding of coding. That being said, I read through the entire hydrus starter guide, and went through some of the older posts in this board concerning my issue. I learned something from some of the posters in General #1, but wanted to know more about my specific issue. I was tinkering with the url importer and made it work with gelbooru, but when I did a loli page import, it skipped all of it. I then did a flat chest import, and noticed it skipped over some of the more obvious loli images. I tried looking into the parsers section but couldn't make heads or tails of it. Any help on this issue? Question 2: How can I import exhentai urls successfully? I read something about a hydrus companion but not too sure about how to go about using that, or if something new has come up over the last year.

I went to update my Hydrus install, I accidentally installed to the wrong folder so I deleted that folder after install. I went to my original folder, clicked client.exe and it was still working and good. I then tried again, selected the right folder. When I click install it says "Hydrus is already installed in this location, do you want to continue" It sounds like it will overwrite it, not update it. I don't remember it asking this when updating in the past. I thought it usually says it will update it, not ask if I want to install it there because it is already installed.

>>18545 >>18541 fedora 36, also still here. I just figured you didn't forget and were still trying to figure it out though, but yeah it sucks not having the player. I gotta remember to not update so often anymore.

could you add a scoring metric to the duplicate filter that's something like "this file has known url from X domain and the other doesn't". certain websites I find have a high tendency of having better dupes (most boorus and pixiv) and some tend to have worse dupes (most imageboards, twitter and reddit). It'd be cool to be able to factor that into the duplicate score. also smaller request but could you make the actual scores visible in the filter somewhere. It would help with fine tuning the scoring a bit.

>>18264 How do I get the URL downloader to ignore "previously deleted" files? If I manually give it a URL, I expect it to show up even if it's previously deleted but I can't find a setting for this. Another question: If I have a text file with URLs (separated by newlines) can I get Hydrus to download them and show them to me all at once instead of having to manually copy and paste them into the URL downloader?

>>18550 > text file with URLs. The paste button on the URL downloader tab will accept multiple URLs so long as it's 1 URL per line in your text file. It's not obvious you can do that, but the paste button has a tooltip that explains it.

Can someone please see if they can do a gallery download with the deviant art tag search? Any tag. Just so I can make sure it's not bad just on my machine. Thanks!

Is there a way to sort the images by the time they were uploaded to booru?

>>18553 Sort by modified time. Hydrus keeping modified times was added relatively recently so it will only work on files downloaded since it was added.

>>18525 I've never even seen a JPEG XL file before. Where would you find them?

>>18555 Not that anon, but you would probably make them yourself for the time being. It's still in development but very close to release, and software is just beginning to build in support for it. JpegXL is extremely good technology, and converting png/lossless webp/bmp to it is a large filesize savings for no degradation of quality. you can also losslessly convert pre-existing jpegs to it for some good compression gains as well, and then if you want to go the route of doing irreversible lossy compression to your images, then JpegXL will make them super tiny with very little quality loss, and practically non-existent generational quality loss ("do I look like I know what a jpeg is" blocking) As of right now, it is unquestionably the image format of the future. The only thing it doesn't do better than everything else is animated images/gifs, where avif achieves slightly better results (probably because it's based on a video codec whereas JXL is not) If it wasn't obvious, I'm very much in support of JXL support being added to hydrus

>>18546 gelbooru hides loli images by default. to turn them on, you have to go to settings > enable fringe content. this sets a cookie in your browser. you need to tell hydrus to use this cookie. exhentai also uses cookies to log you in. you can look at the cookies in your browser's dev tools and manually copy it over under network > data > review session cookies, or you can use hydrus companion to copy cookies to hydrus. it's a browser extension. https://gitgud.io/prkc/hydrus-companion how to install https://gitgud.io/prkc/hydrus-companion#installation-on-chrome-and-derivatives-tested-on-chrome-chromium-and-vivaldi setup steps https://gitgud.io/prkc/hydrus-companion#basic-setup-connecting-the-extension-to-hydrus >>18550 >How do I get the URL downloader to ignore "previously deleted" files? downloaders do this by default. if you give hydrus a url and it recognizes it as previously deleted, it won't download it. if hydrus is redownloading files you've previously deleted when you don't want it to, you should make sure the "exclude previously deleted files" box in the file import options dialog is checked. >If I manually give it a URL, I expect it to show up even if it's previously deleted but I can't find a setting for this. i'm not sure what you mean by this - is the url not showing up in the log at all?

>>18525 >>18527 >>18556 this was really interesting to learn about. I think I've seen various "jpeg" derivative file types but have never paid them any attention because I never saw the point. I watched this (old-ish?) showcase which was easy enough to follow and showed some really good results: https://youtube.com/watch?v=YsbsKimR-Fw&t=57s we really need support for some modern image codecs like jpegxl or whatever else is out there to fight these artists dropping 100MB PNGs or TIFFs. I've got about 30k pieces of art in hydrus and I like to collect the best copy, which often means large PNGs from patreon. I saw the hydrus github thread about adding new image formats but it seems like it's just a place to blackhole any suggestions. I'm not very good at the technical stuff but from what I understand, jpegxl's format structure(?) is frozen, which means that early adoption doesn't need to be changed later on? so I don't think it matters that it's still in development - the only thing that would change is better compression or something else. the video says that everything must be backwards compatible

>>18558 meant to attach this chart from the video

>>18555 You can find a few examples with side to side comparison here https://eclipseo.github.io/image-comparison-web/#nestor-meridionalis*1:1&MOZJPEG=s&JXL_0.3.7=s&subset1 I was really surprised by monstrous effectiveness of JXL and WEBP2 when I saw it.

>>18504 >>18505 I am sorry to say, because this test you provided was excellent, but this all worked as expected for me. It recognised the file was deleted with the sha256 hash and it skipped over it instantly, no redownload. Known URLs captured were: https://kemono.party/patreon/user/4333124/post/64578626 https://data74.kemono.party/data/c7/cc/c7cc3a00d9a19c9ca47f65ff22f435ac23edfbf556f35f200a49ccb8e42d8b1e.png https://kemono.party/c7/cc/c7cc3a00d9a19c9ca47f65ff22f435ac23edfbf556f35f200a49ccb8e42d8b1e.png This was on a completely clean, new client. So I think this may be something in your database. Could be a conflicting URL mapping somewhere causing the client to get confused about what matches what (although the parsed sha256 hash suggests it probably isn't an URL issue--hashes tend to overrule URLs), or it could just be a weird file import options. This behaviour is exactly what you would see from a FIO, in help->advanced mode, set to 'force file downloading even if xxx recognised and already in db/deleted'. Is there any way the downloader is set to that, or your default FIO in options->importing is set that way? Does the behaviour change if you set a specific FIO on the downloader page and ensure those checkboxes are off?

>>18506 Ah damn, that makes sense. I wish them the best. I've been talking with several users who are experimenting with using their already existing hydrus collection to train or update models, and I'm delighted I can help. I'm all about local CPU doing local work, and that ML as a subject seems to be working that way is neat. It is a shame the normal fixes won't do it for the CF stuff. I guess they upped their detection somehow. >>18508 I can add an option, I think. There's a couple advanced feature you would lose, like you wouldn't be able to navigate media with the dialog open, but sure, I'll write that down and I think I can do it. >>18509 >>18521 >>18524 Yes, please put your hydrus stuff in a venv. And use 'opencv-python-headless' instead of opencv itself. The 'headed'(?) version has Qt bindings or something that mess with Qt proper. I brushed up my running from source help this week if you need 'how does venv work' help: https://hydrusnetwork.github.io/hydrus/running_from_source.html >>18510 >>18511 ( >>18469 ) Hmm, I do keep the pHashes of every ever-imported file, so this should be technically possible, but I do prune the search tree to keep it snappy. It might be when the tree regenerates itself, it doesn't load up phashes for files outside the program. I expect to start offering optional automated dupe decisions soon, so I will probably want to change this. I will look into this and see what I can do. I think I can probably straight-up just fix your problem, please watch the changelogs and remind me if it doesn't get done.

>>18516 Damn. I had a quick look around, like here https://pkgs.org/download/libmpv1 , and it seems that ~0.34.x is a common verson to be 'stuck' at. I don't know enough about apt and how all that stuff gets approved to talk cleverly, but this https://github.com/jaseg/python-mpv/issues/212 suggests those are 'old' versions. Maybe it is a version requirement thing, for instance for that older(?) python3-mpv library I see also installed there, maybe it is just some guy was maintaining that build script and abandoned it. Perhaps we can figure out a clever answer here where we build libmpv ourselves, or we find a nice pre-built .so file somewhere like we do for mpv-2.dll on windows. Putting libmpv1.so file in the hydrus install_dir should work, afaik. Or, the probably more practical solution, is to make it easier for you and others to run from source on Linux, and then we can set you to the older python-mpv version really easily. I don't have any easy-setup .sh files yet, but I hope to roll them out in the coming weeks, modelled after the new easy-setup .bat files I figured out for the Windows users. >>18517 Very sorry for the trouble here. I had a clusterfuck of boot and library problems as 502 was coming out. 503 has it fixed. Please let me know if you have any more trouble. >>18520 No. I got delayed most of this year, so that was put off. I'm going to be in 'improving serverside stuff' mode through Q1 of 2023 probably, then I'll be back to big work. Here's the new megalist, btw, sorted by priority by people's reactions: https://github.com/hydrusnetwork/hydrus/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc Image groups is at the top, so I do hope to move on this sooner than later. >>18525 >>18555 >>18558 Just to chime in here, I am not an expert on the new formats, but in my general looking, I think I like JPEG XL the most so far. It is the least encumbered by patent bullshit and it supports a wide range of what I generally think people like us will want, along with sane implementations of HDR and friends. Can't say it will win the pending format war by any means, but as soon as OpenCV or Pillow or even FFMPEG support it nicely, I'm there.

>>18526 >>18528 This was harmless btw, in case you were worried. I catch and silence it better now! >>18529 Sorry, I have copy/paste on my mouse, so I don't think about this stuff. Sure, while I am hesitant to change behaviour now it is set, I think I can have it auto-insert, but I think I should insert at the current caret position, right? Not at the end? Actually: Maybe I should just have an option to say copy to vs insert. If I put the option on that menu itself, that'll keep things simple and easy to find. For >>18530, I'm actually going to do away with this system in the nearish-ish future and replace everything with the StringParser system I use in the downloaders. Instead of namespace/regex pairs, you'll just be regexing whatever from the original filename and then appending namespace with a string conversion rule. It will be slightly more complicated, but massively more powerful. Along with that change, I want unified 'favourites' for the whole system so you won't have to set this shit up so much. Let me know how it works for you when I get to rolling this out! >>18531 >>18532 No, but I like this idea. Search by num_urls and sort. There's a little extra difficulty here with the domains and stuff, and url classes too, since sometimes a file will have several semi-hidden and ugly direct file urls, but I'll see what I can do. >>18533 >>18534 Sorry for the trouble. I got this myself a couple weeks ago and I couldn't figure it out. I restarted my client and it fixed itself, but it seems like some downloaders are having trouble getting a work slot at times. Please let me know what else you discover if this happens again. >>18538 Sorry, I'm afraid I don't really have the infrastructure for this. That advanced input was actually written by another user who wanted to do his clever stuff, but it then perches atop my system as a result. I've never plugged that system in too tight simply because it is advanced and separate. Although it sucks to say, maybe it would be easier to have a sticky on your desktop with common phrases, and use that as an external 'favourites' system that you can refer to? You might also be able to use 'duplicate page' (when you right-click on a page tab) to help your workflow here (I'm thinking duplicate the original page and spam your advanced OR to the dupe. if it is wrong, make another dupe and spam the altered text to that, so you aren't deleting the mixed-in advanced tag junk over and over).

>>18539 Funny joke btw about webp: It has an animated variant, just like .gif or .apng, but (I guess?) due to some patent bullshit, you can't write a webp decoder. FFMPEG can encode animated webps, but not decode! Soon as they add a decoder, I'll be able to support animated webps. >>18543 >>18544 >>18552 Aha, I think I misunderstood your query, sorry. I get the same problem, the 'tag search' doesn't work. I always do artist queries on DA, so never thought to look. I will see what I can do! >>18545 >>18548 Sorry for the trouble. As per here >>18563 and here >>18492 , can you try updating your libmpv1 through apt-get? Any update on latest version numbers, or ideas on a place we could source a new libmpv1.so file? Otherwise, we may be looking at running you guys from source in a few weeks too. >>18547 You are good, and you can click ok. I agree the language there is intimidating. You only have to do this once, and it will go back to understanding that that is the normal place for hydrus to be installed. It is only upset because you are changing location from B back to A, and now this 'new' location A has stuff in it already, and it is letting you know. Best thing to do though, before any update, is to make a backup. Then, if anything goes wrong, you can just restore the backup and you are safe: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up >>18549 I mostly have the scores on hold. I want to replace the whole thing with a flexible new system that'll let you define the rules you want more easily. Having 'has x url' is a good idea for a metadata rule. The whole thing is ultimately going to start feeding info to an optional auto-decision maker that'll make decisions past arbitrary score thresholds. Exposing the scores is a neat idea, I will write that down. I am not sure if it would be ugly or overbearing to say it every time for every line, although I like the idea of displaying the score total somehow. Maybe I can put each score in tooltips over each line?

>>18563 >but as soon as OpenCV or Pillow or even FFMPEG support it nicely, I'm there. AFAIK FFMPEG has JPEG-XL support in trunk, but I don't know if any of the actual releases have it in yet.

>>18563 I'm not a programmer but do either of these projects help at all? https://github.com/novomesk/qt-jpegxl-image-plugin/ https://pypi.org/project/jxlpy/

>>18565 >webp Please don't. Proprietary image formats are cancer. No exceptions. Files with the extension .webp should in all cases be blocked at the data link layer, and the protests of fanboys, and the protests of salesmen trying to hard-sell this l33t new image format they want the world to buy licenses for proprietary software to view their proprietary image format are to be routed to /dev/null.

>>18564 >Sorry, I have copy/paste on my mouse, so I don't think about this stuff. Sure, while I am hesitant to change behaviour now it is set, I think I can have it auto-insert, but I think I should insert at the current caret position, right? Not at the end? >Actually: Maybe I should just have an option to say copy to vs insert. If I put the option on that menu itself, that'll keep things simple and easy to find. Sure, that will work. Thanks.

There is something weird going on with subscriptions. It shows that there should be around 434 new links in the file log, yet there are only 250. It has downloaded all 434 files, I search with the subscription specific tag, yet when I click on file log's arrwo >show new files in a new page, it shows aprox 237. I have a tons of subscriptions that only log the newest 250 files, instead of all the files found.

>>18568 >salesmen trying to hard-sell this l33t new image format they want the world to buy licenses for proprietary software to view their proprietary image format are to be routed to /dev/null I believe the buying part is restricted to create new files with that compression algorithm. Anyway, proprietary formats are undesirables.

Does the Linux version have a way to add local file domains? I looked into the manage services options but the closest thing I can see is to add a hydrus file repository. I assume thats the same thing but wanted to be sure. Sorry if it's a dumb question but I'm still learning this.

>>18568 Webp is and has always been an open format.

>>18565 I want to thank the Hydrus dev again for this awesome (in both senses of the word) work and software. I really have to ask: how did you even learn all the things required to program something like this? Anyways: also big thanks for making it availiable on linux. I'd really love to see you license it under GPL though, so your work can be insured and preserved to some degree. But it's of course only my opinion and absolutely at your discretion.

Just giving you a quarterly reminder like you asked, to work on allowing users to migrate parents and siblings from one tag service to another, only if they involve tags in "all my files". It's been pretty annoying having to manage this entire second tag service that I don't want, but I know you've working on a lot so just reminding you once again.

I have a problem with the kemono.party downloader where if a file was downloaded as part of some post in a subscription, then if the file appears again in a later post, hydrus won't add the metadata (e.g. title tag) of that new post to the recognized file. Is there a way to set hydrus so that it'll fetch post metadata for a file even if the file url itself is recognized, as long as the current post url isn't already a known url of the file (and thus the metadata should already be added)? The way it currently seems to work, if a creator posts 1 or 2 files from a longer post early as a teaser of sorts, then those files will only get the tags, such as the title, from that earlier teaser post and won't also get the tags from the full post that comes later. this means that they won't all have the same title tag from the full post added, even though they should.

>>18576 I should mention that I already have hydrus set to fetch metadata even if the file hash is recognized. The reason it's still not fetching the data is because the file url for a file is the same on kemono.party even if that file is part of multiple posts, so what I'm guessing is happening is that hydrus is recognizing that the file already has the same file url as a known url so it skips the page fetch, even though it's a different post that has different metadata that I also want added to the file. I could set hydrus to always fetch the page even when the file url is recognized since I see that option there, but I still want hydrus to be able to skip files when it's part of the same post, since always fetching would be very inefficient. so I'm kinda stuck with a dilemma atm unless there is a way to fetch data as long as the post url isn't the same.

could you add a way to edit the deletion reason for a file from the file log. I'd like to be able to edit it if I come across the file again but the reason I put before was confusing or vague.

I'm a week into using hydrus and was wondering what is an effective way to get non-booru files tagged? Is PTR the way to go there? I read it will take weeks to get the full tag library. What approach to use with new artist artwork that isn't tagged?

>>18579 try integrating saucenao with a tool, e.g. https://github.com/GoAwayNow/HydrausNao

I had a great week focusing on a full-scale update of the 'neighbouring .txt' import/export system. When you import or export files, you can now send URLs as well as tags; you can use .json instead of .txt; you can transform and filter text as it passes through; and you can set up workflows to multiple files at once. Everything is presented and edited in new UI. The release should be as normal tomorrow.

Absolute brainlet question here from someone that doesn't know github. Which file/files do I need to select and download to get the base hydrus functionality?

>>18582 NVM, Found the guides and videos in the github.

>>18579 Actually I found that the ptr was mostly useful for tags for files on boorus. I had very little luck with files that weren't on them back when I still synced with the ptr. It was pretty disappointing since it felt like I could've just got those tags myself.

>>18581 Sort of related question: Since video deduplicating is not really a thing, I often end up trying to manually trump video files over each other by manually copying tags and urls etc. if you get the two video files in the same tab you can click them both and do it through a deeply-nested command, but the process of getting them on the same tab is very tedious and if you don't navigate the context menu perfectly it will reset 4 levels of mouseover. I find it's just easier and faster to open the tags menu, copy all the tags, and paste them into the new file, then do the same with the URLs. To do this more properly, is it possible to make a generic "copy json metadata" and "paste json metadata" command that can be keybound? or even a "mark file to be superseded" and "supersede marked file" or some sort of paradigm that will allow proper video trumping in a way that doesn't make me want to gouge my eyes out when doing more than a couple at a time. A duplicates API could be really handy here as well, so I could do it myself via python. (also, when setting files as better through context menu, the old file is not trashed for some reason)

Any chance someone could fix the Flickr downloader? I took a crack at it myself, but it's above my head.

>>18580 I don't know what 'run a script' means or how to do it. I already have python installed. My pip is installed but not recognized by system, even though the environment variables are set up accurately. I have python in appdata. >>18584 Did you wait the several weeks to get the full 60gb+ of the repository? If it really is just boorus then it is a complete waste of time for me to wait for it to complete.

>>18587 yes I did and yes it did feel mostly like a waste of time for me, which is why after getting over the sunk cost fallacy, I stopped syncing to it and deleted it. I do think it would be great to have a fast way of getting files tagged, but the ptr in its current form really just doesn't seem that useful to me. It mostly gets tags that you can easily get yourself. It means you don't have to get them yourself though so there's that. If you really do have plenty of storage space and drive writes to spare then I guess it doesn't hurt, but it just didn't feel worth even the overhead of having it there alongside my local tags, so I dropped it.

>>18587 if you don't know how to run scripts you're going to have a hard time with hydrus. I would figure out what's wrong with your python configuration, because most hydrus automation uses python. you're not really meant to interact that much with hydrus in a manual fashion, rather you're generally supposed to grab the hashes of files you want to target (right-click > share > copy > hash sha256) and import them into a tool like HydrausNao so they can access your database and perform wizardry. Being able to write simple python scripts with the "hydus-api" package will help a lot if you have usecases that no one else has scripted yet. I think most people using hydrus the way it is intended have tens of thousands of files, so any sort of manual intervention is generally futile also just a warning, saucenao only lets you make 100 queries per day unless you pay for an account. it's still basically the best way to do automatic reverse image lookup though, so if you get it working and hit the 100 query limit, it may be worth paying for a month to do a one-time mass-lookup on your files.

>>18587 I personally don't believe in the PTR, but I'm a bit of a control freak. it does not sound appetizing to open up your database to other people's decisions, along with troublemakers and tagging styles that don't match your own. imo if most of your database can be sourced from well-tagged boorus, integrate with those instead and don't use the PTR

>>18589 >if you don't know how to run scripts you're going to have a hard time with hydrus >you're not really meant to interact that much with hydrus in a manual fashion those are some pretty bold claims. I consider myself an intermediate hydrus user and I've been using it for a a few years now. I have yet to write or run any scripts like you're saying, and depending on your definition, all my interactions have been manual.

>>18591 You might be doing yourself a disservice depending on how you use Hydrus. You can go without using any of the automation that Hydrus offers, sure, but you will probably be wasting a lot of time. I'd advise reading the manual https://hydrusnetwork.github.io/hydrus/introduction.html and making use of all the features.

https://www.youtube.com/watch?v=W-59sWasI98 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v504/Hydrus.Network.504.-.Linux.-.Executable.tar.gz I had a great week focusing on improving sidecar import/export. Importing and exporting tags and URLs with files is now easy. Qt5 As a reminder, I am no longer putting out Qt5 builds. If you are on Windows 7 (and I heard one instance of 8.1), or a similarly old OS, you probably cannot run the hydrus builds any more. You can still update, but you need to run from source from now on. The good news is that this is now much easier to do on Windows--please check the help here: https://hydrusnetwork.github.io/hydrus/running_from_source.html Windows source users should also get a nice hydrus taskbar icon this week. sidecars There's a semi-advanced feature in hydrus where you can tell the client to look for (on file import) or create (on file export) additional files (sidecars) along with your media. Until now, this has been fixed to 'filename.jpg.txt' files that would include tags for the file separated by newlines. This system expands radically this week. What was static before is now flexible. You can set up multiple workflows for multiple sidecars in one task, grab or send URLs as well as tags, parse or output formatted JSON instead of .txt, and transform or filter the text at every stage. This is available on manual imports and exports, and import folders and export folders. For manual imports, it becomes a new 'sidecars' page tab on the 'add tags before import' window, and for all the others it is now a button on the UI. If you are interested in this tech, please check it out and let me know what is confusing. It is quite powerful now, and users who are familiar with the concepts involved (and the parsing system) should find it obvious, but I think I'll need to add some help text and a full help guide for users new to it. All existing 'neighbouring .txt' settings on import folders, export folders, and the manual file export window should be migrated to the new system when you update. Let me know if there are any problems! There are many potential next steps here. I'd like a 'favourites' system first, so it is possible to save/load/share workflows, and then we can think about more content types (ratings, notes) and .xml parsing/export. full list - Qt5: - as a reminder, I am no longer supporting Qt5 with the official builds. if you are on Windows 7 (and I have heard at least one version of Win 8.1), or a similarly old OS, you likely cannot run the official builds now. if this is you, please check the 'running from source' guide in the help, which will allow you to keep updating the program. this process is now easy in Windows and should be similarly easy on other platforms soon - . - misc: - if you run from source in windows, the program _should_ now have its own taskbar group and use the correct hydrus icon. if you try and pin it to taskbar, it will revert to the 'python' icon, but you can give a shortcut to a batch file an icon and pin that to start - unfortunately, I have to remove the 'deviant art tag search' downloader this week. they killed the old API we were using, and what remaining open date-paginated search results the site offers is obfuscated and tokenised (no permanent links), more than I could quickly unravel. other downloader creators are welcome to give it a go. if you have a subscription for a da tag search, it will likely complain on its next run. please pause it and try to capture the best artists from that search (until DA kill their free artist api, then who knows what will happen). the oauth/phone app menace marches on - focus on the thumbnail panel is now preserved whenever it swaps out for another (like when you refresh the search) - fixed an issue where cancelling service selection on database->c&r->repopulate truncated would create an empty modal message - fixed a stupid typo in the recently changed server petition counting auto-fixing code - . - importer/exporter sidecar expansion: - when you import or export files from/to disk, either manually or automatically, the option to pull or send tags to .txt files is now expanded: - - you can now import or export URLs - - you can now read or write .json files - - you can now import from or export to multiple sidecars, and have multiple separate pipelines - - you can now give sidecar files suffixes, for ".tags.txt" and similar - - you can now filter and transform all the strings in this pipeline using the powerful String Processor just like in the parsing system

[Expand Post]

- this affects manual imports, manual exports, import folders, and export folders. instead of smart .txt checkboxes, there's now a button leading to some nested dialogs to customise your 'routers' and, in manual imports, a new page tab in the 'add tags before import' window - this bones of this system was already working in the background when I introduced it earlier this year, but now all components are exposed - new export folders now start with the same default metadata migration as set in the last manual file export dialog - this system will expand in future. most important is to add a 'favourites' system so you can easily save/load your different setups. then adding more content types (e.g. ratings) and .xml. I'd also like to add purely internal file-to-itself datatype transformation (e.g. pulling url:(url) tags and converting them to actual known urls, and vice versa)

- importer/exporter sidecar expansion (boring stuff): - split the importer/exporter objects into separate importers and exporters. existing router objects will update and split their internal objects safely - all objects in this system can now describe themselves - all import/export nodes now produce appropriate example texts for string processing and parsing UI test panels - Filename Tagging Options objects no longer track neighbouring .txt file importing, and their UI removes it too. Import Folders will suck their old data on update and convert to metadata routers - wrote a json sidecar importer that takes a parsing formula - wrote a json sidecar exporter that takes a list of dictionary names to export to. it will edit an existing file - wrote some ui panels to edit single file metadata migration routers - wrote some ui panels to edit single file metadata migration importers - wrote some ui panels to edit single file metadata migration exporters - updated edit export folder panel to use the new UI. it was already using a full static version of the system behind the scenes; now this is exposed and editable - updated the manual file export panel to use the new UI. it was using a half version of the system before--now the default options are updated to the new router object and you can create multiple exports - updated import folders to use the new UI. the filename tagging options no longer handles .txt, it is now on a separate button on the import folder - updated manual file imports to use the new UI. the 'add tags before import' window now has a 'sidecars' page tab, which lets you edit metadata routers. it updates a path preview list live with what it expects to parse - a full suite of new unit tests now checks the router, the four import nodes, and the four export nodes thoroughly - renamed ClientExportingMetadata to ClientMetadataMigration and moved to the metadata module. refactored the importers, exporters, and shared methods to their own files in the same module - created a gui.metadata module for the new router and metadata import/export widgets and panels - created a gui.exporting module for the existing export folder and manual export gui code - reworked some of the core importer/exporter objects and inheritance in clientmetadatamigration - updated the HDDImport object and creation pipeline to handle metadata routers (as piped from the new sidecars tab) - when the hdd import or import folder is set to delete original files, now all defined sidecars are deleted along with the media file - cleaned up a bunch of related metadata importer/exporter code - cleaned import folder code - cleaned hdd importer code next week I focused on sidecars this week almost exclusively, so I want to catch up on small work that piled up in the meantime. It would be nice to have some more Client API commands too, but we'll see. I can feel myself going down with something, so it might be a slim week.

>>18492 >MPV trouble I tried the method of copying my system "libcdio.so.19.0.0", "libgmodule-2.0.so.0.6600.8", and "libmpv.so.1.109.0" into the main Hydrus directory and deleting the ones shipped with Hydrus but it didn't work either. See pic 1. So, I restored the v504 to its original state by replacing my system files with the Hydrus original ones. Then I followed the advise found in the below link: >I think this is the basic issue we are dealing with here: https://github.com/jaseg/python-mpv/issues/212 although his patch clearly didn't give you a nicer error message. In any case, I think the answer is to update your libmpv1 if possible. Any specific version info you learn would help, and if you can give the exact apt line that gives a working version, that'd be great and I can roll it into the install instructions. I installed the required packages as indicated in the above link but MPV still won't load giving me the exact same error shown in Pic 1. Also see pics 2 and 3. At this point my options left are: 1- To change Distro ---> No, no. Not again 2- To run from source ---> A leap way too far for my knowledge 3- To try Wine ---> I will kill myself first 4- Suck it up and wait. I sticking to option 4 for the time being. Thanks OP, I understand what is the "Dependency Hell" in Linux and I won't push the issue further.

>>18562 >Yes, please put your hydrus stuff in a venv. Ended up fixing Stable Diffusion's venv. This was meant to happen in webui.sh, but because it had a stale venv when I failed to get the docker guide to work it ended up falling back to ~/.local. Works after deleting the broken venv folder.

So to switch from an extracted version to the source version I just need to copy over the db folder? Both files and thumbnails are separate.

Drag and dropping images to the tab bar to make a new tab with them does pic related. Right click -> open in a new page works as expected. Restarting Hydrus... apparently fucks up all the other tabs. Shit. Opening a new tab doesn't open a properly working tab now. That was with 503 and W7, but looks like it's the same with 504.

>>18598 I think reinstalling fixed it. After that I was able to drag the image area back to existance. I don't think it was there before that.

>>18589 Hi this is the anon you're responding to. I don't know how to run scripts, but I did manage to connect hydrus and hydrusnao, but it is extremely slow and I reached the 100 query limit. Is it really the most effective form of 'wizardry' for tagging tagless files? I was looking at the deepdanbooru maching-learning thing but I ran into a snap with the complexity and issues with pip installation. >>18590 Here's my approach: 1) Get artist work from booru for free perfect tags 2) Get artist work from 2nd source, usually pawoo, baraag, pixiv, and/or exhentai compilation. 3) Sort through duplicates, manually add tags on taggless 2nd source. Tags normally limited to artist, series, and general theme, so I don't spend a week with tags. I am also a control freak, but I feel like there should be an easier way about this. I have only done the above method once, with an artist with work in gelbooru, and a whopping 700+ of work outside of it, that so far I have been tagging alone. If paying saucenao is the way to go I'll go ahead and do that, but if there is an alternative I would like to explore those as well.

>>18600 >it is extremely slow and I reached the 100 query limit how slow? saucenao has roughly a 2 second interval between API callouts, but otherwise can be run indefinitely at that speed, and that's plenty fast in my experience. it shouldn't take long to go through an entire database with a paid account. I have my own saucenao tool that runs ~100 queries per night, starting with unsauced images and then images that haven't been sauced in the longest time, as an example of what you can do with a free account. also, if your boorus have any internal IQDB tools, you can hook into those and they probably don't have any daily limits. a couple tools here: https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/wiki/0-Hydrus-Apps-and-Scripts https://github.com/GoAwayNow/HydusTagGrabber https://github.com/nostrenz/hatate-iqdb-tagger https://gist.github.com/corposim/b7ccb6a2c8814032ddd65db91b371dc2

>>18600 as for your workflow, you shouldn't be manually tagging images that are direct duplicates, not sure if this is what you're implying. duplicates will transfer over their tags and URLs automatically by default I believe, or they can be set to do so under filtering > "edit default duplicate metadata merge options" I don't remember how the normal saucenao tools work, but for my integration I grab every single related copy (with one API call), then merge all the duplicate copies in order to consolidate tags into one place. I use a tag service for every tag source so they don't get tangled up in each other. Any copies that aren't related just get deleted (anything still left in inbox afterwards)

>>18602 also I haven't built it yet but I'm planning on making a "minimum viable tags" project where it grabs files from hydrus that don't have a minimum amount of tags based on a certain level, e.g. Level 1 - does it have a creator Level 2 - does it have genders tagged Level 3 - does it have sfw/nsfw tagged etc. and then prompt for tags before sending the new tags back to hydrus. it would grab all level 1 files first, then all level 2, etc. that way I can have the most important tags on more files instead of fully tagging files one at a time. I feel it's also probably faster to tag the same things on multiple files instead of multiple tags on one file. If anyone has already done something like this or has any thoughts let me know - it sounds like a good idea to me

when adding a new query to a subscription, I noticed that the default tag import options for the query is set to "set custom tag import options just for this importer", but the tooltip when I hover over it says that normally hydrus uses the defaults here. That's confusing. I figured that when I add a new query to a subscription, it would not have "set custom tag import options" by default, and for the subscriptions themselves, that is how it works. The file and tag import options start out being on "use defaults", but that's how it is for the individual querys' tag import options, and I have to manually set it back to "use default tag import option" each time, which is annoying and error prone if I forget. Is this behavior supposed to be how it works?

I noticed when editing one of my parsers that it looks like there's a way to make parsers for notes now. I don't remember seeing that before. None of the hydrus downloaders that I use have any though. Is it "ready" to be used and it just hasn't been added to any of the ones I use yet, or is it still experimental?

>>18601 I'm on free account for saucenao and hatate recommends no more than 30 seconds, so for the 800 tagless images it would take me 8 days. You're saying with a paid saucenao I can do 2 second intervals between queries as opposed to 30 seconds? What I'm doing now is simply tagging an entire tagless import into hydrus with just the artist name, then I use their name as tag as an anchor from hydrus to the saucenao tagger application.

>>18606 I didn't realize free accounts actually can't query as fast in burst. Free tier: Burst: 4 queries per 30 seconds (7.5s/request) Daily: 100 Paid tier ($5): Burst: 17 queries per 30 seconds (1.76s/request) Daily: 10000 if you only have 800 images I would recommend you just use the free tier over 8 days, or make another alt account (on a different IP) and cut it to 4 days. I currently use two free accounts in tandem, but they'll prevent you from using the API if you share IPs between the two api keys (even with a VPN IP, for some reason)

>>18585 >Since video deduplicating is not really a thing, I often end up trying to manually trump video files over each other by manually copying tags and urls etc. video files won't be automatically detected as duplicates, but you can just set them as potential duplicates. then you can deal with them in the duplicate filter as normal. >if you don't navigate the context menu perfectly it will reset 4 levels of mouseover. you can set shortcuts for file relationships actions. i have f12 as "set these files as potential duplicates", which is how i send video duplicates to the duplicate filter. >the process of getting them on the same tab is very tedious it is? are you manually adding a tag to both and then searching it or something? i have a shortcut for "open files in a new page", so i juts select one of the video files on one page, press the shortcut to put it in a page on its own, then drag and drop the other video file into the new page.

Is there any way to parse EXIF data in Hydrus? I was digging through my unsorted files and found that some site I found at some point left all the tags in the keywords. If not, I guess I can make a script to parse them and put them into a text file alongside the image, but fuck that seems like something I don't want to do.

when files are set to be deleted in the duplicate filter and you have the "don't delete archived files" setting enabled, it doesn't mention anything while you're in the duplicate filter, but once you leave you get spammed with a bunch of "this file couldn't be deleted" all at once. it would be better if instead it tells you when you press space (or whatever shortcut you have set for "this is better and delete the other) that the file is archived, and it gives you the option to confirm that you want it deleted anyway, since it's a worse duplicate.

I can't start hydrus after updating vtk to 9.1.0-21 earlier today OS: arch linux hydrus version: 504-1 v504, 2022/10/30 00:05:38: hydrus client started v504, 2022/10/30 00:05:39: hydrus client failed v504, 2022/10/30 00:05:39: Traceback (most recent call last): File "/opt/hydrus/hydrus/hydrus_client.py", line 230, in boot from hydrus.client import ClientController File "/opt/hydrus/hydrus/client/ClientController.py", line 27, in <module> from hydrus.client import ClientCaches File "/opt/hydrus/hydrus/client/ClientCaches.py", line 15, in <module> from hydrus.client import ClientFiles File "/opt/hydrus/hydrus/client/ClientFiles.py", line 21, in <module> from hydrus.client import ClientImageHandling File "/opt/hydrus/hydrus/client/ClientImageHandling.py", line 6, in <module> import cv2 ImportError: /usr/lib/libvtkInteractionStyle.so.1: file too short v504, 2022/10/30 00:05:39: hydrus client shut down

Hello Hydrusdev, Currently on v502 on Fedora Linux (running from executable, not source), I'm having trouble updating to either v503 and v504. Here's the error I'm getting trying to start: ``` v504, 2022/10/30 00:59:40: hydrus client started v504, 2022/10/30 00:59:41: hydrus client failed v504, 2022/10/30 00:59:41: Traceback (most recent call last): File "hydrus/hydrus_client.py", line 230, in boot from hydrus.client import ClientController File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/ClientController.py", line 27, in <module> from hydrus.client import ClientCaches File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/ClientCaches.py", line 17, in <module> from hydrus.client import ClientParsing File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/ClientParsing.py", line 21, in <module> from hydrus.client.networking import ClientNetworkingJobs File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "hydrus/client/networking/ClientNetworkingJobs.py", line 6, in <module> import requests File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "requests/init.py", line 45, in <module> File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "requests/exceptions.py", line 9, in <module> File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "requests/compat.py", line 40, in <module> ImportError: cannot import name 'JSONDecodeError' from 'simplejson' (unknown location) v504, 2022/10/30 00:59:41: hydrus client shut down ``` The above is for 504, but 503 is the exact same. I tried to `pip install -U simplejson` just in case, but it does not fix my issue unfortunately. I downgraded back to 502 without issues, but I'd love to keep up to date. Thanks for your help as always!

Hello devanon, could you please check the version libmpv version we get on the Linux binary release? It seems like we need at least 0.33 to work with the current (latest?) python-mpv. Complete error I'm getting: """ File "hydrus/client/gui/canvas/ClientGUIMPV.py", line 25, in <module> import mpv File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "mpv.py", line 579, in <module> File "mpv.py", line 502, in _handle_func File "ctypes/init.py", line 386, in getattr File "ctypes/init.py", line 391, in getitem AttributeError: hydrus/libmpv.so.1: undefined symbol: mpv_event_to_node """ Linking https://github.com/jaseg/python-mpv/pull/224 where this error has been made more obvious. Thank you! Trying to manually update with the one I get from my system is making everything even angrier: """ File "PyInstaller/loader/pyimod03_ctypes.py", line 53, in init File "ctypes/init.py", line 373, in init OSError: hydrus/libmpv.so.1: undefined symbol: wl_proxy_marshal_flags The above exception was the direct cause of the following exception: File "PyInstaller/loader/pyimod03_ctypes.py", line 53, in init File "ctypes/init.py", line 373, in init OSError: hydrus/libmpv.so.1: undefined symbol: wl_proxy_marshal_flags The above exception was the direct cause of the following exception: Traceback (most recent call last): File "hydrus/client/gui/canvas/ClientGUIMPV.py", line 25, in <module> import mpv File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "mpv.py", line 54, in <module> File "PyInstaller/loader/pyimod03_ctypes.py", line 55, in init pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll 'hydrus/libmpv.so.1'. Most likely this dynlib/dll was not found when the application was frozen. """

>>18613 So I checked the build logs to make sure; we're getting 0.32.0. Sorry, as this is probably annoying, but could you please consider upgrading the Ubuntu builder from 20.04 to 22.04? They're both LTS but 20.04 is very outdated; https://packages.ubuntu.com/focal/libmpv1 (that's 20.04) says that the very latest is 0.32.0-1ubuntu1 (which is incompatible with the current python-mpv), but https://packages.ubuntu.com/jammy/libmpv1 (that's 22.04) has 0.34.1-1ubuntu3, which should work.

>>18593 >As a reminder, I am no longer putting out Qt5 builds. JFYI it seems you can't run in easy way Qt6 version on Ubuntu 18.04, since its libc version is 2.27. [13467] Error loading Python lib '/opt/hydrus-network/libpython3.8.so.1.0': dlopen: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /opt/hydrus-network/libpython3.8.so.1.0)

>>18566 Sounds good. Some of the neater metadata would probably be difficult to pull with ffmpeg, but if this happens, I can at least add optional support and we can start playing around with this tech. >>18567 Maybe. That second one looks nicer. Anything on that pypi site you can get with 'pip', which is the super easy way to get any python 'extension'. It also has partial Pillow support, which is great Pillow (aka PIL) is one of our image libraries. I wonder if this will get integrated into Pillow officially. I'll keep an eye on this, thanks. >>18570 Sorry for the obscurity. Subscriptions are a slightly odd and resource-intensive object, and to keep load and save snappy they regularly cull their private URL store down to 250 URLs. The database itself still knows about all your URL-file mappings, so nothing is different with the files themselves when imported via sub or downloader page, but as long as the immediate file log cache inside the subscription has more than 250 URLs, it will clear out anything older than twice its death velocity. It mostly uses this file log to calculate its next check time (and so the user can drill in to fix recent problems), so it isn't meant as a permanent history. I made this change for technical reasons. Subs used to store everything, and some users' large subs ended up getting so big they too thirty seconds or more to load, even just for checks. I've since rewritten the whole object structure and load order and header caching to make subs fast as they are today, but they still regularly purge themselves of older import objects. There is no permanent subscription history in the client yet. I'd like to add that one day, and have 'system:was downloaded with blah sub' and 'this file was downloaded in blah sub' on thumbnail right-clicks, but unfortunately for now these connections are forgotten, and 'known url' is the best we have. >>18572 Are you sure you have a newer version of the client? I guess I added that tech around six months ago, so ~>475? Every modern client should be able to add them from first boot. You were looking in the right place, ''services->manage services->add button->local file domain'. Second option in the add button menu for me. Don't get confused between 'review services' (look at) and 'manage services' (edit).

>>18574 I am glad you like it! I like to emphasise that I am not by any means super special, and my code is very unprofessional. I'm a bit clever, and I have some education in programming, and I am interested in every topic. My real advantage in hydrus terms is I'm a weird Anon who wants to create stuff that works for our community, and I'm stubborn as anything so I haven't given up yet. We're coming up on eleven years this December. When I first started, I didn't know what I was doing, but over the years I just looked up examples, cobbled something of my own together, and kept working at it. I learned how UI stuff is 'supposed' to work, how databases and MVC models are 'supposed' to work, and then everything else the program does. When I was in college, Agile development was the big new thing, so I'm very much someone who tries to rush out a 1.0 that I can then iterate on. I'll say that although I am a better programmer now, I'm still shit, and there's a ton of legacy bad code still in hydrus. I'm slowly working through it. It hasn't just been the technical side, though. Doing hydrus I also learned about how to talk to people better, learn to take bug reports and discuss new features without stressing out, and most importantly how to avoid burnout on an endless marathon. I learned before I started that I just can't work in teams (I'm a drama bomb), so that was already set. I talk to plenty of younger programming students as hydev, so if that happens to be you, and you want to try something interesting but think you need to go somewhere to learn it, I strongly recommend you just download the package and start coding something up. You will fail over and over and over, and if you release it you will sometimes disappoint people, and it will suck at times. There are many problems and difficulties, but if you take it with the right attitude, you can learn from it and keep moving forward. It sounds cliche to say 'learn from your failures', but it is the only way anything worthwhile gets done. Forgive me on the license front, with the WTFPL. Any time anyone wants to explain or tell me one license is good or another is bad, I'm afraid my eyes glaze over. My brain just does not care about the subject. I often think of programming as having a wizard/sorcerer distinction, and license discussions are so wizardpilled that I just can't engage without getting bored and ultimately saying something rude. WTFPL is my 'idgaf' compromise. >>18575 Thank you for the reminder. I'm writing it down in my todo now, and I have a couple of ideas actually. We've been getting more custom import/export tech in the client recently, and in the Client API too, so I might have the tools I need to do this more easily now.

>>18576 >>18577 Damn. Yeah, I think you are right, this thing is working off URL calculations, not hash, so you'd have to set the 'check if URL recognised' option. I am not totally sure what the answer would be here, though. What logical change do you think would allow you to trigger a page fetch when you want, and not when you don't? What's the critical difference that I can recognise? Reading your posts again, this idea of file url causing it to skip a page fetch is, I think, untrue. It normally only knows a file url after a page fetch, at which point it got the tags and stuff with the page fetch. If it has tags and has somehow matched the file to something 'already in db', it should be able to assign the tags ok. Maybe the kemono parser works differently, and I'm just not thinking it through right. >but I still want hydrus to be able to skip files when it's part of the same post, since always fetching would be very inefficient. Just to check, are you aware that this 'check even if xxx recognised...' checkbox does not fetch the file again? It'll only fetch the html/json again. So while it is wasteful, it isn't incredibly so. I think setting a custom 'tag import options' for your subscription here would be ok, or if you are very concerned, then only for the artists who have this posting habit. There is a 'download file even if you think you recognise it', but it is under 'file import options', and I think it is hidden under advanced mode too. >>18578 Sure, I will add it to the list. >>18583 I'm going to be rewriting the front-facing text you see when you land on this page https://github.com/hydrusnetwork/hydrus over the next couple of weeks. Can you tell me what I could add there that would have directed you better to the release list or whatever you needed? I know it doesn't direct people well enough to the help and getting started guide https://hydrusnetwork.github.io/hydrus/introduction.html , which is really the on-ramp to get going.

>>18585 Thanks, this is something to think about. Now I haven't messed around with videos and the duplicates system as you and >>18608 have done, so you are more expert than me, but I think the answer to what you want here may be to map that 'set as better' command to a shortcut. I hate that it is so deeply embedded in the menu--I added that as an advanced routine hidden from normal users, and then it proved so useful to power users that it got normalised, and its 'set as xxx' brothers added too--however, in shortcuts, under the 'media' shortcut set, you can set a shortcut for the simple command 'file relationships: set the focused file as king duplicate'. That does 'set this file as better', and should apply the default merge options just like the dupe filter does. It doesn't do the file delete, because the dupe filter does that separately. I feel like this does the copy/paste you want, except in one step, so you still need the files next to each other. BTW, did you know you can drag and drop files between tabs? If you have been mickey-mousing with file searches on temporary tabs or something, maybe just having a staging tab you drag two files to, select them, click on the better, and then hit the shortcut on, would work out for you. Note that if you always set to delete the worse of the two files, you can batch your deletes here. Just do a search for 'system:file relationships: is not the best quality file of its duplicate group', and you'll get all the non-kings, and you can delete them in one go. All that said, I'll keep thinking about what you've said. We are getting more import/export options recently, and it would be neat to have a kind of mega-pull copy and a mega-push paste. I also want to completely rework the duplicate merge options. I have how much of a pain in the ass is it to edit them, how technical and non-human they feel. >>18595 I have been talking with another Linux user about this exact problem, including the 0.34.1 mpv version. I now know the problem is definitely the new python-mpv mixed with multiple Linuxes not shipping a new mpv in their apt. The solution is going to be 2, but please do not worry. The recent 'it is now easy to run from source in Windows' scripts I rolled out have worked well, and I have figured out a requirements.txt just for your specific problem. If a Linux user doesn't pre-empt me, I am going to translate the Windows .bat files to Linux .sh, update my 'running from source: easy' guide to be multi-platform, and direct you to it. No clever technical knowledge required, just enough to open an archive and maybe open a terminal window. >>18597 Yep, just the db folder. If you inspect the two installs, you'll see several shared folders, and that structure is basically the same. The 'db' folder is the same in both, so just move it across from one to the other. Good general advice: Make sure you make a backup before you do anything, and then if something does go wrong, you always have something safe you can roll back to.

>>18598 >>18599 Thanks for this report. I had this from another user too, and I don't know what it is. I don't get it on my client, and I think he got it rarely, so maybe it happens if you drag and drop at just the wrong moment. I'll ask you what I asked him, which is if this happens again, please play with the commands under ''pages->management and preview panels. I am not sure if the thumbnail page is being removed here, or if it is shriking to 0 pixels wide. Use the commands in that menu to see if you can show/hide the other panels, and then restore everything back to normal. Does the thumbnail panel snap back into view, or do you just get void? Also, if you discover better how to induce this problem, that'd be great. I just don't know how to reproduce it yet. Sorry for the trouble! >>18604 Thank you for this report! This is bad UI on my part. The 'tag import options' attached to a subscription query is actually a stub TIO and isn't plugged into that system. All it does is 'add additional tags' if you want to add something just for that query. When I updated the import options UI recently to be more default friendly, the TIO on sub queries must have inherited the changes, and now it is confusing. Thank you, I will rewrite this and make sure it doesn't let you select to use the default. When it comes to subs, it is the sub itself that has the TIO that starts default and is plugged into that system. >>18605 Yeah, it was a couple months ago. I made an example note parser for Hentai Foundry if you want to check it out. I said at the time I would keep working on note parsers for the other default sites, but that got buried under an avalanche of work and some personal IRL stuff. It is all ready though. One of the downloader makers wants me to update it so that notes have a kind of enumerated label, something like 'artist description' or 'booru translation', to make it easier for users to choose what they want in the new Note Import Options. At the moment, my system works on note names, which is more awkward for normal tasks that I had hoped. So, expect there to be some future changes too. If you play with it, let me know what you think! >>18609 Yeah, it is shit at the moment, but I have basic support. Check the cog icon in the media viewer's top hover. I am going to work on this in the coming weeks, I hope, teaching the database which files have exif so it remembers, and add 'system:has exif', and show it better. Also looking to add some png metadata, so those new ML pngs people are making (which the popular webui embeds with the creation prompt) can reveal their stuff.

>>18610 Thanks, I'll check this out. If I remember right, that spam at the end was a technical compromise until I fixed something else. >>18611 Damn. Are you on the AUR package? I don't know what vtk even is, so I cannot be an expert here. I don't make the AUR package, so if you are on that, this may be their issue to handle. If you are running from source, can you talk about how you set it up? Did you use my running from source guide, and are you using a venv? cv2 is OpenCV btw. I am assuming you are not on the Linux build I put out, because the traceback looks like source, but let me know if I am mistaken. >>18612 Damn! I've just looked at the code, it looks like 'requests' has a bitrot bug here. You have simplejson, but that library it wants to import I guess got renamed. Ok, I reordered the libraries and fixed some versions in v503's requirements.txt (in an attempt to be more stable!), so I bet that is what triggered this. I will investigate this and see what I can do. Sorry for the trouble, what a pain. >>18615 Thank you, I will update the help about this!

>>18613 >>18614 I've read your posts a couple times and researched it more, and I thought I was going to say '0.34.1 doesn't work either, as in >>18595 ', but I think I am wrong. EDIT: I've now thought about it more and checked things, and I think you are correct. I was mistaken about what pyinstaller was including in the build and the way it was all being loaded. I think your idea to update to 22.04 is the correct next step to try. I'm not certain this is the solution, but I think it is. Thank you very much for doing this research. LINUX MPV TEST I have made a test build here: https://github.com/hydrusnetwork/hydrus/releases/tag/v504-test It is exactly the same build as 504 above, but on 22.04. It has a new 0.34.1 libmpv.so (stub?). Anyone with Linux libmpv problems, please download this and try it! Make a backup before you try. Make sure you have the latest version of libmpv1 for your Linux flavour (most probably with apt-get, and most probably version 0.34.1). Questions: Does the client boot for you? If not, what's your OS flavour/version? Is mpv ok now? What does it say in help->about? If you have mpv, and you know how to do this, please uninstall libmpv1 with apt and then try again. Does it work? Any changes/problems/issues?

>>18622 >Does the client boot for you? If not, what's your OS flavour/version? It is not booting. My system is Debian GNU/Linux 11. Konsole output complains: "libm.so.6: version `GLIBC_2.35' not found" See pics for Konsole and packages installed.

>>18621 >Damn. Are you on the AUR package? I don't know what vtk even is I'm sorry it was an issue on my end, I reinstalled Hydrus and its dependencies and now it works. I'm not too sure what went wrong or how I fixed it but thanks for your time anyway. I'm using the AUR package yes. It works fine aside from the mkdoc and qtpy issue.

>>18620 I'm not 10389/10390, but I'm getting the same issue. 503, modified AUR package. Dragging any thumbnails from the grid to the tab bar to create a new page causes it to spawn without the thumbnail grid without fail. 'pages->management and preview panels->restore all pages' sash positions to saved value' caused the thumbnail grid to reappear. During testing I had accidentally clicked the 'save current page's sash positions now' but clicking the restore option immediately after moved it slightly and created a handle to resize the thumbnail grid. Interestingly my about page says I'm on qt 5.15.6, I thought used qt6, but I guess not. I'm gonna copy this install somewhere else and see if I can get it to use qt6, since that may be the problem.

>>18625 Update: The issue does not seem to occur with qt6. It's worth noting, for Arch users, that qtpy does not seem to like the version of Pyside6 on the main repos, which I think is how I ended up with qt5. Using the python-qtpy-git AUR package seems to have resolved the issue.

If would be useful if there was an option in the duplicate filter to only show dupes if the two images are of different dimensions. That way you could quickly go through those while ignoring alternates, which is the majority of my dupes. Kind of like the pixel dupes option.

>>18622 I'm currently using v502 and I haven't made a backup yet so I'm not using this on my actual installation, but I downloaded it just to start it up and see if mpv works, but actually it won't boot at all. in the terminal it just says Cannot mix incompatible Qt library (6.3.1) with this library (6.3.2) and then it freezes and does nothing. I'm on Fedora 36 KDE with the kde-copr

>>18620 >I'll ask you what I asked him, which is if this happens again, please play with the commands under ''pages->management and preview panels. I am not sure if the thumbnail page is being removed here, or if it is shriking to 0 pixels wide. Use the commands in that menu to see if you can show/hide the other panels, and then restore everything back to normal. Does the thumbnail panel snap back into view, or do you just get void? It happens literally every time for me, on a completely fresh install too (a new db and everything). -Loading the correct sash settings recovers it. -If I save the broken view, show/hide now flips between the the thumbnails and the sidebar, but the sidebar does then have the drag available. -If I break it normally with the drag & drop image, show/hide first recovers the correct view, and then hides the sidebar the next time. -When it breaks, the sidebar drag is not available until I either load, or show/hide

I'm still on 499 of the AUR version. Should I just keep staying on this version for a while? AUR is trying to pull in python2 and it seems like 80% of the posts in this thread are about problems with dependencies.

>>18630 You can update just fine it if you follow the instructions on the AUR, you just have to modify the PKGBUILD to remove the doc. Then there's various fixes for the qtpy issue, personally I just edited line 90 in /usr/lib/python3.10/site-packages/qtpy/QtCore.py because everything else seemed like a hassle.

So I had to reinstall windows the other day, and apparently the both the manual copy I made before the reinstall and the latest backup of that drive resulted in errors about the database image being malformed. Fortunately an older backup I have is working and the actual images are stored on a different drive, but now hydrus cant see anything I have imported since then and gives file missing exceptions for everything that I had deleted. Is there a way to make hydrus look in the directory that all its media files are stored and have it import everything it doesn't expect to be there and then forget about all the files it expected and couldn't find?

>>18627 that's actually a pretty cool idea. seconding that idea although I'd also like an option to inverse it (only show dupes of the same dimensions)

>>18627 +1 to this request. This would help a ton with duplicate processing.

I am trying to run Hydrus v504 from source, but it's not working with Python 3.10, as requirements.txt has "opencv-python-headless==4.5.3.56", which depends on numpy 1.19, which is incompatible with Python 3.10. Would it be possible to upgrade opencv to at least 4.5.4? I have no idea what this would entail, but I'm guessing that you pinnng the version so precisely must have a reason, so I'm scared of changing it myself. Relevant opencv-python issue: https://github.com/opencv/opencv-python/issues/525 Thank you!

>>18622 >>18623 >>18628 Ok, this is what I was afraid of, and now why I hate moving to the newest version of anything for a build for everyone. Feels like 22.04 is going to have some compatibility problems for older flavours that I just don't know enough about to easily fix. I expect to revisit it in six months or a year. Therefore, I think I need to hustle on making it easier to run from source on Linux, and then I can just roll out different requirements.txts for people in different situations. Here is another test build: LINUX MPV TEST 2 https://github.com/hydrusnetwork/hydrus/releases/tag/v504-test-2 This this is again the same as the v504 build, but it is rolled back to versions that I know worked at one point. If there's no other bitrot, it should load the old version of mpv ok and be compatible with more flavours/versions of Linux, fingers crossed. Like the one above, please make a backup before you try to update any real database. You can also just try to extract to your desktop and see if it boots and plays a video with audio. I want the build to be the easy mode, and if someone wants whizz-bang new features, they can move to running from source specific for their environment. If this test build works out of the box and plays noise, that'll be the default settings going forth. >>18635 Thanks. You are encountering a similar issue here, too, as above. The newer versions of things seem to be rapidly breaking older things. We're stuck on that version of OpenCV for the main build as later versions won't freeze the the exe build with PyInstaller. I'll have to think about nicer solutions to this. Maybe even more requirements.txts, like 3.8/9, 3.10, argh, but that's probably the right answer. Don't be too afraid of trying to fix this yourself. You can't really break anything with a venv, since you can just delete the folder and start over. If you know how to open a command terminal in your install directory, then try this: On Windows: venv\Scripts\activate.bat On Linux: . venv/bin/activate pip install --upgrade opencv-python-headless I'll update the requirements.txts to have a nicer solution.

>>18636 >>18635 I just did some tests, and I don't think updating opencv on its own will cause numpy to update. Add this too, if you try a manual update: pip install --upgrade numpy You can also do this for specific versions: pip install numpy==1.20.3

>>18636 Fedora 36 here. Tried it in a new test install again just to check. It booted up this time, but going into about to check if mpv works returned what appears to be the same error as before. v504, 2022/10/31 18:32:27: If this information helps, MPV failed to import because: v504, 2022/10/31 18:32:27: Traceback (most recent call last): File "PyInstaller/loader/pyimod03_ctypes.py", line 53, in __init__ File "ctypes/__init__.py", line 373, in __init__ OSError: /lib64/libgio-2.0.so.0: undefined symbol: g_module_open_full The above exception was the direct cause of the following exception: Traceback (most recent call last): File "hydrus/client/gui/canvas/ClientGUIMPV.py", line 25, in <module> import mpv File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "mpv.py", line 53, in <module> File "PyInstaller/loader/pyimod03_ctypes.py", line 55, in __init__ pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll '~/Downloads/Hydrus Network/libmpv.so.1'. Most likely this dynlib/dll was not found when the application was frozen. I was going to also try the hydrus flatpak on flathub out of curiousity just to see if that one worked, but that one's still on v499 which was the last version that worked properly for me, so that wouldn't be helpful. I kinda wish I could just roll back to v499 at this point, or that I knew enough about programming to run it from source like you suggest.

your help might come in handy in this situation >>>/v/724311

>>18639 in line with this, how well does hydrus handle booru child posts?

>>18636 >LINUX MPV TEST 2 It works!!!!!!!!!!! I did a fresh install and it boots fine, also MPV is up and running. My system is Debian GNU/Linux 11. Outstanding fix. Thanks.

>>18641 Same anon on Debian GNU/Linux 11. I did the update from v499 to v504 LINUX MPV TEST 2 and also is working flawlessly. Thank you so much OP.

>>18640 basically not at all yet. Hydrus doesn't have a concept of parent or child posts, although imo those don't make sense anyway. They seem to be used for sequences of images and alternative versions of images, and parent/child posts don't suit either very well. Hydrus does have a concept of alternate versions, but it's not really exposed in the ui and is mostly a holding area for groups of files until the dev decides what he wants to do about them.

>>18640 >>18643 I think I misunderstood what you were asking. If you mean, "can hydrus downloaders download child posts in boorus" then that should be possible with a post parser that checks for them.

>>18638 >I kinda wish I could just roll back to v499 at this point You might if you still have the v499 backup.

How the fuck can I get rid of the files currently in the import queue? I messed up and didn't tag them and I just want to fucking STOP THE IMPORT, not just pause, but STOP it and completely redo it. Shit is driving me insane, this shouldn't be this hard to find.

I may be stupid but is anyone else having problems with Gelbooru 0.1.11 parsing? It mostly works but "Rating:" pulls in all the post info, what should be "Rating:Safe" is instead "rating:? tagme 4485statistics id: 24327 posted: 2022-10-31 20:21:35 by: anonymous size: 480x360 source: smuglo.li /a/ safe score: 0". I even reset the gelbooru parser and this happens. Considering booru.org still uses Gelbooru 0.1.11 I'm surprised I haven't seen this mentioned. The simple parser works, but only because it doesn't grab the rating namespace at all, which is one of the more important tags IMO.

For some reason mpv on linux stopped working properly for me, if i run it on X11 it plays sound but video is nowhere to be found, on Wayland mpv appears in a separate window. Also now for some reason trying to open a video takes atrociously long time, even just selecting it.

could the panel in the duplicate filter where you have the info and buttons be reduced in size vertically. It doesn't look like it needs to be as wide as it is, and I'm always thinking in the back of my mind that there could be some differences between the 2 images that would make them alternates, that are being hidden behind the panel. That actually happened to me a few times and I only caught it because I decided to pan right to check, so it's not just speculation.

>>18647 Yeah that parser is broken. I just fixed that issue myself when I noticed it, but there's also other things broken that I don't know how to fix (or I didn't. I got better at making parsers since then so maybe I do now) so I just try not to use it really.

>>18648 ok so smplayer which also uses mpv in a similar way (?) seems to be working on x11 but hydrus just gets blank screen. Probably related to kwin...

>>18646 Yeah hydrus's ui is pretty bad. You get used to it so It's not a problem forever, and it's slowly getting better, but it's not beginner friendly. The way I do it is by opening the file log, selecting all the entries I want to delete and then pressing delete.

I was ill this week and am short on work time. I will spend tomorrow doing some more normal work instead of the release. 505 should be on the 9th of November. Thanks everyone! ---- I did just commit changes for users who run from source. I updated the easy-setup scripts and wrote variants for Linux and macOS. My test options are limited, so if you are an advanced Linux or macOS user, please do test these new scripts and let me know where things break or are otherwise confusing. The updated help walks through it all: https://hydrusnetwork.github.io/hydrus/running_from_source.html

>>18647 here's a fixed one

I'd appreciate it if hydrus was able to also display the known urls of any same-quality duplicates as well in the top right when viewing a file. it'd very helpful to my workflow to be able to quickly see and access those.

https://www.cnet.com/tech/computing/chrome-banishes-photo-format-that-could-save-space-on-your-phone/ https://bugs.chromium.org/p/chromium/issues/detail?id=1178058#c84 fucking google politically killing JPEG-XL in chrome. send some hate mail on their bug tracker about this, no one else in the tech industry is going to let this go without a fight

>>18654 Based anon, thanks

I've noticed then when parsing source URLs for files, mega.nz files get their encryption key dropped. e.g. mega(.)nz/file/ns98rn#tnr8u3fhriRESTOFKEYETC turns into mega(.)nz/file/ns98rn At first I thought this was the parser misbehaving but the parser actually keeps the full URL intact all the way to the end. I'm assuming hydrus itself is responsible for dropping the key? Is it possible to stop this behavior?

Is there a way to get the relative times to be more precise. I wanna see the import times of some files but it just says "1 year ago". How many months? How many days?

>>18659 There's a "Prefer ISO time" option under gui -> misc in file -> options. I know it does not perfectly answer the question, but it's what I use because I had the same grief. >>18658 Under network -> downloader components -> manage url classes, find your Mega downloader and make sure that in its options you have the "keep fragment when normalising" option enabled. I'm not sure that this is your issue, but it would fit what you're describing.

>>18660 >find your Mega downloader and make sure that in its options you have the "keep fragment when normalising" option enabled Sorry I misspoke, these are mega links that are "additional URLs" to another service I use. E.g. I'd grab from a page and there would be a mega URL that would be related to the post, and that would be stored as an "additional URL" I turned on the "keep fragment" option for the original post but as expected it didn't have an effect for the related URLs

>>18660 >Prefer ISO time I should've mentioned that I already know about that. I like relative times a lot better, especially for the duplicate filter. it's easier for me to reason about, and the ui of hydrus seems to be designed with those in mind, rather than iso time.

>>18656 They're taking to apple route huh? I don't know why I expected better of them. This isn't gonna stop me personally though. Fuck chrome.

>>18656 One more reason to hate google. I already converted 1Tb of photos to JXL, fuck the google jew.

>>18664 I use Firefox on my Android. Works great!

>>18665 if your implication is "don't use google chrome and you aren't affected", then sadly that's not what's happening here. obviously not using google chrome is a good idea, but because most normal people use google chrome, it also has a stranglehold on driving the adoption of JXL. if chrome doesn't support it, other companies aren't going to convert their databases into JXL just so they can serve them to firefox users. there's quite a bit of pressure from big tech names in the bugtracker so I'm sure this decision will be reversed, but it's a great reminder of why everyone should stop using google chrome/brave/chromium-based browsers. by continuing to use them, you are allowing google to have a stranglehold on the internet.

Having problems with kemono downloading super slow with message "This domain has had several serious errors recently. Waiting a bit." But the log doesn't return any problems. I've been downloading a 14 page gallery for two days (collectively maybe 20 hours). I asked in the discord and was told it could be a temporary kemono thing and to try again later. There was another user experiencing the very same issue as me. Also started using e621 just today, and I'm getting this message after failure to import gallery page or URL: 401:hydrus.core.HydrusExceptions.MissingCredentialsException: 401: The server's error text was too long to display. The first part follows, while a larger chunk has been written to the log. The rest was what looked like the page's code setup. Any idea of what I can do? I've already tried being logged in, using login script, already sent cookies, tried without cookies and without login, removed all e621 items from hydrus and re-downloaded via importer. None of those things worked for me.

>>18638 I'm on Mint (`mpv -v` -> mpv 0.34.1) with the similar error. I've noticed something interesting, not sure if it's a personal case or if it affects you too. If I use any version (tried downgrading to v497 and upgrading to v504-mpv-test-2), it doesn't work. "MPV is not available", "open mp4 externally" But on a fresh install or if deleting /db yes of course you should back up first , any of those versions, it can open the videos and play fine, but without any sound. It still comes up with no mpv version in the About page and the same dynlib/dll error that you had. This puzzles me because I don't recall changing any options except for thumbnail size and window/qt theming.

>>18668 I think that's because if hydrus detects mpv it switches the settings to play videos with mpv, but by default it uses the native player, which can't play audio. it doesn't switch back automatically if mpv stops working though

>>18667 I think it's a Kemono thing. I'm using gallery-dl to download from Kemono, and it fairly frequently gives timeout errors. Gallery-dl just retries a few times and so far it's always worked.

Does that recent dangerous openssl bug that delayed fedora 37 affect hydrus? I see that it's running what appears to be an older version of openssl.

Since somewhere this week I'm getting only 404 errors from Twitter for any link, both with subscriptions and url imports (download -> urls). The log gives this message: AttributeError: type object 'CloudScraper' has no attribute 'is_Firewall_Blocked' I can load the urls in my browser just fine. Does anyone else experience this? My version is 450.

>>18672 I'm having the same problem on 502 (except there's no errors in my log). Opening an image in a new tab and downloading that link still works.

>Unsupported file type >.txt Really? I can't tag any of my copypastas without first putting them in a zip file?

>>18674 It's a big pain point for me too. I've been hoping for text file support for a while, but dev says that it's actually pretty difficult to add because of the way hydrus determines what filetype to treat a file as.

>>18673 I found someone with the same issue on this Github: https://github.com/y-young/nazurin/issues/57 The maintainers claim to have fixed it by switching endpoints: https://github.com/y-young/nazurin/commit/3976f6d3bd3c8af6dcf3f6f138cfb3b0145f4b44 I will try and see if I can do the same.

>>18673 >>18676 This worked for me, both downloads and subscriptions: 1. manage url classes > twitter syndication api tweet > path components > replace tweet with tweet-result 2. manage url classes > twitter tweet > optional api url converter > edit the regex, replace tweet with tweet-result The url class name may differ for you.

>create a new account because of the retarded PTR ban >only manually add tags to the PTR, don't even download from boorus during that period >stop using hydrus for like two weeks >try to upload a a couple tags today >banned for "uploading sankaku url tags" I'm fully convinced this is a power tripping janny banning people at random.

>>18623 >>18635 >>18638 >>18668 >>18669 Hey, sorry for the trouble here. I am launching the 'running from source' easy-setup scripts for Linux this week. (if you know git, they are already in the hydrus develop branch if you want to test). The help here will be updated https://hydrusnetwork.github.io/hydrus/running_from_source.html to walk you through it for Linux. I am sure there is more work to do on these scripts and the help, but my early testers are saying positive things. Please give it a go next week and let me know if you have trouble! Going forward, I am going to suggest all Linux users with version problems go to running from source, and if things really go well, I expect I will retire the actual Linux build in three to six months. You can update in like three seconds using git, so since Linux already has all these tools, and since venv is so easy to work with these days, I think it makes sense to move in this direction. As for problems like >>18624 on the AUR, maybe my new running from source tech can help. I don't regularly communicate with the AUR guy(s), but if my venv setup can help them figure out easy mkdocs setup or pin down better requirements restrictions, great. >>18625 >>18626 Thanks. I hadn't realised this was Qt5 only. I can reproduce this now and will have it fixed for v505. And yeah I don't know the details for AUR, but I do know the latest qtpy doesn't work with the latest PySide6. There's a qtpy monkeypatch that is now invalid with latest shiboken6 or something. >>18627 Thanks, this is an interesting thought!

>>18629 Sorry for the trouble. I figured out how to reproduce this and will have it fixed for v505. Let me know if you have any more trouble after that. >>18632 Yep. Two commands: database->file maintenance->clear orphan files This will remove files that are in your newer file structure, but "shouldn't be" according to the old database. (added since the backup) Move the files to a folder on your desktop, then import them manually. database->file maintenance->manage scheduled jobs then in 'add new work', click 'all media files', and then select 'if file is missing, remove record' This will remove the file records for files that aren't in your file structure old, but the older database thinks "should be". (deleted since the backup) This will take some time and will work in the background. You can speed it up on the same window. EDIT: wait, that 'if missing, remove record' job doesn't leave a deletion record, meaning you could reimport those files later. We don't want that, since you did legit delete them. I will add an alternate of this job for v505 that does leave a deletion record. Please wait and use that instead. Please check out 'install_dir/db/help my db is broke.txt' for info about malformed databases. You don't have to fix your newer databases, but check that document for background reading and think about whether you still have any hardware faults. Probably this was a one time problem that affected the main db, and then the backup when the backup was made, but if your hard drive is near death, that's the important problem to fix first. >>18635 I forgot to say, I worked with two other Python 3.10 users this past week, and now the setup script I will be launching next week, and the help, installs a newer version of opencv and fixes this problem for you. Follow the new steps and you'll be sorted!

>>18641 >>18642 This is great, thanks for letting me know. I'll say the roughly the same thing to you as the guys having trouble however, and if you have the time, please try out my easy-setup 'running from source in Linux' guide and scripts in next week's release. I think we may be able to get you on the newer mpv, if you have 0.34.1 installed to your system, and maybe relieve some other incompatibility niggles. If you give it a go, let me know if anything did not work or was confusing. >>18646 Click the down-arrow/triangle beside 'file log' text on that file log button. That has some advanced tools to mass-remove all 'successful' results etc... Otherwise, if you want to clear them all out, click the 'file log' button itself to open the import list, and ctrl+a->delete to remove them all. >>18647 >>18654 Thanks for this. I have to think they changed something in their markup, since I am confident thought this worked before. I will roll this update into the next version. >>18648 >>18651 Same as above >>18679 . Please try the 'running from source for Linux' update in v505 next week, let me know if that helps. >>18649 I'll write this down, thanks. I am planning a complete dynamic rework of the scoring system, hopefully that shortens this thing for users who want it. I can probably make it thinner too. >>18655 I am planning easy-navigation between set duplicates in my next big duplicates iteration (which will include the first proper 'alternates' setting, and similar navigations). I'm thinking similar to how a web booru will say 'here is this file's parent post'; I want easy quick 'perpendicular' navigation in the media viewer to alternates and dupes that doesn't interrupt the normal browsing chain.

>>18661 You need to make a URL class for mega and set that to keep the fragment. Pic related.

>>18682 That worked. weird but I'll take it.

>>18656 >>18663 >>18664 Sad, but I'm heartened by the comments in that issue. Best case scenario is this is middle management making a dumbass decision on local metrics that will be undone when they catch up to the external fallout, but I don't expect anything virtuous from google now, so maybe it is just them employing their clout to make avif win. It is the easiest decision to undo this dumb move, so if they don't, we probably know the answer. >>18659 >>18662 I can add an option here to have it say more time units. >>18667 Sorry about the bad domain error handling. You usually get that when the server is going down/under heavy load. Hydrus hits a stop gap and just takes long breaks to try again later, but this tactic doesn't work so well on servers that are always on fire. A future domain rework will add more options and tools here. The 401 on e621 might be cloudflare. They've been giving some new status codes recently. They used to always give 503 or something, but we've seen 403 and maybe 401 (not sure I remember right) recently based on different defense modes. 401 usually means some sort of 'you aren't allowed to access this'. You can try help->debug->network->fetch a URL and then put in an e621 URL. Save that to desktop and see if it has cloudflare shit in its html. If so, see if you can use Hydrus Companion to sync your cookies and User-Agent across to hydrus. If not, let me know what you see. Don't upload the thing to pastebin or anything though--it may have private info for your e621 account. >>18671 I know nothing about that bug, so can't talk cleverly about it. The openssl version bundled with the client will be what is on the github runners for Ubuntu 20.04, so I assume it will be fairly good. The best way to get the latest openssl version for you, though, is to run from source. As I've said to several other Linux users in the above posts, please try out the 'running from source in Linux' easy setup coming out next week. Then hydrus will use whatever version you have on your system.

>>18672 >>18673 >>18676 >>18677 Thanks. A user fixed the parser, rolling it out next week in the update. We'll see how crazy Elon's guys go here in the coming weeks. >>18674 >>18675 Yep, sorry, .txt is tricky for some bullshit design reasons. It will come eventually, since lots of people want it, but I have to do a bunch of work first. In the meantime, I recommend something like Joplin for storing notes. I use it every day, and it lets you have multiple clients sync with the same folder on a simple network share. Great for writing notes from your work machine to your personal one, or your repository of ascii art, stuff like that. >>18678 Sorry for the trouble. They are working with some shit tools. I will double-check all the stuff here is working correct, since we've had more than one report of confusing bans related to this tag-account lookup stuff. I felt it was due to unusual duplicate filter content merging until now (people uploading stuff they didn't know), but your situation is clearly not that.

>Have exact duplicates of a file >Different filenames >Always add filenames as tags whenever importing >Grabs both filenames, just have to pick one Nice.

I'm having trouble with mpv not being found on the Linux executable release from the Github (v504). First libcrypt.so.1 was missing, so I had to install a package (libxcrypt-compat on Arch) as it was not in the ./Hydrus Network/ folder. After that I get this more cryptic error: Traceback (most recent call last): File "PyInstaller/loader/pyimod03_ctypes.py", line 53, in __init__ File "ctypes/__init__.py", line 373, in __init__ OSError: /usr/lib/libgio-2.0.so.0: undefined symbol: g_module_open_full The above exception was the direct cause of the following exception: Traceback (most recent call last): File "hydrus/client/gui/canvas/ClientGUIMPV.py", line 25, in <module> import mpv File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "PyInstaller/loader/pyimod02_importers.py", line 499, in exec_module File "mpv.py", line 54, in <module> File "PyInstaller/loader/pyimod03_ctypes.py", line 55, in __init__ pyimod03_ctypes.install.<locals>.PyInstallerImportError: Failed to load dynlib/dll '/path/to/Hydrus Network/libmpv.so.1'. Most likely this dynlib/dll was not found when the application was frozen. My guess is that my system's version of libgio-2.0.so.0 is different from the one Hydrus is expecting. This file is not in the ./Hydrus Network/ folder either. Could this be solved by including Hydev-approved libcrypt.so.1 and libgio-2.0.so.0 in the release? I know there is an AUR package, and it works, but I want to keep a backup of the program along with my database backups so I can boot it without having to go spelunking through the release pages. Also the AUR package's install size is nearly twice that of the extracted executable release.

>>18668 Ran current from source, mpv works! With the setup_venv_advanced.sh script: >had to change `python` to `python3` [I don't know if `python3` works on other distros] >had to `apt install python3.10-venv` These were both made obvious due to descriptive error messages so it's not a significant issue, just letting you know.

>>18687 Had this error, was fixed when running a fresh install from github source: https://hydrusnetwork.github.io/hydrus/running_from_source.html Process is super easy so definitely give it a try.

This may be useful for devanon compiling with Ubuntu. Python shenanigans.

a new query from a kemono.party patreon subscription I have is saying that it found no files on it's first sync, but it did and it downloaded them. Now the query says it's dead because of that, but it shouldn't be because it downloaded the files, so I don't know why it's saying there's none.

>>18691 I think I figured out the problem. The query's last file was over a year ago, so it's marked as dead because of that, but hydrus is saying that the query is dead because it found no files on its first sync. This is it's first sync, but it still found files, just none with a date within a year ago as the source time, which is the cut-off point for a subscription to still be considered active, at least for me it is. So it seems like the messages that hydrus gives are being mixed up here somehow.

Is there any way to exclude certain tags from being copied over when filtering duplicates? Specifically the filename: tag. I prefer to keep the original filenames, but filtering duplicates ends up with creating multiple filenames and things getting all jumbled up. I'm (slowly) working through a huge 38k potential duplicates backlog and this is creating a lot of extra work for me.

I had a great couple of weeks working on a variety of features. I fixed a heap of bugs, made it possible to search for files that have EXIF and other human-readable file metadata, and updated and translated my easy setup scripts, so it is now easy for any user on any platform to run the program from source. The release should be as normal tomorrow.

>>18694 So you're suggesting that all the linux users who are currently having trouble with mpv switch this week to running from source now that it should be easy to do. How should the switch be done? Is it safe to just delete everything except for the db folder and then put the source in place of the linux version like you're doing a clean update?

>>18695 For what it's worth, here is what I did (riiiight before we got the easy Linux scripts, but it still felt painless thanks to the help page): I cloned the repo somewhere else (not removing my "old" built hydrus directory), installed my venv then python3 client.py -d /path/to/hydrus/db/. I since cleaned up a bit, but I kept both versions side by side for a little while, just in case, with no issue (only one instance running, of course). Seeing how easy it is to handle and update from source, I'll probably never go back to the built version (but IMO it can still be valuable, especially for newcomers).

>>18677 Thank you, kind anon.

I'd like to preface this by saying that I'm a double-digit IQ brainlet but tag siblings have me stumped at the moment. My goal is simple, when I search for 'character:lana (pokemon)', I also want files with the tag 'スイレン(トレーナー)' to show up in my searches. I thought it was as easy as setting up the sibling relation you see in pic related but this does not produce the result I was hoping for. Searching for 'character:lana (pokemon)' does not show results that are only tagged with her Japanese name. I think I'm either misunderstanding the sibling function entirely or I set up the sibling relation in the wrong tab (my tags). My PTR tags are also not fully sync'd yet but I don't think that matters, seeing as the files in question already have their tags and such.

Yep, check the updated help here: https://hydrusnetwork.github.io/hydrus/running_from_source.html Basically you'll do a 'clean install', clearing out your old .so files, then extract a fresh source install and run the new .sh script. The guide walks you through all of it. Make a backup before you do it, and let me know if you have any trouble!

https://www.youtube.com/watch?v=Z1MV0WbrNwY Windows release got a hotfix! If you got 505a right after the release was posted and everything is a bad darkmode, get the new one! windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v505b/Hydrus.Network.505b.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v505b/Hydrus.Network.505b.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v505a/Hydrus.Network.505a.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v505a/Hydrus.Network.505a.-.Linux.-.Executable.tar.gz I had a great couple of weeks fixing bugs, exposing EXIF and other embedded metadata better, and making it easier for anyone to run the client from source. Full changelog (big one this week): https://hydrusnetwork.github.io/hydrus/changelog.html EXIF I added tentative EXIF support a little while ago. It wasn't very good--it never knew if a file had EXIF before you checked, so it was inconvenient, and non searchable--but the basic framework was there. This week I make that prototype more useful. First off, the client doesn't just look at EXIF. It also scans images and animations for miscellaneous 'human-readable embedded metadata'. This is often some technical timing or DPI data, or information about the program that created the file, but, most neatly, for the new AI/ML-drawn images everyone has been playing with, many of the generation engines embed the creation prompt in the header of the output png, and this is now viewable in the client! Secondly, the client now knows ahead of time which files have this data to show. A new file maintenance job will be scheduled on update for all your existing images and animations to retroactively check for this, and they will fill in in the background over the next few weeks. You can now search for which files have known EXIF or other embedded metadata under a new combined 'system:embedded metadata' predicate, which works like 'system:dimensions' and also bundles the old 'system:has icc profile' predicates. Also, the 'cog' button in the media viewer's top hover window where you would check for EXIF is replaced by a 'text on window' icon that is only in view if the file has something to show. Have a play with this and let me know how it goes. The next step here will be to store the actual keys and values of EXIF and other metadata in the database so you can search them specifically. It should be possible to allow some form of 'system:EXIF ISO level>400' or 'system:has "parameters" embedded text value'. running from source I have written Linux (.sh) and macOS (.command) versions of the 'running from source' easy-setup scripts. The help is updated too, here: https://hydrusnetwork.github.io/hydrus/running_from_source.html I've also updated the setup script and process to be simpler and give you guidance on every decision. If you have had trouble getting the builds to work on your OS, please try running from source from now on. Running from source is the best way to relieve compatibility problems. I've been working with some users to get the Linux build, as linked above, to have better mpv support. We figured out a solution (basically rolling back some libraries to improve compatibility), so more users should get good mpv off the bat from the build, but the duct tape is really straining here. if you have any trouble with it, or if you are running Ubuntu 22.04 equivalent, I strongly recommend you just move to running from source. If these new scripts go well, I think that, in two or three months, I may stop putting out the Linux build. It really is the better way to run the program, at least certainly in Linux where you have all the tools already there. You can update in a few seconds and don't get crashes! misc highlights If you are interested in changing page drag and drop behaviour or regularly have overfull page tab bars, check the new checkboxes in options->gui pages. If you are on Windows and have the default options->style, booting the client with your Windows 'app darkmode' turned on should magically draw most of the client stuff in the correct dark colours! Switching between light and dark while the program is running seems buggy, but this is a step forward. My fingers are crossed that future Qt versions improve this feature, including for multiplatform. Thanks to a user, the twitter downloader is fixed. The twitter team changed a tiny thing a few days ago. Not sure if it is to do with Elon or not; we'll see if they make more significant changes in future. I fixed a crazy bug in the options when you edit a colour but find simply moving the mouse over the 'colour gradient' rectangle would act like a drag, constantly selecting. This is due to a Qt bug, but I patched it on our side. It happens if you have certain styles set under options->style, and the price of fixing the bug is I have to add a couple seconds of lag to booting and exiting a colour picker dialog. If you need to change a lot of colours, then set your style to default for a bit, where there is no lag.

[Expand Post]

next week I pushed it hard recently, and I am due a cleanup week, so I am going to take it easy and just do some refactoring and simple fixes.

>>18693 click "edit default duplicate metadata merge options" on the duplicates window. https://hydrusnetwork.github.io/hydrus/duplicates.html#duplicates_merging you can set certain tags or namespaces to be blacklisted. >>18698 there could be a couple things going on here. >Searching for 'character:lana (pokemon)' does not show results that are only tagged with her Japanese name. the way siblings work is that it will look like the file has the ideal tag instead of the tag it actually has. in your case, the japanese name will display as the english name. you won't see the japanese name anymore. files with the japanese name will look like the have the english name. if you open the "manage tags" dialog for the file, you'll see something like pic related - the true tag, and the ideal tag it displays as. >I thought it was as easy as setting up the sibling relation the other thing about siblings and parents is that the client has to take some time to apply them. you can leave the client idle for a while and siblings/parents will catch up on their own. to force them to catch up, go to "tags" > "sibling/parent sync" > "review tag sibling/parent maintenance", and then make sure you're on the "my tags" tab (or whatever tag service you created the sibling relation in). it will say something like "2 siblings left to apply". click the refresh button. your client will hang for a bit while it does the work.

>>18701 >if you open the "manage tags" dialog for the file, you'll see something like pic related - the true tag, and the ideal tag it displays as. thanks for clearing things up. i can't replicate this at all even though my sibling/parent tags are fully synched. could the issue be that i'm setting these sibling rules for 'my tags' but the existing tags on the files are all in the PTR section in the 'manage tags' window? if so, would i need to migrate the PTR tags to 'my tags' somehow?

>>18702 >could the issue be that i'm setting these sibling rules for 'my tags' but the existing tags on the files are all in the PTR section in the 'manage tags' window? yes, i think that would be it. >if so, would i need to migrate the PTR tags to 'my tags' somehow? you could do that, it's under "tags" > "migrate tags". but migrating tags is a one time thing and it takes a while and it isn't easy to undo if you make a mistake. i think it would be easier to just add your sibling rules to the PTR service instead of 'my tags'? i don't use the PTR, but i think you can do that? also, i think it's possible that the sibling relations you're looking to add are already in the PTR but simply haven't been added to your client yet because the PTR isn't done syncing? hopefully someone more familiar with the PTR can chime in.

On the help page for hydrus's multiple local file services, it says you can use them to perhaps separate your SFW images from your NSFW images. https://hydrusnetwork.github.io/hydrus/advanced_multiple_local_file_services.html However, unless I'm missing something, it seems that tag favorites are entirely unrelated to either file services or tag services. You have the same favorite tags no matter what file service or tag service you're looking at. This makes it kind of difficult to use tag favorites at all if you're planning on making a SFW/NSFW split. Are there plans to do something about this? Will tag favorites become tied to a tag service? Perhaps 'tag favorites' could become a new type of service, so you could separate tag favorites into groups however you want.

>>18700 I'm an anon running on Debian GNU/Linux 11. I updated from v504 LINUX MPV TEST 2 to v505a successfully using the "505a.-.Linux.-.Executable.tar.gz" package. So far, so good. Thanks anon. ----- Now regarding to the SOURCE file. I decompressed the file and moved it to an USB drive. Then the next step, according to the instructions at "https://hydrusnetwork.github.io/hydrus/running_from_source.html#environment_setup" is to double click on "setup_venv.sh", but it didn't do anything, so I made it executable but double clicking on it won't launch it. So I used the terminal and it worked with the command "./setup_venv.sh" (by the way, you have a misspelled typo in the instructions). Next I tried to build the Help, but I have the message: "You need to set up a venv! Check the running from source help for more info!". So, I skipped building the Help and tried to run Hydrus with the script "client.sh", but I have the same error mensage than above: "You need to set up a venv! Check the running from source help for more info!" I have no idea what a "venv" is. I'm attaching screenshot of the whole Konsole's output showing the process.

>>18705 I'm also running Debian. I think the venv script may need to target python3 specifically instead of python for certain platforms. I ended up following the "doing it manually" section further down the page to set up the venv. I needed to first install nenv itself (apt-get install python3-venv) so that should probably be added to the setup script as well. I was able to get the client running successfully from source. >>18700 I was able to update SQLite on Linux by using the following command: pip3 install pysqlite3 Note: I had to first install libsqlite3-dev (apt-get install libsqlite3-dev) before the above would work.

>>18706 I did as you said but there is no success yet. By the way, I deleted the previous failed folder and I did the new attempt to build from source with a fresh decompressed source file. Thanks anon anyway.

I switched to running hydrus from source like you said by downloading the .tar.gz file from github, but is it possible for me to now update hydrus using git? I don't really know how git works but you make it sound like updating that way is much easier.

>>18708 just reinstall from source using git instead. I don't know if the tar.gz files are exactly the same as the git source, but running: git clone https://github.com/hydrusnetwork/hydrus and following the venv setup steps again is all you need. I finally got sick of the AUR version being fucked and switched, and it was pretty easy. you should be able to "git pull" to update from then on, but I don't know if we're able to git pull while the program is running or not?

I'm using hydrus on kde with kde's breeze dark theme. Is there a way to make the default hydrus Qt style have light text in the prompt text of these text boxes since the text boxes themselves are dark because of my dark theme? With dark mode in hydrus on or off, the textboxes and the prompt text are both still dark.

Just switched from the AUR version to source and it seems to work perfectly, but Hydrus can't find my Qt styles I really don't know anything about Qt or python virtual environments

Is it possible to edit the 4chan and 8chan parsers so that files downloaded get the text of the post they were attached to added as a note. Is this something that downloaders are capable of? I wanted to try to learn how to make downloaders to see if I could do this but I figured I should ask if it's even possible, since I don't think any downloaders I've used do anything like this.

>>18707 I didn't use the setup script I just did the manual setup: https://hydrusnetwork.github.io/hydrus/running_from_source.html#what_you_need Also you need to switch to the venv before running the client. https://hydrusnetwork.github.io/hydrus/running_from_source.html#running_it Just run "source venv/bin/activate" in the hydrus directory then run the client. Afterwards type "deactivate" to exit the venv.

>>18713 I did as you said, but still no luck. It is complaining about a "yaml" missing module, so I installed with the package manager a package named "python3-yaml" and executed again "client.py", but no success yet.

>>18714 >running from source >chasing dependencies That is why the normies and even seasoned anons will run for the hills.

>>18714 I think you forgot the most important step; "python -m pip install -r requirements.txt". That's what will install dependencies; no need to chase them. (You should also start the client with "python client.py", so that you don't modify the permissions on the file and don't have some merge conflicts on your next git fetch.)

I've got a question. How does System:Known URL work? I'm getting URL's from places I have never visited. Is there some kind of online database it's getting these from?

>>18717 the parser adds additional URLs Like a booru source's URLs

>>18716 >I think you forgot the most important step; "python -m pip install -r requirements.txt". That's what will install dependencies; no need to chase them. They are all already satisfied but still it is complaining of a missing module "yaml".

>>18719 I think I know what's wrong; the venv creation and activation need to come before pip stuff; IMO you should: - Delete the "venv" directory - Be consistent with your "pip" calls; it's safer IMO to always call "python -m pip" (or "python3 -m pip") rather than "pip" (or "pip3"), as you don't know what your system's default is. They may be the same, but IDK, so let's assume the worst - Create the venv again with "python3 -m venv venv", then "source venv/bin/activate" - Run the pip stuff here, all of the "python3 -m pip install <stuff>" - Try to start hydrus again from the venv (your shell needs to mention "(venv)" I believe you've installed deps outside of the virtual env, then went inside and tried to start Hydrus; your Python env is incomplete because of that.

>>18684 hi dev, i'm the guy with e621 problems. here's the error after doing the URL thing. I don't know what cloudfare identifiers would look like. v503, win32, frozen MissingCredentialsException 401: The server's error text was too long to display. The first part follows, while a larger chunk has been written to the log. Traceback (most recent call last): File "hydrus\core\HydrusThreading.py", line 401, in run callable( *args, **kwargs ) File "hydrus\client\gui\ClientGUI.py", line 1282, in thread_wait network_job.WaitUntilDone() File "hydrus\client\networking\ClientNetworkingJobs.py", line 1972, in WaitUntilDone raise self._error_exception hydrus.core.HydrusExceptions.MissingCredentialsException: 401: The server's error text was too long to display. The first part follows, while a larger chunk has been written to the log. <!doctype html> <html> <head> <title> Unexpected Error - e621</title> <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png"> <link rel="icon" type="image/png" sizes= <!doctype html> <html> <head> <title> Unexpected Error - e621</title> <link rel="shortcut icon" href="/favicon.ico" type="image/x-icon"> <link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png"> <link rel="icon" type="image/png" sizes=

>>18688 Great, I am glad it works. And thanks, I will update the scripts/help about this! I feel like I can detect if your system has the 2/3 dichotomy symlinked and adjust accordingly. I do something similar in the windows script. I can probably safely set it to always be 'python3' on linux, but I'm sure there is some exception somewhere. >>18690 Thanks. The Ubuntu build is on 3.9 now, and it seems like the step up to 3.10 is causing several little problems like this. There's another one with OpenCV, which PyInstaller also has problems with freezing if it goes above a certain version. I am feeling more confident that I should drop the build entirely in a couple months once I have the 'running from source' kinks ironed out. >>18691 >>18692 Thanks. This is a special case, and I'll see if I can catch the logic and put a nicer message out. >>18695 Shit, this was me failing to reply to you >>18699 This >>18696 sort of plan is great. >>18698 Your logic looks good here, and you have understood how siblings work. As >>18701 says, I think you might have to wait a bit for your processing and sibling sync to all catch up. One thing to be careful of--make sure you are setting siblings on the correct tag service, and make sure the siblings are set to apply where you want them. This system can get fivehead if you want, but if you added those siblings to something other than the PTR, they will not apply to the PTR's tags by default. Hit tags->manage where tag siblings and parents apply if you need to say something like 'set my custom siblings I set on "my tags" to the PTR'.