https://www.youtube.com/watch?v=v6ng_bGozSo windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v640/Hydrus.Network.640.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v640/Hydrus.Network.640.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v640/Hydrus.Network.640.-.Linux.-.Executable.tar.zst I had a great week. There's a mix of work. v636 was the last release with a macOS App. Please check that release, https://github.com/hydrusnetwork/hydrus/releases/tag/v636 , for information on how to convert a macOS App install to a source install! Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html new UI navigation Thanks to a user, we have several new UI navigation quality of life features: A normal media viewer now has a 'show random' button in the top hover window, next to the previous/next stuff. It does a pure random navigation, not a shuffle. Right-clicking the button will undo! The Main GUI's 'pages' menu now has a 'history' submenu, which tracks the tabs you have recently gone through, in that order. Users who use multiple simultaneous media viewers have a new shortcut to highlight the media viewer's current media+page from the media viewer, this time without closing the media viewer. duplicates options->duplicates has a new 'auto-commit completed batches of this size or smaller' setting. It defaults to '1', so if you get a 1/1 group, you'll now just click through it, no annoying interstitial commit prompt. You can try boosting it to 3 or 5 if you like, or just turn it off. The 'show some random pairs' button is now nicely asynchronous and will not block the UI. Duplicates auto-resolution has better OR support inside 'test A or B' comparators. There's two new hardcoded comparators, for 'both files have the same "has exif/icc profile" value'. And there's another new comparator that does an OR of a list of comparators it owns. I've used some of this tech to re-do and merge the suggested auto-resolution rules a little. We are narrowing in on what I want this system to look like for 'launch'. If you have been following this stuff closely, please check the changelog. some other stuff Huge file deletes should be a bit smoother, and if they take a long time, a file progress popup appears. Export Folders get two new checkboxes to handle sidecar file regeneration, either just on the next run or on every run. If you are a downloader creator, check the changelog. I worked on a bunch of technical stuff. The Client API can now fetch potential duplicate pairs with a sort and 'group mode'. next week

[Expand Post]

I want a boring cleanup week. There's some help documentation I want to get to, as well.

>>24279 >you'll now just click through it, no annoying interstitial commit prompt. It's annoying when I make a mistake at the last pair and can't undo it.

>>24279 Thank you.

Hello /t/ hydrus network users. I haven't lurked here much so jumping in for immediate self-promo is probably a bad idea. But anywho, I wrote a webshit client for your hydrus network setups. This seems to be the best venue to reach hydrus network users, so here I am. The software is available here: https://github.com/hydrui/hydrui - and hosted on hydrui.dev. I wrote this because I run hydrus network on a headless linux box and haven't been satisfied with the different solutions for remoting to it. I still use xpra, but I've been using hydrui and hydrus companion for a whole lot of my hydrus network usage these days. I have zero clue how this will hold up to more serious usage than mine, but I hope someone finds it useful. I don't really have anything in particular to gain here, i'm grateful for hydrus network and wanted to contribute something to the ecosystem.

>>24284 needs more screenshots imo

>>24286 Yeah, good point. Fixed that. https://hydrui.dev/en/docs/screenshots/

>>24284 Very cool, thank you for sharing! I don't know if you're open to feedback (I know personal projects aren't often made for that), but in case you are, here are a few things I wish it did; first one is "as a regular user" and others are "because my client is huge": - I wish you talked about (and checked for, ideally) granular permissions; you use "permits everything" on the API and IMO you should always try using as little as you can - I wish I could use `system:` tags, or at least limit the number of results fetched - I wish you handled pages of pages more gracefully; at the moment the app is flattening the pages into a single line (it definitely works though) Likely a typo, but there's also an https link to localhost in your doc on the getting started page -> Accessing the Client API where ""use https" is disabled on the screenshot right above. Great docs overall, congrats!

>>24290 >I don't know if you're open to feedback Definitely. >- I wish you talked about (and checked for, ideally) granular permissions; you use "permits everything" on the API and IMO you should always try using as little as you can Good idea. I need to look into this sometime. >- I wish I could use `system:` tags, or at least limit the number of results fetched There's no good UI for it, but you can do `system:limit is 100` or `system:filetype is image/vnd.adobe.photoshop` in the meantime. >- I wish you handled pages of pages more gracefully; at the moment the app is flattening the pages into a single line (it definitely works though) I didn't realize this was a feature of hydrus network until recently. Supporting this should be easy, but I definitely need to clean up some ugly hacks first. >Likely a typo, but there's also an https link to localhost in your doc on the getting started page -> Accessing the Client API where ""use https" is disabled on the screenshot right above I want to redo all of those screenshots, both because of this issue, and also just because they kind of look like ass. Originally I was going to recommend http for simplicity, but once I deployed Hydrui to a real domain it turned out that dealing with mixed http/https content errors was actually much harder and less consistent than just adding the TLS exception for a self-signed certificate. I couldn't find a way to do the former without globally disabling useful security features in Librewolf, so I gave up on it... but I didn't fix the screenshots to reflect that. Thanks for the feedback.

>>24291 Thanks for the reply! Good to know about copying the `system;` stuff, I did not realize. I'd also love to see sorting options, but the interface is already very usable as-is. I wanted to reply on the TLS stuff, as I had the same issue a while ago with https://hydrus.app; same as you, I use Librewolf so no good way of disabling this verification from a real domain. I now use Tailscale's Serve, which provides a valid TLS cert; it allows me to expose Hydrus through HTTPS, but (of course) only inside my network (I use one-letter machine names for extra anonymity). I highly recommend you take a look! Doesn't fix your docs issue, but maybe it helps you.

Hey what's the status with BTRFS for holding my database? I heard there were some issues with it before? I have been slacking with db backups since it takes 12 goddamn hours to make my fancy deuplicated backup and I'm looking to btrfs' snapshot feature to reduce the downtime, plus having the db on a self healing array would be nice for peace of mind, anyone done anything like this, have any advice?

>>24284 docker run --rm -it ghcr.io/hydrui/hydrui Unable to find image 'ghcr.io/hydrui/hydrui:latest' locally docker: Error response from daemon: Head "https://ghcr.io/v2/hydrui/hydrui/manifests/latest": unauthorized

>>24294 Sorry, I fucked up two different ways. (Package was set to private, no `latest` tag because docker metadata action wants valid semver tags.) Should work now.

>>24293 There are no issues. I'm running on btrfs for exactly the same reasons. Recently updated docs: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#installing If you want to keep db latency minimal, you do have to be careful with compression. The Disk latency, CPU latency, compression factor equation is extremely machine dependent and can change when your DB grows or fragments.

>>24280 Thanks, yeah, I guess this one is a bit subjective. You can turn it off in the options, and you should go back to the old behaviour. >>24284 I just emailed you--looks great, and love the website. I'll add it to the Client API help and mention it in the v641 release post. >>24290 >>24291 I'm planning a large overhaul of the permissions, btw. It is all a mess, and the granular search permissions ended up so inconvenient that users mostly do just say 'allow everything'. I'll collapse/merge the permissions down significantly, probably more to a 'read content/write content/do admin stuff' kind of mix rather than specific content types, and then search permissions will be defined by local file domains, which have nice clear barriers between each other throughout the code. If anyone has opinions on how they'd want permissions to work, I'm open to ideas. What do you want to not allow?

Hydrui dev again. I just added a new experimental feature, automatic tagging. This is more or less just a web port of the wd-e621-hydrus-tagger project and it uses the same models, just with weights converted from onnx to ort for web usage. https://github.com/hydrui/hydrui/releases/tag/models I think it's pretty useful, though I have to iron out some issues: no support for ratings tags, no great way to tag more than one image at once, automatically downloading models is broken in Server Mode, it hard-depends on OPFS which unnecessarily makes it not work on old iPads and Ladybird Browser. But hell, it works in most browsers right now with no setup and even without WebGPU. Just need a decent CPU. If anyone has any other ideas for what ML-based features would be cool throw em at me. Some stuff I'd love to add would require a huge model that can never work directly in a browser. Like, we could support image text transcription/translation with Gemma 3-27b, but that is not going to fit in a browser. I could look into llama.cpp/ollama API support maybe. Ideas welcome. Can throw them at my Github issues too. >>24299 Thanks! Regarding permissions: If I understand you correctly, I think having the granular permissions like for files be per-file-service is a swell idea and works for any use case I care about. You could possibly extend this to have these granular permissions for other services, like tags or ratings services, i.e. to allow access to some tag services and not others, or readwrite access to some tag services and readonly access to others. As a developer, I would like to be able to provide a minimal policy (or perhaps multiple different policies to choose from) that users can easily import to get minimum privileges for an API key. Just some kind of file. (Maybe similar to the way downloaders are installed.) It would also be nice if API clients had a way to discover their granted permissions. That would enable me to ensure I don't allow a user to waste their time editing tags if they can't actually submit it anyways.

I downloaded an artist from danbooru (with these standard downloader) and now I want to fetch them again to apply a tag change. How do I do that? Tried the tag import options but can't figure out how to do it.

>>24298 Nice, yeah I wasn't sure about compression, it would be nice to keep the db small but I often wait on the db to finish tasks already, compressing it is probably not a good idea. I knew btrfs was mentioned in the docs but I couldn't find it, guess I should look harder next time, lmao.

>>24301 right-click, urls, force metadata refresh

Would you mind adding a command line log message for when Hydrus enters and exits idle mode? I'm trying to figure out when my client decides to enter and exit idle mode based on my current settings for it, but it's hard to know when it activates and deactivates. It can be optional and in the debug menu if you want.

>>24305 +1, this could be useful for me as well.

When you try to add a query to a subscription, and Hydrus tells you that you already have that query, it should also tell you the label that you set for the query, if it has one. I was having trouble finding a query that I wanted to edit in a large subscription of mine, and since Hydrus said was that it's already there but not what I labeled it as, it was very difficult for me to find it.

I had a great week. There's more duplicates work, a mix of small jobs done, some new help, and an important improvement in crash reporting. The release should be as normal tomorrow.

I'm surprised! You can run hydrui (and hydrus.app too, it seems) without even having to run a server: > Hydrui acts entirely as a pure frontend web application. There is no Hydrui server component; Hydrui connects directly to the hydrus client API. This is how hosted Hydrui works. Connect to your local hydrus from someone else's website! Official site is very clear and effective, you've got pretty good writing skills. >>24284 >headless box Is there any way to use this from hydrus_server.py? That one's actually headless.

Is there any way to add a follow url to a content parser only if it ``didn't`` find anything? (that is, the opposite of "next gallery page") In order to avoid causing an xy problem: I'm trying to modify the danbooru parser so that if a gallery search returns empty it repeats it with a "status:deleted" added, I currently have a separate gug that adds it which I use when doing md5 searches but that ends up doing unnecessary extra queries since it still triggers even if the file is found the first time.

I have a rating service, and in that service I'm trying to make sure that I give files each value of the rating (like 3/5, 7/20, etc.) roughly evenly. I've been checking this using a page of pages that has every file that's given each rating loaded, but that adds a lot of session weight and it's also annoying to slowly reload everything to check. I'd appreciate if instead, the "review services" tab for a rating service gives the number of files that have each value of a rating, so I can easily see how the ratings are distributed. I see that it already gives the number of files that have any rating in that rating service, so having values below that for each value individually would be really helpful.

>>24312 first thing I came up with, solves your x. Not sure how to get the existing query back, but maybe you can transform something on the danbooru page instead. - Added a pre-parsing conversion that inserts an URL unconditionally at the end of the document. - Write a content parser that expects to find one URL, but incidentally also finds your injected URL. - Select the first result.

https://www.youtube.com/watch?v=O4ssOwsAJ8s windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v641/Hydrus.Network.641.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v641/Hydrus.Network.641.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v641/Hydrus.Network.641.-.Linux.-.Executable.tar.zst I had a great week cleaning old things, writing new help, and polishing some duplicates work. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights First off, I had hoped to launch a new crash reporter today, but I'm disabling it for now. The tech is cool and dumps useful data on a crash, which we haven't had access to before. Unfortunately, in testing I discovered that it actually causes crashes after a few seconds of mpv playback. I will put more work into this next week. I wrote some proper help on how to merge two clients: https://hydrusnetwork.github.io/hydrus/database_merging.html . I expect this document to mature in future as we figure out nicer ways to do all this. I put more work into duplicates auto-resolution. The new duplicate filters that launch from the preview/review panels work better and have approve/deny buttons (which you can map to shortcuts for quick workflow), and I've adjusted the suggested rules yet again based on feedback. I have also given the help a pass, particularly talking about the new suggested rules: https://hydrusnetwork.github.io/hydrus/advanced_duplicates_auto_resolution.html . I am close to a 1.0 of this whole system and hope to invite all normal users soon. A user launched a neat web portal for the Client API this past week. It creates an interface similar to the normal GUI, but in your web browser. You can find it here: https://hydrui.dev/ . I also added links to 'hydrus-automate' and 'hybooru' to the project list in the client api help here: https://hydrusnetwork.github.io/hydrus/client_api.html next week I would like to finish off a new tuning suite for my 'visual duplicates' detection algorithm, to improve its confidence and precision and get it working across differing jpeg subsampling.

>>24315 got an error trying to dl an image from a source I used to no problem v641, win32, source AttributeError module 'datetime' has no attribute 'UTC' ==== Exception ==== AttributeError: module 'datetime' has no attribute 'UTC' ==== Traceback ==== Traceback (most recent call last): File "D:\hydrus\hydrus\hydrus\core\HydrusThreading.py", line 450, in run callable( *args, **kwargs ) File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworking.py", line 465, in MainLoop self._jobs_awaiting_bandwidth = list( filter( ProcessBandwidthJob, self._jobs_awaiting_bandwidth ) ) File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworking.py", line 238, in ProcessBandwidthJob elif not job.TryToStartBandwidth(): File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworkingJobs.py", line 1813, in TryToStartBandwidth result = self.engine.bandwidth_manager.TryToStartRequest( self._network_contexts ) File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworkingBandwidth.py", line 661, in TryToStartRequest self._ReportRequestUsed( network_contexts ) File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworkingBandwidth.py", line 199, in _ReportRequestUsed bandwidth_tracker.ReportRequestUsed() File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 762, in ReportRequestUsed dt = self._GetCurrentDateTime() File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 380, in _GetCurrentDateTime return datetime.datetime.fromtimestamp( HydrusTime.GetNow(), datetime.UTC ) AttributeError: module 'datetime' has no attribute 'UTC' ==== Stack ==== File "C:\Users\Nick\AppData\Local\Programs\Python\Python39\lib\threading.py", line 937, in _bootstrap self._bootstrap_inner() File "C:\Users\Nick\AppData\Local\Programs\Python\Python39\lib\threading.py", line 980, in _bootstrap_inner self.run() File "D:\hydrus\hydrus\hydrus\core\HydrusThreading.py", line 465, in run HydrusData.ShowException( e ) File "D:\hydrus\hydrus\hydrus\client\ClientData.py", line 174, in ShowExceptionClient ShowExceptionTupleClient( etype, value, tb, do_wait = do_wait ) File "D:\hydrus\hydrus\hydrus\client\ClientData.py", line 213, in ShowExceptionTupleClient message = HydrusData.PrintExceptionTuple( etype, value, tb, do_wait = False ) File "D:\hydrus\hydrus\hydrus\core\HydrusData.py", line 349, in PrintExceptionTuple stack_list = traceback.format_stack() ===== End =====

>>24316 trying to open up review serviecs causes this too v641, win32, source AttributeError module 'datetime' has no attribute 'UTC' File "D:\hydrus\hydrus\hydrus\client\gui\ClientGUIMenus.py", line 280, in event_callable callable( *args, **kwargs ) File "D:\hydrus\hydrus\hydrus\client\gui\ClientGUI.py", line 5896, in _ReviewServices panel = ClientGUIClientsideServices.ReviewServicesPanel( frame, self._controller ) File "D:\hydrus\hydrus\hydrus\client\gui\services\ClientGUIClientsideServices.py", line 4101, in __init__ self._InitialiseServices() File "D:\hydrus\hydrus\hydrus\client\gui\services\ClientGUIClientsideServices.py", line 4187, in _InitialiseServices page = ReviewServicePanel( services_notebook, service ) File "D:\hydrus\hydrus\hydrus\client\gui\services\ClientGUIClientsideServices.py", line 1742, in __init__ subpanels.append( ( ReviewServiceRemoteSubPanel( self, service ), CC.FLAGS_EXPAND_PERPENDICULAR ) ) File "D:\hydrus\hydrus\hydrus\client\gui\services\ClientGUIClientsideServices.py", line 2459, in __init__ self._Refresh() File "D:\hydrus\hydrus\hydrus\client\gui\services\ClientGUIClientsideServices.py", line 2499, in _Refresh bandwidth_summary = self._service.GetBandwidthCurrentMonthSummary() File "D:\hydrus\hydrus\hydrus\client\ClientServices.py", line 909, in GetBandwidthCurrentMonthSummary return CG.client_controller.network_engine.bandwidth_manager.GetCurrentMonthSummary( self.network_context ) File "D:\hydrus\hydrus\hydrus\client\networking\ClientNetworkingBandwidth.py", line 369, in GetCurrentMonthSummary return bandwidth_tracker.GetCurrentMonthSummary() File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 622, in GetCurrentMonthSummary num_bytes = self._GetUsage( HC.BANDWIDTH_TYPE_DATA, None, True ) File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 545, in _GetUsage usage = self._GetRawUsage( bandwidth_type, time_delta ) File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 450, in _GetRawUsage dt = self._GetCurrentDateTime() File "D:\hydrus\hydrus\hydrus\core\networking\HydrusNetworking.py", line 380, in _GetCurrentDateTime return datetime.datetime.fromtimestamp( HydrusTime.GetNow(), datetime.UTC )

>>24316 >>24317 Hey, I am sorry--are you by any chance running in Python 3.10 or earlier? I do a boot test every week with Python 3.10, but this slipped through. I have just fixed this on master. Please git pull and let me know if you have any more trouble.

>>24305 >>24306 Thanks, good idea. I'll make a 'report mode' for it. I'll try not to make it too spammy. >>24308 Thanks, will do. >>24313 Great idea! This will be a little computationally expensive, but I can figure out a button that fetches the current values and draws a nice bar chart. We'll see what shape curves pop out.

>>24321 That fixed it. Thanks.

>>24321 also yes i use python 3.9 something

So I'm writing a parser for this gallery API I use that has multiple posts on each gallery page. That API gives everything right in the gallery page; title, tags, URL to the content to download, and a link to the original post URL. Because this is declared as a gallery (and not a post) in the downloader's URL class, hydrus never registers the original post URL that I set with "associable/source url", even though the parser sees it when tried manually. Is there a solution for me to register that original URL with the files? Or do I need to get the individual posts' URLs and fetch those instead? Hopefully I'd avoid parsing the individual pages as they're pure HTML (no json API there) and I already have everything I need from the gallery.

>>24284 Can't seem to log in at all, even having disable https global and mixed content in about:config (using librewolf, but couldn't get in with firefox either) Not sure what I'm doing wrong, I can get the api splash page to pop up no problem.

>>24326 Does the url class have this checked?

>>24259 I can't get any of the API stuff to recognize. Also, the parse only seems to work if I *don't* include &json=1, even though Hydrus is saying it doesn't look like HTML or JSON. My guess is it's the way that rule34.xxx redirects, as it'll go from an address that looks like rule34.xxx/index.php...etc. to api.rule34.xxx//index.php... etc. which it isn't recognizing. I'm also just now noticing the double slashes after the domain

>>24328 I have tried it, but it associates the URL of the gallery page and not the URL that's parsed by the downloader, which isn't what I am after. Thanks though!

>>24326 >>24330 I can't exactly model your problem in my head, but I think I get the gist of it. I think what you want is a 'subsidiary page parser', which is a mostly-undocumented way to split a gallery page into separate posts that contain groups of metadata that is to be assigned to each file separately. Check the 'e621 gallery page api parser' and '4chan thread api parser' for examples. I'm not totally sure why the url isn't being associated here. If you are getting the file imported ok, then presumably the URL is 'working', unless we are talking about a gallery hit that includes both the 'Post URL' and the direct file URL. If that's the case, then yeah the subsidiary page parser is the trick. The e621 one is probably what you want to look at. >>24329 If none of the API URLs are recognised in the URL Classes dialog, then that's your problem, it sounds like. You have to get a matching URL Class to get a URL to parse (remember the URL Class-Parser links in 'manage url class links'). If the URL is not recognised, hydrus will keep thinking that URL is a direct file URL that it should download. If you need to match a URL with double leading slashes, try adding a path component that matches anything, and example text the empty string, "", to remind yourself, and make sure the 'DEBUG: remove leading double-slashes...' option at the bottom of options->downloading is unchecked (it should be). If you need to match the 'json=1' URL, then add that as a parameter to the URL Class. Good luck!

I'm on Linux and going to try building from source to fix the mpv issues many have talked about. If I uninstall hydrus will it leave the database alone? And then I can just point it to the database again after I re-install from source? Of course, I'll be sure to backup everything before I start.

>>24332 >subsidiary page parser I should have started there; this is what I am using already. So this API has a `posts` object with a dict inside, and in that dict, each post has a downloadable file (which is the URL that gets associated with the file currently), some tags (those are correctly associated), and a source URL (the one I want associated with the file, and isn't currently). I do use the subsidiary page parser, but I'll review the e621 gallery parser to see what I'm missing. I'll share my downloader here when I get it to work, thank you!

If you have a duplicate filter page open, and it's still searching for duplicates while you click the button to start going through the filter, the searching tab will continue to search while the duplicate filter is loading. Because of this, they both go at half speed since Hydrus is needlessly searching twice. If possible, I think that the duplicate search that the tab does should just be "carried over" to the actual duplicate filter when you start it, so that Hydrus doesn't have to do every search twice. If that's not possible, then can you at least have Hydrus immediately stop searching on the page once you start the filter, so that it doesn't slow down a bunch from searching both at once?

>>24336 oh and you've been making great progress with the auto resolution system :)

I've noticed some old files that I dropped into Hydrus directly from my computer have retroactively gotten downloader tags. Just the creator: namespace though, since I disable the rest and prefer my personal tags. How exactly is this happening? The pages these hash matching files are being found on wouldn't be scraped by any of my subscriptions. Like this pic related for instance. I imported it over three years ago, and now I've noticed it has a pixiv url from a six year old post attached to it. This is the case for about a hundred or so other files out of tens of thousands. When, how, and why is Hydrus accessing these pages to compare hashes? Is it from the PTR? I don't use the PTR, but I did try once when I first got Hydrus before shortly canceling it upon realizing my computer couldn't handle it and that I prefer my own tags anyways. Could it be leftovers from that?

>>24338 >Could it be leftovers from that? If you once synced to the PTR, and never wiped the mappings from your DB after you stopped, then yes that's exactly what's happening

>>24339 Okay, I was a little freaked out and only thought of the PTR halfway though the post. I had thought everything related to it was wiped back then, but apparently not.

>>24293 >it takes 12 goddamn hours to make my fancy deuplicated backup Try vorta for borg backup. You can select what folders to backup where, and it does incremental, compressed, backups. I'd make sure to close Hydrus before backing up though. >>24300 Hellll yeah I love this. A good ML feature would be image/video upscaling, though that is obviously going to more than double storage size and take a long long time to process everything. >>24333 Source fixed my mpv issues. IMO source is misleading, because it's just a portable program with a binary launcher anyways. >uninstall hydrus What do you mean? It runs in a folder. Just download a source version and copy or move your db into the source one.

>>24340 nah, if you just stop syncing, then anything that was already processed stays. It's been years since I stopped myself, but I think to fully remove it, you need to go to the tag migration menu, and wipe the mappings and relationships, then delete the services, then vacuum the database files. Once you do that, it should be fully gone.

Hey hydev I have some borked files that I found when running hydrus video deduplicator. Most don't run at all so just assume every single one is nsfw. I ran a force redownload and only a few got fixed so these are bad on the site's side. https://catbox.moe/c/3uqay7 There's a text file with every url in it.

I was having a lot of trouble finding the place where you set which urls get displayed in the media viewer pop-up frame. I eventually found it, but I wasn't expecting it to be under "network → downloaders", since you're not changing anything about how files are downloaded and technically it's not even about downloading, just what's displayed. I was expecting to find it in the Hydrus options menu, and the first place that I checked was "media viewer hovers". I think it'd be a good idea to duplicate the settings for url display there, or at least have some text to tell users that they're actually in the "downloaders" menu bar item.

I had an ok week working on a mix of things. There's more duplicates work, some crash reporting tech, a new stability test for mpv, and a bunch of smaller stuff. The release should be as normal tomorrow.

Would you mind adding an option (or maybe just make it the way it works if no one minds, but I'm fine either way) to have the queue for auto resolution pairs to be sorted randomly? I'm noticing strings of similar pairs in my semi-automatic rules' queues, and that's not great for making sure that a rule is solid before making it fully automatic. Going through a random sample would give you more confidence that the rule is correct for all kinds of pairs that it'll encounter, rather than just the ones it happened to queue up first. If an actual random sorting isn't possible for whatever reason, then sorting it by sha-256 hash should also be fine. I think sorting that way is basically random.

https://www.youtube.com/watch?v=Y-IUZB3PI90 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v642/Hydrus.Network.642.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v642/Hydrus.Network.642.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v642/Hydrus.Network.642.-.Linux.-.Executable.tar.zst I had an ok week working on a mix of things. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights I have tweaked my visual duplicates algorithm with a new tuning suite that generates lots of synthetic data to build performance statistics on. Some false negatives are now 'probably duplicates', and some true negatives now have richer summaries than just 'not duplicates'. I still trust and recommend using the test at 'almost certainly' confidence in auto-resolution rules. I also triaged my remaining auto-resolution work. There are four medium sized jobs and perhaps a dozen smaller ones. I feel confident I can clear it all by the end of the year, and that'll be the system done for a v1.0. I improved the new crash reporter I was working on last week. Unfortunately there are still some false-positive crashes and logspam when the mode is mixed with mpv, so the mode is launching default off. You can turn it on under help->debug->debug modes->use faulthandler to log crashes. If you have had crashes and can test them without using mpv, please give it a go and let me know how you get on. If you are an advanced user and would like to help me test a new mpv thing, please check the changelog. next week I may tweak the new mpv interface a little more, but I mostly want to clear the final auto-resolution work. >>24343 I forgot to say: thank you, I got these. Some worked ok for me, some wouldn't render at all. Seems like some versions of either mpv or ffmpeg could or couldn't handle them. It is useful to have broken files like this, and this is a real mix of weird stuff. A couple of them helped me debug my mpv crasher code this week

How do I follow webcomics with hydrus? I've made a post parser, but now I need something to plug into a subscription or a watcher, right? Any tips / examples?

I dual boot linux and I use Hydrus on both windows and linux, using the same db. Stuff works nicely, but now on linux I have the weirdest issue (just the usual) When playing videos like mp4s... only every second video shows. The rest only play sound. However "every second" means the order of clicking on them >click on video 1 - no image >click on video 2 - image works >click on video 3 - no image >click on video 1 again - image works I have no idea how this shit happens, it's always on linux. I guess I should run hydrus from terminal and toggle mpv reporting or something?

>>24359 I had this issue before. I wish I could remember the fix, have you rebuilt your venv or are you very out of date? Looking back at my posts in a previous thread it looks like it ended up going away after some update. There was a debug mpv command added for this, see my pic.

>>24358 Basically, you need a post parser like you have, but then you need a gallery page parser that returns post urls for the post parser, and also tells Hydrus how to advance to the next page if there are more results. Once you have that, you create a GuG (gallery url generator) which is how you input a query to get the gallery urls. With all 3, you can create a subscription, which uses the GuG for queries, then runs automatically based on your timing settings for the subscription. You also want to make sure that the url classes for those pages are properly liked to the parsers. If you put example urls in the parser definition, then Hydrus will usually do it automatically, but sometimes it gets the linking wrong. So if Hydrus complains about there being nothing to parser, or not finding a parser, check the url class links menu to make sure that the classes and parsers are linked properly. The Hydrus webpage parser is kinda limited (particularly, it can't do Javascript at all) so there are many websites that just can't be used directly by Hydrus, but if you already made a post parser that works, you're probably fine there.

I'm the one who asked for the info on idle mode. Thanks for adding it. It helped me figure out the issue I was having. I had the mouse movement setting enabled for blocking idle mode, but only had it set for 1 minute. I was confused though because I never saw idle mode turn on, even if I haven't touched Hydrus in several minutes. But the idle report mode that you added made it clear that I was misunderstanding what the mouse movement setting was checking for. I assumed it was checking for mouse movement in the Hydrus gui, but it actually watches you mouse for movement anywhere while Hydrus is open, even if it's on a different window, or even a different monitor (or virtual desktop) than the one that has Hydrus. I wanted to use the mouse movement mode to have it block idle mode when I'm moving the mouse in Hydrus, since that means I'm probably using Hydrus, but let it turn on if I haven't moved the mouse in Hydrus for a little bit, since that means I'm probably not using it, so the idle mode lag won't bother me. Since the mouse movement setting looks for the mouse moving anywhere at all, that basically means that idle mode will only turn on when I'm not using my computer at all, which isn't what I want. So I turned it off, and now only have the "user action" one enabled. Anyway, that mode's helpful so thanks!

>>24018 >put your DB in the spot the DB is supposed to go, and everything should work fine. >>24037 >https://hydrusnetwork.github.io/hydrus/database_migration.html#to_new_OS >Install a fresh new hydrus on the new machine and then, using a network share or a USB drive, copy the database folder, files, and thumbnails from the old to the new. What about all my settings? There's stuff I figured out I wanted a certain way ages ago that I really don't want to have to redo and relearn, like color coding and manually ordering the display order for all namespaces.

>>24374 Your DB folder contains everything about your Hydrus install. All the options you have set will still be there if you're using the same DB

>>24375 This is awesome. I heard DB folder I thought it refers primarily to the files themselves and the tags/urls/notes applied to them.

>>24336 Thank you for reporting this. I can't carry over, since the first guy is just counting while the second guy gets richer info, but I will figure out some sort of deferred count or pause or whatever feels good and saves you time. >>24338 >>24339 >>24340 >>24342 Just to add here, if you fully deleted the PTR service under services->manage tags, and you don't see any PTR tab when you open 'manage tags', then these tags are not from the PTR. Tags won't bleed from the PTR to your like 'downloader tags' or 'my tags' etc... If you only paused the PTR sync and it still exists, then these tags could be from the PTR. If you open up 'manage tags' on one of these problem files, under which tag service tab are these tags listed? Is it 'my tags'? If you have local tags appearing on a file as soon as it is imported from hard drive, with no 'add these tags on import' stuff like sidecars or filename tagging going on, then there's two possibilities I can think of: 1) The file was in your client before, you tagged it then, and then deleted it, and then it was re-imported. Hydrus doesn't delete tags from deleted files, so all that survives a delete/re-import cycle. 2) You've done some funky tag migration stuff before, likely with a Hydrus Tag Archive or a PTR migration, using tags->migrate tags. If you set up a tag migration on 'all known files' using sha256 hashes, then hydrus can apply tags to files it hasn't imported yet. Let me know if you discover anything else here! If you hadn't thought of it, I think you can go ['system:number of tags > 0', '-creator:*'] to better find these files. >>24350 Thanks. Yeah, this sucks. I'll put some work in here. >>24354 This is an interesting question. I had intended for auto-resolution rules to have a clever 'examine pairs of lowest quality against the next highest quality first' so I could reduce some false negatives, but when I figured out the sort code for pairs, it just doesn't fit into the auto-resolution system, which I have tuned to have extremely low per-pair overhead. For now, it just reads straight off the db without sorting, and I think due to luck really, and probably legacy structures, that mostly means smaller ids first. We are seeing similar files next to each other because of similar import and tagging times. I think I agree with you--a completely random feed is more useful, and we should choose to do that. I'll have a think about it all. I still want my low overhead, but perhaps I can be clever about the initial insert or something.

>>24358 >>24370 I have some 'how to make a downloader' help here: https://hydrusnetwork.github.io/hydrus/downloader_intro.html It is a little old, but the basics are there. A web comic is tricky because it isn't like a normal booru or imageboard. You don't put a query in to search, and there's no 'gallery' result. TBH I think the KISS answer is you just drag and drop the URL of the web comic into hydrus every time you read it, assuming you check every mondays and fridays or whatever for the page yourself. If you want to set up an automatic thing, you'll want to teach hydrus how to find the newest URL, which means a 'gallery page'. I see your URL Class is in the form domain/date/blah, which is excellent. I also see we have this nice archive page: https://www.buttersafe.com/archive/ I think you should tell hydrus that that archive page is a 'gallery'. Write a parser to grab all the page links, and that lets hydrus queue up all the Post URLs. Hitting that gallery URL a week from now will also get the new URL, and that's how the subscription will want to work. The only problem, where we have to trick hydrus, is that this site doesn't have a 'query text'. You aren't typing 'sexy anime girl' into this site to get the Gallery URL, it just has the one. My downloader engine doesn't have a slot for this yet, so you should make a fake GUG. Check my pic related. You type 'archive' as the query, and the normal URL is created no worries. That should work in a gallery download and a subscription.

>>24359 >>24363 Yeah. Due to crashing and flickering issues, I recycle mpv windows, so when you go from one video to another in the media viewer, I grab an mpv window from the pool and send the old one back to it. If you go through a sleep cycle and an 'active' mpv window doesn't restore correctly because of some GPU driver issue or whatever, you can get this situation where that continually recycled mpv window is fucked in some way, and you are seeing it 'every other time'. That debug command basically seals away all existing mpv windows and generates a fresh pool. It can 'fix' the sleep-restore issue, but if you get a bad mpv window very regularly, during normal Linux work, there might be something else going on. I apologise for how much of a headache this is--my mpv solution is really held together by duct tape. I have a help->debug->report modes->mpv report mode. It'll spam a LOT of stuff to log. If you can turn that on as a broken mpv window is instantiated and then a couple times as you try to load a file to it, maybe we'll learn something. Most of these problems on Linux are solved by: 1) Editing the mpv.conf in your db dir. Usually disabling some audio driver or something that your OS has trouble with 2) Moving from a build version of Linux Hydrus to the source version. Source version nearly always runs mpv better: https://hydrusnetwork.github.io/hydrus/running_from_source.html Let me know what you discover! >>24373 Great, thank you for the feedback! I will make it more clear that's what I am testing. >>24376 Thanks I'll make a note in that section about the db holding everything.

>>24379 >Thanks I'll make a note in that section about the db holding everything. I'm not sure that's necessary, I'm just retarded.

I've got Hydrus working and my whole db copied over to Linux/CachyOS now. However two things. I hate the fucking Wayland icon that's replaced the Hydrus icon and I hear Wayland is fucking gay. And I can't fucking get libmpv1/libmpv2 to get the media player working for files with duration. Instructions say apt install libmpv1. Firstly, every time I do this it want to also open some huge help menu. Secondly, closing the help menu and choosing install returns No manual entry for libmpv1. I already have mpv player itself installed, but the instructions say that that is different from having these libraries installed.

>>24383 using bazzite and linux for the first time and I actually have the exact same issue and was looking for what to do

>>24384 I hear bazzite is even more of a pain to install things onto due to being immutable, so good luck. I chose CachyOS over it for that exact reason.

>>24383 >>24384 okay after looking around and at the past thread I found this >>23486 anon's bit about dropping [code]DISPLAY=:0 WAYLAND_DISPLAY=[/cod] into my launch script to do the trick right away also nipped the issue where panning images was massively over sensitive didn't really know what wayland was and kept glossing over the warning in the docs (the first suggestion there didn't work either) thank you anon

>>24386 Yeah, it's been a bit tricky, especially given so many guides and info for linux out there assume you can just go into the terminal and dl shit no problem I installed hydrus from source via a distrobox and things seem to be going well, now that the mpv issue is done with.

>>24387 How do I modify the launch script? And this seems to be about fixing some other display issues? Will this really also fix Hydrus not loading mpv? >>24388 I just extracted the tar file for 640, the same version I had on Windows, before micgrating. I don't know jack about running from source yet.

>>24389 Oh, I also tried following the "run from source instructions", but sudo apt install python3-pip doesn't work. It just returns sudo: apt: command not found.

>>24390 Figured it out. Hydevs instructions probably only work for whatever distro he's on. For CachyOS, the command is sudo pacman -Syu python-pip and probably a similar format change for the venv install as well.

>>24391 Oh wait, Python 3.3+ already has venv setup.

So I managed to install from source, jumping only two versions from 640 to 642. I couldn't figure out how to get libmpv1, but instead I cloned and installed the libmpv.git package from the AUR. This wanted to overwrite a file used for the mpv player, so I let it, breaking mpv. Not even sure this was necessary, but I have other players I can used outside of Hydrus. I ran it, and things immediately got really funky as soon as I try to touch anything animated. If I click something animated, instead of displaying it in the preview window, it immediately opens a new media viewer window and plays. It cannot be paused and there is no UI for this window at all. Once a video window is open, the main Hydrus window suffers severe UI glitches, cloning random parts of the UI in various places, and cannot be used properly. Attemping to close the video window will throw an error notification, >the file with has "a bunch of fucking characters" seems to have failed to load in mpv. In order to preserve program stability, I have unloaded it immediately. This freezes the video, but does not close the window, and at this point shutting Hydrus down is the only option. Something neat I noticed about CachyOS though. When I copied my 200GB+ db folder into the source hydrus folder, it didn't waste another 200GB, but instead has the new folder just also point to the same files where ever they actually are in the system. Wangblows would have just needlessly created another 200GBs of files. Pretty cool. I can open either the 640 install or or the 642 source install and they both find all my files just fine.

>>24387 >>23486 (Cross-thread) I figured it out. Thanks a ton. I ran DISPLAY=:0 WAYLAND_DISPLAY= ./hydrus_client.sh and it just werks now. Other things I tried >Opening the .sh file in Kate to try and paste DISPLAY=:0 WAYLAND_DISPLAY=, but it felt like a bad idea so I didn't >/hydrus_client.sh DISPLAY=:0 WAYLAND_DISPLAY= >DISPLAY=:0 WAYLAND_DISPLAY=/hydrus_client.sh Now I just need a way to modify the launch like anon said so I don't have to do this in command line every time.

>>24395 It's also blazingly fast after the first time loading thumbs. Instant playing and seeking of even my largest 4GB videos.

>>24395 >Now I just need a way to modify the launch like anon said so I don't have to do this in command line every time. Got it. I even found the hydrus.ico in /static and made the shortcut look nice. >Right Click Desktop >Create New... >Link to Application >Application >Name: Hydrus Client >Enviroment variables: DISPLAY=:0 WAYLAND_DISPLAY= >OK At first I tried pasting DISPLAY=:0 WAYLAND_DISPLAY= into Arguments:, but that just broke it the shortcut.

I have run into a workflow effecting issue in CachyOS with Hydrus 642 from source launched with the DISPLAY=:0 WAYLAND_DISPLAY= enviroment variable. When I press F3 to edit tags while in the media viewer, and then press it again to exit the tag manager, I can no longer enter the tag manager without clicking on the media viewer. It loses focus on the media viewer window, probably as soon as I open the tag manager, but doesn't return focus to the media viewer window upon closing the tag manager like it used to. I could just not close the tag manager as I scroll through the media since tags are automatically applied here, but it is a bit annoying. Normally I'd close the tag manager and hit F7 to archive a file that I then thought was mostly tag-complete. It also means I can't navigate between files in the media viewer with the arrow keys after I finish tagging something, just the mouse scroll wheel since it doesn't care what window is in focus.

>>24398 The same issue applies to the thumbnail viewer. Opene the tag manager, and then closing it, does not return focus to the window, and I have to click on Hydrus again. This makes it a real pain to do group tagging.

>>24386 >>24384 The Hydrus flatpak works just fine imo. You might need to give it broader permissions to put your db elsewhere but that's it.

>>24400 I tried the flatpak but it's outdated by a few versions and really just wanted the capability of building from source myself I'm good now though it seems >>24398 Hmm, the DISPLAY=:0 WAYLAND_DISPLAY= doesn't affect my tag dialog that way. Also appreciate the mention about the shortcut, I wasn't too worried about it since I could get the hydrus icon to display on the toolbar by choosing "q" in the setup venv for Qt but I couldn't change the icon for the startup shell script

>>24401 >Hmm, the DISPLAY=:0 WAYLAND_DISPLAY= doesn't affect my tag dialog that way. I'm not trying to say it is, I was just giving full information about my setup.

>>24402 oh okay, I just wanted to chime if if you were thinking that might've been at play>>24402

>>24401 Oh, and the desktop is KDE Plasma, since I'm guessing that's in charge of window focus and how it interacts with Hydrus is the issue.

>>24363 >>24379 Thanks both of you, I'll check it and report back when i'm back on linux, I booted into windows for the weekend because of gaming, ofc hydrus works perfectly here. really makes me think why I bother with linux, stockholm syndrome at this point

Is there a way to get around cloudflare checks?

Hello, I was trying to do stuff with parsers... Anyone know how to implement a "child post" uh parser? So when it goes through the gallery pages, gets the posts and subsequently gets the url to the gallery page with additional posts linked to the "parent post" it adds those to the links to be downloaded?

>>24359 >>24363 Encountering this as well now, from a fresh startup too. It was working just fine yesterday. The debug command fixed it. Thanks.

Would you mind adding a sort of "conditional" template syntax to the export file name patterns? By that I mean, a way to have something be added to the filename of an exported file, but if it's not present (so the placeholder ends up just being nothing) then you add something else? Specifically in this case, what I want to do is add a file's filename tag if it has one, but if not then its title tag, and if not that either, than its Hydrus ID. afaik there isn't currently a way to do this, so currently if a file doesn't have a title tag, it just ends up having the filename `.png` or something like that.

Now that I'm on CachyOS, the "set child" and "set parent" boxes in the the tag parent manager window are both super wide and take up more space than I have, forcing a horizontal scroll bar for that window. Is there a way to shrink them? Nothing in the GUI settings seems quite right to me.

>>24393 >When I copied my 200GB+ db folder into the source hydrus folder, it didn't waste another 200GB Not to harsh your mellow but that's not just a Cachy thing, almost any FS that Linux uses can do that, either with symlinks or Copy-On-Write (COW). FS's are just way more mature on Linux compared to Winblows. It may be a Cachy thing to do it through the GUI though.

>>24414 >Not to harsh your mellow Why would that harsh my mellow? This is still amazing and way better than Windows. Even more so it drives home how shit Windows is if this is mostly standard for Linux distros.

>>24415 Really it's more so I don't have to say >uhm, ackhtually but also some people (4/g/ users) are kind of protective about their distro choice, me included.

>>24416 I only just switched. Why can't Linux autists just be united in their hatred of Mac and Windows?

>>24417 All Linux autists are, it's kind of like the Balkans, where we fight amongst ourselves unless a bigger threat shows up.

>>24378 Suggestion: Reverse the order of the downloader tutorial. When you start with a GUG, it can't really do anything until you have the post parser & url class. If you start by making the url class & post parser, you immediately have something useful where you can paste URLs into a downloader or even watch your clipboard. Useful on many sites even those that have no gallery.

>>24417 >Why can't Linux autists just be united in their hatred of Mac and Windows? No comments.

In CachyOS with KDE Plasma running Hydrus 642 from source launched with the DISPLAY=:0 WAYLAND_DISPLAY= enviroment variable. Drag and drop seemed to work fine at first, but now it's almost totally inconsistent whether it decide to properly drag and drop files into my browser, Ungoogled Chromium. If I manually export the files to a folder first, then drag and drop behavior seems inconsistent as well, but I can force it to work by Ctrl+drag and dropping. However, I can't do this in Hydrus, because Ctrl+clicking has the function of selecting/deselecting whatever I click. Is there a solution?

>>24421 Also, I have tried checking and unchecking the "move flag bugfix" option under file->options->exporting->drag and drop and it resulted in no change to drag and drop behavior.

>>24421 >>24422 I found a setting in KDE plasma that changes the behavior of drag and drop. Changing it to the latter option mimics standard Windows behavior somewhat, and makes it so I don't have to Ctrl+Drag to consistently drag files from a folder to my browser. However, it has had no effect on dragging files from Hydrus, both with and without the move flag bugfix.

>>24398 >>24399 I've found some settings in KDE Plasma that effect Window focus. I fiddled around with "Focus stealing prevention" a bit which led to some weird behavior, and then noticed the setting, >Multi-screen behavior: Seperate screen focus It didn't hit me at first, but I have my tag manager and main Hydrus window on different screens so I can have plenty of space to enter and read the tags for file(s) while actually looking at the files to make sure those tags are correct. Unchecking this option fixed things immediately. Now I just need drag and dropping to browsers working again.

I had a good week. I cleared some more duplicates work, fixed some file import crashes, finished the new smoother and more stable mpv interface, and added animated JpegXL support. The release should be as normal tomorrow.

Is anyone else using the experimental Qt Media Player as the video player? Are there meant to be media controls overlaid over the video or is it normal that it's the video itself and no seek bar? I'm on Wayland so mpv doesn't work great (it does play, but with significant caveats that make it infeasible) but Qt Media Player actually works pretty well and doesn't seem to be as crashy as the description suggests, at least on my system.

>>24356 >new mpv thing I unchecked the box, restarted, and everything ran just fine. Videos in the preview window and in the media viewer will run and loop fine by themselves. I can click through the seekbar, pause, and unpause. I can use hotkeys to seek, pause, and unpause. All good! let me know if there is anything more specific you'd like me to test. I did see: `v642, 2025-10-15 15:53:27: [MPV error] main: Too many events queued.` a couple times, but no spam.

https://www.youtube.com/watch?v=NZRUQE1AnWQ windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v643/Hydrus.Network.643.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v643/Hydrus.Network.643.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v643/Hydrus.Network.643.-.Linux.-.Executable.tar.zst I had a good week. There's some duplicates and mpv stuff, and animated jxls. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html future build Only for advanced users! I am making another future build this week. This is a special build with new libraries that I would like advanced users to test out so I know they are safe to fold into the normal release. More info in the post here: https://github.com/hydrusnetwork/hydrus/releases/tag/v643-future-01 highlights When you search for duplicate pairs, the count label now gives an estimate once it hits 1,000 matches, rather than counting exactly all the way to the end. Let me know how this feels with real world data. The suggested rules in duplicates auto-resolution get another couple tweaks, and I added an "AND' comparator for clever OR blocks. The new mpv interface is polished and switched on for everyone. When you scrub through a video, it should now be a bit smoother. Crashes and other problems should be rarer. If you suddenly can't play anything with mpv, hit up options->media playback->LEGACY DEBUG: Use legacy mpv communication method to go back to the old way. Hydrus now supports animated JpegXLs! I hacked it together with ffmpeg, so it isn't super beautiful and it can't do variable framerate, but it works. I'll keep checking in future for native library support (and mpv support--it was pretty log-spammy when I tried it) so we can make this nicer. The 'manage tags' dialog now has a 'deleted mappings' label and show/hide button that appears when the list above actually has some. I've always had awful support for undoing deleted tags (previously you had to dive into the cog menu button, and it wouldn't remember between dialogs), so we'll see how this goes. Some PDF/SVG import crashes are fixed. next week I want to add transparency support to my visual duplicates test.

>>24437 running the prebuilt linux one on bazzite 42 it boots gets this upon about->help opening files with audio don't seem to have audio playable, file still plays though and the video works, no crashing weirdly enough. I assume this isn't mpv being used.

>>24439 MPV not found. Also I see your install directory is in VAR, kinda you are running the Flatpak version instead of the Source one that works every time.

>>24441 that was the future build i downloaded straight from the github, no flatpak involved my main client is from source as well

>>24442 I'm not devanon but you may want to try the source version. If your system has already installed the libmpv library, and/or the MPV player software, the source package must work. https://github.com/hydrusnetwork/hydrus/releases/tag/v643-future-01

>>24443 good idea having built the venv it booted and didn't scream anything upon about opening Opening a video file caused ui errors, the file played but couldn't paused and only closed, attempting to reopen the file caused a crash (not sure where the crash file is tbh)

tested and upgraded to v643-future-01. Win Server 2022, no issues.

>>24383 I use Fedora. #1 make sure you're running from the "source" build. It's not really source it's binary. The "Linux binary" build is for Ubuntu only and breaks when distros put libraries in different locations. You will need mpv and mpv-libs (or whatever the mpv library package is called) >>24393 Wow CachyOS has built in deduplication? That's cool. ZFS filesystem? >>24411 >>24398 This is probably related to your distro and breakage between the specific packages that make up Arch. At this point I think you're reporting CachyOS/Arch issues more than Hydrus ones. That's why I don't use Arch :P >>24421 >>24425 Wayland in general is going to be buggy. I'm going to assume it's a CachyOS/Arch issue. Please try a more stable distro like Debian or Fedora. I've had horrible experiences with Arch's instability.

>>24417 The UNIX philosophy. Every part of the OS is an independent module. Anyone can build their own custom distro using the GNU utilities and Linux kernel. There's so many choices it's hard to pick one unified way to do it. Another factor is licensing. GPLv3 and MIT. GPLv3 is like HIV. If you use any GPLv3 code in your project, and the WHOLE project has to be licensed the same. This heavily discourages commercial advancement of the software. IBM does contribute a lot with Red Hat, but it's mostly highly technical background stuff for server use only. Now the MIT license. It lets you take the code and sell it commercially without releasing the source code. Now you have the opposite problem. Look up "Intel Minix." The most deployed operating system in the world, because it's running on a second CPU on all Intel chips and is used for deploying malware for governments (through intel AMT/ME). How much money did the MiniX OS dev get? ZERO DOLLARS! Mac OS uses older versions of Linux utilities that are MIT licensed. Now apply this to the shitpiles that are Linux githubs. I've made 100% factual, basic, recommendations to get feature parity with things like Max OS. 100% indisputable objective improvements for the user experience IN LINE WITH THEIR DESIGN PHILOSOPHY. Then the response is some canned bullshit like >*trans flag* uhhh a problem of open source is having options so it's bad to improve things in line with our minimalist design philosophy >*trans flag* uhhh people might not like that >*trans flag* uhhh it's a free project so you can't complain that I use it to feel superior! Then there's the problem of Indians and other shitty devs adding to repositories just to add shitty code so they can add projects to their resume. You can see it in Windows 11. FUCKING FILE SEARCH IS BROKEN. FILE GROUPING SUCKS ASS. START MENU SPIKES CPU USAGE. START MENU SEARCH HAS BEEN BROKEN FOR OVER A DECADE. Want to see the opposite of this lib-left anarchist shitpile that is Linux? Look at Hydev! Dev-sama! A single person making an exceptionally customizable and advanced piece of software with minimal help. A true gem who should actually be a professional corporate software project manager. /rant sorry for the wall of text

>>24451 >I use Fedora. #1 make sure you're running from the "source" build. It's not really source it's binary. The "Linux binary" build is for Ubuntu only and breaks when distros put libraries in different locations. >You will need mpv and mpv-libs (or whatever the mpv library package is called) Sorry anon, my issue was already resolved. >>24395 >>24397 >>24451 >ZFS filesystem? BTRFS >>24451 >This is probably related to your distro and breakage between the specific packages that make up Arch. At this point I think you're reporting CachyOS/Arch issues more than Hydrus ones. That's why I don't use Arch :P The issue with focusing was already resolved. Just a minor tweak to KDE Plasma focus rules. >>24426 >>24451 >Please try a more stable distro like Debian or Fedora. That sounds like a ton of fucking work, and also CachyOS is allegedly optimized for games. I've already gotten 5 different console emulators working on it. >>24452 >Another factor is licensing. GPLv3 and MIT. GPLv3 is like HIV. If you use any GPLv3 code in your project, and the WHOLE project has to be licensed the same. This heavily discourages commercial advancement of the software. IBM does contribute a lot with Red Hat, but it's mostly highly technical background stuff for server use only. >Now the MIT license. It lets you take the code and sell it commercially without releasing the source code. Now you have the opposite problem. Look up "Intel Minix." The most deployed operating system in the world, because it's running on a second CPU on all Intel chips and is used for deploying malware for governments (through intel AMT/ME). How much money did the MiniX OS dev get? ZERO DOLLARS! Mac OS uses older versions of Linux utilities that are MIT licensed. So you have one where absolutely no one can make money off the code you've made, and one where only other people can make money off the code you made? Why is there no in-between? Why can't I license something such that >My code is free and free to use for other free projects, because it's useful >But if you decide to sell anything using that code, you have to give me a cut as you're profiting off my work

>>24458 >Why can't I license something such that You can do anything you want if you write your own license, it's just more work because your custom license isn't battle tested and you are going to have to spend time in court if someone decides to ignore it's terms (potentially to no avail, if your brand of simon says isn't legally enforcable). Also nothing actually prevents you from selling/making money from GPL stuff, there are multiple games that do so even without exploiting the loopholes at all. (Said loopholes are also why the actual 'fuck corpos' license is AGPL, btw.) There is also the fact that as the creator you can hand out sub-licenses to whomever whenever you want so whatever license you put on there is only the default.

>>24458 > So you have one where absolutely no one can make money off the code you've made, and one where only other people can make money off the code you made? You can absolutely make money off GPL software, it's just that companies are so greedy that money is not enough for them. They also want you to be dependent so when they cut off support your software or hardware becomes trash and you have to buy a new one. They want you to own nothing and be happy. Let's say I want to make some application using a GPL library. Then my entire application becomes GPL. So what? I can still charge money for it. Now if I sell it to Bob and Bob wants the source code I can ask him for the receipt because people to whom I have given the software are entitled to the source code. If Alice comes to me asking for the source code and she is not my customer I can tell her to fuck off. Could Bob make money re-selling my software? Sure, but why would anyone but it from some other guy who does not understand the code instead of from me where they can get actual support? Yeah, Bob could give away the software for free, but that's no different from software piracy which is already happening, so what is the loss? The only loss is that I cannot rip Bob off. I cannot tell him "buy the upgrade, goy" because if he wanted to he could repair the software himself to be secure and run on modern systems. Is he going to do that? Maybe, maybe not. But just the fact that he could if he wanted to is already too much for these greedy pieces of shit. Doom is a great example of commercial GPL software. The game is still being sold long after id lost interest in it, and it's still making money. If anything, id was profiting off free work done by the community to keep up interest in that old game. >Why is there no in-between? Why can't I license something such that >>My code is free and free to use for other free projects, because it's useful >>But if you decide to sell anything using that code, you have to give me a cut as you're profiting off my work Such a license would be non-Free because it goes against Freedom Zero (to run the software for any purpose) and non-Open Source because it discriminates against fields of endeavor. The problem is that it is maintainer lock-in. If you get bored of the software and no longer want to maintain it no one else can. I would not touch this sort of shit even for free. It's the same greedy crap.

Now that I've been using Linux/CachyOS for a bit, I've noticed Hydrus can be pretty slow. I thought it might just take a while for some things the first time, because it was a fresh install on a new OS or something and some background work might need doing, but the slowness is persisting. Initialising on startup always took a few seconds before on Windows, but it takes much longer now and running a new search takes significantly longer. I'm constantly getting randomized one-to-several-seconds of hang time doing anything but scrolling through small files in the media viewer, though half the time it's still instant. Tag auto-complete, hitting enter to add a tag, opening the tag manager, closing the manager, all hanging very often. It's even worse if I'm concurrently doing something like a large file transfer elsewhere, but whether or not my computer is multi-tasking, my resource use seems pretty low. I tried closing all other applications save for System monitor. I had Audacious that wasn't currently playing anything, a text file open in Kate, and 4 tabs in Ungoogled Chromium. All this cut about 2 gigs off my total memeory usage. Once I did this, Hydrus ran a little better, but there was still barely a bit of hanging. My CPU usage didn't break 15% and it only went over 5% when doing intense scrolling of thumbnails, GPU didn't break 45% and only for the largest videos did it go over 35%, Memory didn't break 6GB, including other brackground processes. Having opened everything again, the general hanging seems to have remained barely noticeable, so it may just be random. I tried loading in 1000 files with duratoin sorted from largest to smallest, both with and without other programs active. Even though it took a while, I saw no spike in any resource usage. All of this was much faster in Windows, which surprises me given I expect Wangblows to have much more unneccesary overhead. Is Hyrdus's use of resources being limited by the default settings of CachyOS or something? Can I allow it to use more resources?

I can't download from Kemono. I duplicated all the existing downloader components from kemono.su and replaced everything with kemono.cr, and the URLs now get recognized by Hydrus. I sent cookies from kemono.cr to Hydrus as well, but I still get 403 errors. Has anyone been able to get Kemono working since they switched domains?

>>24391 I'm no expert at Linux, but if you were able to get this going with pacman rather than apt, can you tell me the lines you had to go through, other than this, to get going? I'd love to update the source help to have non-apt stuff, but I don't know what I am talking about. >>24395 >>24397 If you look inside hydrus_client.sh with a text editor, it talks about a 'hydrus_client-user.sh' you can make for custom script stuff. Github won't touch that filename, so feel free to dupe and rename the script and add your envs and stuff inside, whatever you need. >>24398 >>24399 >>24426 That's super interesting that the OS has a setting for it. I do have a bunch of hacks that try to force this focus stuff to work as you want, since different Window managers do screw around with this behaviour, but I hadn't heard of screen-separate focus stacks before. >>24406 Change your IP sometimes works, otherwise check out Hydrus Companion. Get into the site with your browser, then copy user-agent and cookies across to hydrus. https://gitgud.io/prkc/hydrus-companion Otherwise look at gallery-dl or hydownloader (which uses gallery-dl) for difficult sites: https://gitgud.io/thatfuckingbird/hydownloader >>24408 I'm not sure of the exact shape of what you are describing here, but you generally don't have to do anything too clever to get that stuff to work. Check out the Pixiv parser for an example, but your gallery parser should probably produce a list of Post URLs. If those Post URLs produce multiple URLs, then those will be queued up as child URLs. If those child URLs are recognised by URL Class as Post URLs, they'll get parsed, and if they have no URL Class or are explicitly File URLs, then they'll be downloaded directly. Make sure everything has an URL Class, and unless you are being very clever, try to make your initial Gallery URL parse produce Post URLs (i.e. don't try to make a Gallery URL produce a Gallery URL other than a 'next page URL'). If a Post URL can produce multiple sub-Post URLs or File URLs, make sure its URL Class says 'can produce multiple files'. Any URL will get its parent (i.e. the guy that created it) as its referral URL, and it inherits any tags and stuff parsed by the parent too. If you have complicated metadata parsing, then once you are up and running, hit the 'file log' and right-click stuff to see what tags were parsed by each level.

>>24475 >I'm no expert at Linux, but if you were able to get this going with pacman rather than apt, can you tell me the lines you had to go through, other than this, to get going? I'd love to update the source help to have non-apt stuff, but I don't know what I am talking about. I don't know what I'm talking about either. I'm running around in the dark. Every issue I've encountered and resolved that I can actually recall, I've certainly posted in this thread already.

>>24410 Tell me about it. I hate how the export patterns work and I totally agree. If you look at the 'tag banner' stuff under options->tag presentation, that was the tech I originally planned to replace this, but I kind of fucked it up and it is awkward to work with. I wanted some logic for stuff like 'if there are parent tags, take the top tag parent of the same namespace' and stuff too to collapse some spam. It has been on the back burner for a few years. I just rewrote all the actual raw 'generate a filename part' here in the past few months, so maybe I'm a bit freer to throw weird stuff at filenames now, since I know slashes or quote marks or simple character overruns will be dealt with. I'll keep this in mind and have a think. Maybe the answer is to have an object that can work in a simple mode if you just want to type out '{hash}', but also a more complicated mode that allows for conditional stuff. >>24411 Try right-clicking the multi-column list header. There should be something like 'reset settings for (id name)'. Otherwise, you are basically looking at scrolling the sub-panel all the way to the right so you can see the last column's rightmost caret and dragging it back to a normal size. Sometimes this shit goes bananas (because I am dealing with some stupid legacy code I wrote years ago) and the initial or some recalculated width somewhere is bad. Let me know if you run into a dialog/list whose width you fix once but then it reliably breaks again (or grows five pixels every open, can also happen). >>24419 Thanks, good idea. I'll have a look at it. >>24421 >>24422 Yeah this option is the best shot from my end. Let me know if you try it and it doesn't work. Drag and Drop problems in OSes tend to be permission related, and I bet Wayland and/or Flatpak, which are very careful about UI permissions, are going to be particularly touchy. In Windows it tends to be DnDs between programs where one is admin mode and one isn't. In many modern programs the permissions issues are more granular, and they are worried about a move flag vs a copy flag, which I guess is what you are seeing with the ctrl+DnD working while the bare one doesn't. Hydrus sets the copy flag, and that debug option sets a move flag, which some older versions of discord needed for reasons I do not understand and were probably incidental. Try the 'copy to temp folder' setting there too. If it fails the 'it sucks' solution is to drag and drop to a folder on your desktop and then from there to the destination. Let me know if you discover anything and would like me to add any debug mode for the hydrus side of stuff. >>24435 I could never fix the issue with the seek bar. The widget does exist, but the Qt Media Player initialises some OpenGL window, I think, that is 'always on top' and it draws over my controls. I was mostly putting work off on this until reworked my whole media player layout system, which is still completely fucked atm, so I can make a 'put the seekbar below the video' mode. I'm sorry to say the Qt Media Player is a huge pain in the ass to work with, or at least it was. My overall plan has been to figure out a secondary, safer mpv rendering solution, but I'm somewhat open to revisiting the Qt Media Player. I'll try and keep this in mind when I eventually get to the media viewer layout overhaul, but please remind me at that time if I forgot. >>24436 Thanks, I saw this earlier. Good stuff. Yeah, I got the 'too many events queued' thing and in v643 I have reduced those. You have to spam the seekbar pretty hard on a heavy video for like twelve seconds to get it to pop up now. Let me know how things go for you in future! >>24439 >>24441 >>24444 Damn, that's a shame. It worked ok on my Mint. It is good that the connection can be made when you run from source. I'm encouraging all Linux users to run from source if they can, since anything too not-like-Ubuntu just won't load the more complicated packaged libs like mpv. The 'mpv goes crazy' and the nested menubar are classic symptoms of hydrus in Wayland. Do you think you are running in Wayland (rather than X11)? If so, check the Wayland stuff here under the Linux tab: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#installing Open the hydrus_client.sh script to see how to make your own 'hydrus_client-user.sh' copy, and add the environment variables that help suggests. Is everything suddenly fixed? Check this thread and the previous one for more users grappling with this, and let me know how you get on! If you want to capture a crash, hit up help->debug->debug modes->use faulthandler to log crashes and then try to trigger the crash. A bunch of thread data will be written to your 'client - date.log' file in your install_dir/db directory.

>>24447 Great, thanks! >>24472 options->speed and memory is the main place to handle this stuff, but if you are getting seconds of actual non-responsiveness during normal browsing, that doesn't sound like a cache issue. My code still sucks really bad in some places, and if your Linux is enforcing some policy somewhere, maybe that is expressing itself in a way that Windows is unwisely/wisely skipping over. Please check out help->debug->profiling and pastebin or email me some profiles of your slow stuff, and I'll have a look. I actually got two sets this week of some slow big sessions (big downloader pages, looks like), so watch for v644 too. >>24476 Ok, no worries. I'll do some research and update that help to talk about non-apt stuff too. The tone I want to aim for is 'you need python and venv mate, it doesn't matter how'. I've been using Linux on and off for decades but switched to a Mint desktop this year. It sounds dumb, but ChatGPT has been invaluable. There's a billion Stack Exchange threads about some weird conf file you need to get your USB thing to mount or your keyboard layout to work, and it has read all of them. The shit that was driving me nuts in previous attempts at a Linux desktop was now doable because I can just ask ChatGPT and it'll generally give me a decent idea of what conf I should be looking at or how to interpret some log.

>>24477 >Try right-clicking the multi-column list header. There should be something like 'reset settings for (id name) That fixed it! Thanks. I didn't even notice those were super wide for some reason because the "notes" column was out of view when scrolled to the left. I'm guessing I exteneded them some time ago for longer tags, but it was before the notes column was added? Or just some inexplicable jank caused by all the switching over I've been doing. >Yeah this option is the best shot from my end. Let me know if you try it and it doesn't work. >Try the 'copy to temp folder' setting there too. Was doing that by default, as it's the only way to retain filenames when exporting. Also, the move flag bugfix cannot be enabled without this being enabled first as well. I'm having issues with regular drag and drop even from my file browser to my internet browser, and it seems to be an unresolved issue with KDE. https://archive.is/55mQX https://bugs.kde.org/show_bug.cgi?id=484018 Given that, there's probably little to be done about it until someone works on fixing KDE itself. >options->speed and memory is the main place to handle this stuff I found a couple things to raise that might help in other area since I have the memory to spare, but nothing that would effect where I've experienced slowdowns. At least with this I can sacrifice some memory in order to seamlessly scroll through my few super huge alternates that are around 8000x5000. >Please check out help->debug->profiling and pastebin or email me some profiles I noticed there's a popup that counts how many fast and slow jobs there were. Set it to db profile and managed to get 6 slow jobs and 15 fast ones iirc by loading a page with 1000 of my largest files withouth duration. Since I had already done files with duration and didn't want cacheing to muddy the waters. Took like, 15-20 seconds I think. I've emailed the log. I'll try to remember to turn on profiling if I see the other hanging issue. This one is less of an issue, but things like the tag manager and tag autocomplete hanging even slightly throw me way off when I'm processing files quickly. >ChatGPT I can't even get a local AI chat model to consistnently speak in second person. It baffles me that this is somehow consistent enough to reliably learn about code when the text generator literally doesn't understand whether something it says is true or not.

>>24478 >>24479 Oh yeah, that log is still on Hydrus 642. I can update if necessary. Updating from source isn't too much different, right?

>>24480 (You) Oh thank God, it was easy. Even faster than regular updating by running another .exe. I'm starting to get used to basic things being a headache on Linux/CachyOS.

>>24478 >>24479 Managed to catch it acting up again while doing some manual tagging. Turned on db profiling for a minute. 98 slow jobs, 213 fast jobs. Will email it soon.

>>24452 This anon knows his craft. R

When I use manage tag siblings, any way to make it also delete the replace "wrong/old" tags from all my images and automatically replace it with the "correct" one whenever I download an image?

I had a great week. I cleaned a bunch of old code, figured out transparency comparison in the 'visual duplicates' test, improved transparency detection for all files, and optimised some file and UI work for very busy cilents. The release should be as normal tomorrow.

Yo! I'm fairly new to all of this. I recently got a system set up with Hydrus (database), Hydownloader (for downloading), and Hatate (for checking the thousands of files I already have downloaded over the years). I was wondering does Hydownloader not work for Twitter? I saw some threads from 2023 say it doesn't work but I just wanted to check if that ever changed. I was able to get WFdownloader working for twitter but WF rapes the file size and I like having meta data. Oh another thing. Can you add the metadata side bars for more websites? They're useful for me as an artist but they don't work for every site (most notably instagram) I'm fairly new to all of this. Is this the best place to talk about Hydrus?

Can someone else test the gelbooru downloader? It seems to not download "all site content." I made an account, login successful, "display all site content" is saved in accoutn settings, and the search still only returns 2 results. I'm searching zhou135627 for reference. >>24458 >So you have one where absolutely no one can make money off the code you've made, and one where only other people can make money off the code you made? Why is there no in-between? You're touching on the difference between "open source" and "free" as in freedom software. You can resell GPL software, but what's the point if I can download it from github and compile it? Why would I pay you? You can have open source, "non-free" software, but that's hard to enforce since the code is sitting there for anyone to use. Especially if they use it in a proprietary program, how are you going to prove they stole your code?

>>24497 Did you send your cookies/http-headers to hydrus? Gelbooru stores that setting in one of those(should be the headers iirc) so you don't even need an account for it and it should still work if hydrus has them.

>>>24496 I'm not hydev or the hydownloder dev, the programs work together but they aren't related, just in case you were unaware. Not trying to sound mean or anything just ensuring you know the difference. I also use hydownloader and it works for twitter for me. I'm not sure what I did, but I think I probably sent my cookies over through hydrus companion. What do mean by metadata side bars? If you mean pic1 you can enable them in network>downloaders>manage downloader & url display>media viewer urls. I'd say this is probably the best place, hydrus is mentioned on 4/g/ occasionally but it's usually not the thread topic. This place is actually consistent which helps a lot. >>24497 I'm getting 49 items with that search. Are you logged in through hydrus' login tool? Also most people are too lazy to compile, even though it's usually not hard, occasionally it can be a real bitch though so I can also see wanting to skip the hassle. Also also, if code reacts in a specific manner you can prove it is yours, apparently. Lawsuits have been won by the FSF after all.

>>24502 >>24497 I am illterally literate, it says right there in your post login successful. I guess try what the other anon said.

>>24496 No one has shared an updated 'downloader' for Instagram since 2020 afaik. If you have any requests for sites besides instagram I might give it a try. Just give me some example URLs. Instagram requires a Zuck account to access, right? Not too keen on stuff like that personally. The downloaders that add automatic metadata to your files can be created/imported without updating hydrus. Someone with a bit of html/regex skill just needs to make one for that site. When the site gets a new layout, often the downloader needs to be updated, which is probably why the Instagram downloader is broken.

https://www.youtube.com/watch?v=WGkBMPgt8oc windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v644/Hydrus.Network.644.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v644/Hydrus.Network.644.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v644/Hydrus.Network.644.-.Linux.-.Executable.tar.zst I had a great week. There's transparency stuff, duplicates stuff, and busy client stuff. On update, you will probably get a popup about 'Hey, you have 12,345 files with transparency, do you want to recheck them?'. I recommend saying yes. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html new libraries I am folding the updates from last week's future build into today's release. There do not seem to be any special update instructions needed, but the Linux build is moved from Ubuntu 22.04 to 24.04, and it seems to have broken some users' mpv. If you are a Linux user and mpv stops working for you today, I recommend you move to running from source: https://hydrusnetwork.github.io/hydrus/running_from_source.html Thanks to a user, the Docker package is updated from Alpine 3.19 to 3.22 and has a richer VNC library. transparency I recalibrated how hydrus interprets transparency this week. There are many files that have a handful of random 98% opaque pixels due to encoding or brush artifacts. This is invisible to the human eye, so Hydrus no longer tries to render it or give such files the 'has transparency' property. On update, you'll probably get a popup asking if you want to rescan your existing transparency-having files--I recommend saying yes. The 'visual duplicates' test in the auto-resolution system now supports pairs where both have transparency. The test isn't supposed to be too strict, so let me know if you see any new '(transparency does not match)' false negatives. duplicates The new duplicates count estimate tech is more precise. The duplicates count in duplicate pages will refresh more often, basically on any change. (I think it'll be annoying, so I'm going to work on making it respond faster to smaller updates) I tweaked the suggested rules a bit more and added 'pixel-perfect gifs vs pngs'. other stuff Clients that have very heavy sessions, including lots of importing, should, I hope, be less laggy this week. I optimised a bunch of file and UI update calls. Let me know how it feels, even if it seems the same. The window that pops up when you drop files on the client has been overhauled. It works smoother and now shows rows for files that failed to parse. Ugoiras now have the little 'animated' icon on their thumbs, even if their duration is synthetic. next week I want to improve some potential duplicates db tech. It doesn't yet nicely delist pairs when one of the files is deleted.

>>24506 >Clients that have very heavy sessions, including lots of importing, should, I hope, be less laggy this week BASED, thank you hydev for enabling my hoarding habit.

>>24506 >Ugoiras now have the little 'animated' icon on their thumbs, even if their duration is synthetic. Holyshit, Hydrus plays ugoiras files? I've been going out of my way to convert them.

>>24502 I was having trouble understanding how to make hycompanion and hydownloader work together. It said somewhere I have to edit something but I could never find the thing i had to edit.>pic1 Yeah that's exactly what I was asking about. I didn't know they were called "media viewer urls" I checked the menu and there isn't one for instagram >>24505 I don't care about using a downloader for Instagram. Hydownloader does the site well enough for me (second pic rel. It gets images and metadata but doesn't import it correctly). I was asking if there could be a media viewer url for instagram here (first pic rel).

>>24502 Update. >I also use hydownloader and it works for twitter for me. I'm not sure what I did, but I think I probably sent my cookies over through hydrus companion. Thank you. This worked for me. I figured out how to do it. Some of them have multiple "twitter tweet" media things. I'll have to fix that. Another tangential thing. This happens with every downloader (WF downloader and Hydrus) I've tried. The site (Twitter in this instance) always lists more posts than I get downloaded. >165 vs 135 Am I doing something wrong?

>>24509 I think all I had to do for hydownloader was add these two things to hydrus-companion. It's right under the hydrus api access stuff. You can find it in hydownloader's data directory within the file "hydownloader-config.json".

>>24509 For instagram, you need the url mapping, even if hydrus doesn't download instagrams it needs a way to "label" the urls, for lack of a more technical term. So if you have the instagram downloader from the user repo (https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts) it should be toggleable even if it doesn't work for actual downloading. I attached everything I have that matches instagram in pic 2 in case instagram got removed from the user repo for being nonfunctional. >>24510 >The site always lists more posts than I get downloaded It may be seeing posts that don't have pictures or have already been imported. If the guy reposts the same picture hydrus will see the file matches an already existing file and will simply associate the new url with the picture, there's no reason for exact duplicates, see pic 3.

>>24506 I booted 644 after a backup since the changelogs seemingly had a lot of db stuff and it felt slower to populate than usual? But it may have been the transparency checking also happening, not sure. Although the hangs I usually get from procrastinating cleaning up hydrus seem lesser than usual so I guess it evens out.

>>24511 I already figured it out. I had to do that which I did before making the post and I had to edit the "import cookies" option in the settings. Third pic I had to add 'api = "hydownloader"' >>24512 I found why every image had multiple Twitter options. It was displaying the same link twice with a minor difference. The Github you linked worked well. Thank you! It didn't use the tags that Hydownloader grabs but it's works besides that.

Hey hydev does your visually similar auto-resolution use gpu acceleration or lots of compute or something? I have some hardware issue causing kernel panics that I've never been able to properly nail down, it is generally caused by sudden bursts of computer load, and I think hydrus is triggering it with whatever happens when trying to find visually similar pairs.

>>24515 >I have some hardware issue causing kernel panics that I've never been able to properly nail down, it is generally caused by sudden bursts of computer load If you have a Zen cpu disable power state 6, it causes a hardware fault in some chips. If not it might still be something similar so look up things along those lines for your chipset.

I'm not sure whether this is still a problem, but could you add an automatic bypass for when in something like URL import, the entire import queue hangs because one of its items has hit a bandwidth limit, even if there are further items that don't have the same bandwidth limit? To be more specific, say I download the default 2GB off rule34, but then queue up more downloads off 4chan. I want it to download the 4chan files immediately because they might be gone tomorrow, but it'll hang for 24 hours on the rule34 download before going any further and it needs manual intervention.

>>24515 shot in the dark, but I recently fixed some incessant graphics hardware hangs with the swiotlb=65536 kernel parameter. https://wiki.gentoo.org/wiki/IOMMU_SWIOTLB#SWIOTLB_for_high_input_output_(such_as_graphics)

>>24519 >>24517 I'll check these out when I get the chance, thanks bros.

I just realized that my hydrus isn't downloading any loli from gelbooru like a non logged in acc but I do have a login, what do I need to do to fix that?

>>24341 I'm new to Linux so I'm still learning it. Will having two versions of Hydrus on my PC not mess anything up with the database?

>>24534 Scroll Up It's a http header you need to import, gelbooru doesn't check account status for it because they already track that option for anon users as well

>>24488 Not with this system. Before I overhauled it to the current 'virtual only' system, It supported a mix of 'hard' mappings replace and soft/virtual display-only swap-ins. It was a huge mess to deal with with many bugs, so I decided to move to entirely soft/virtual, where no mappings are hard-replaced and any decision has 100% undo-ability. I am thinking and generally planning to add a hard-replace system, something a bit like tags->migrate tags, that will be one-off powerful jobs you fire. I'm sure it will talk to siblings and allow you to hard-collapse some stuff, but there is often little actual benefit in doing this and often a bunch of frustrating pains in the ass when something unexpected happens. >>24507 I've noticed what I'll call 'navigation' lag in the past say three months in my IRL client. I click on a page tab, and one in fifty times when big imports are going on it might take 800ms to actually switch tab. I am not sure if it is the page doing some 'on page show, recalculate some taglist stuff' or if it is just some file import job eating up some CPU. I'll keep working here, so please continue to let me know how this goes, and feel free to send in some profiles as under help->debug->profiling. >>24508 It is a long story, and the answer is 'somewhat'. Basically if you import a zip full of numbered images that looks like an ugoira more than a cbz, we call it that and we've got some rendering tech. The big source of Ugoiras online, Pixiv, has some jank javascript solution to deliver frame timings, and I've been holding off for years on actually downloading Pixiv Ugoiras (the default hydrus Pixiv downloader skip/vetoes them) because I've wanted a nice way to grab this timing data and put it in a note text or .json file or something to actually embed in the Ugoira zip, but there's no really nice solutions so it all got put off. We have the note-parsing tech for synthetic framerate, which is part of what I did last week, but no nice Pixiv site-parsing solution to grab it yet. If you grab an Ugoira from danbooru or something, hydrus will import it and usually render it with a fixed framerate of like 12fps. Fingers crossed we'll one day figure out a nice solution here and we'll finally be able to get them from Pixiv, with accurate variable frame rate. Most of this work was done by a user who was keen to make Ugoiras work, and I am very grateful. I'm personally a little pissed off by the whole decade-long debacle; the way Ugoiras work behind the scenes is dumb, and imo the whole thing should have been converted to apng or something years ago (although webm is fine because almost all Ugoiras these days are jpegs rather than pngs anyway, defeating the original cool thing they did), but I'm really just expressing my frustration. >>24513 Ok, thank you. Some jobs were split into smaller pieces or made more polite, so there is a little more overhead or just latency in order to reduce lag. Some of that I'm ok with. Let me know how this continues to go, and if you notice anything in particular hanging for ages, I'm interested.

>>24515 I can't talk about the hardware or config stuff here, but to answer this specifically--yes, mostly CPU. I basically render the image to a bitmap, resize it to 1024x1024, cut that guy into like 64 tiles, and do some hardcode histogram and gaussian blur math on it. There might be a touch of GPU in there, at least initially, but it'll mostly gonk one CPU core for, I'd say, 50-400ms depending on the image and your machine and stuff. I haven't properly profiled this IRL though, so it may be a bit faster, and the follow-up delay is other stuff all floating around the duplicate filter or auto-resolution. I assume you have some duplicates auto-resolution rules that use the new visual duplicates stuff, or is this in the duplicate filter? If you do have a rule doing this, if you want to test, try turning off the duplicate auto-resolution rules work in 'normal time' and 'idle time' under the cog icon in a duplicates page->auto-resolution tab. The rule won't do anything and no visual duplicates work won't happen outside of the duplicates filter. You can schedule this work under options->maintenance and processing also. I can't slow a single job down, but you can reduce the size of the bumps to minimum size by changing the 'packet time' for duplicates auto-resolution work to like 100ms and boost the rest percentage to like 9000%. That'll mean it usually just only ever does one meaty job and then takes a break for ten seconds. There are other times that hydrus eats up CPU. The bigger your client, the more it'll happen. Similar files search can be pretty meaty, and PTR processing, and file imports in general. If you've noticed this just recently, I'd hesitantly say that it might be on the OS side, some new driver update or something, since most of my work in the past year has been in reducing the size of jobs and spreading things out to be less rude, with the exception of the visual duplicates test, which is indeed new and does some hardcore math on memory. Let me know what you figure out, and if I can fix anything or add any new options, I'll give it a go. >>24518 I have a plan to deal with this, and similar stuff for waiting on domain errors, where we'd rather retry later. Unfortunately it will take quite a bit of cleanup and overhaul work in the 'file log' and some downloader stuff. I have it in mind, but I can't promise it any time soon. It sucks, but the best solution for now is to split large URL downloaders by rough domain, or have a separate one for a regularly bandwidth-choked domain.

In one of my duplicates auto resolution rules, I found a pair that was resolved as a better/worse pair, but the notes weren't merged. The worse file has a note with body text that's the exact same as the better file's note, but the better file and worse files' notes have different names. For some reason, the note from the worse file wasn't merged to the better file. Is this intentional, and if so, is there a way to change it so that the note is always copied to the better file even if they only have a different name but the same body text? I have the "if possible, extend existing note" setting enabled, but that says that it only happens if the notes have the same name. I also have the merge conflict setting set to "add note under new name" so that shouldn't be the cause of the issue either. So I don't know why this is happening.

>>24540 >Not with this system Figures, I just thought it was annoying to see all these tags I set as children pop up constantly everywhere whenever I tagged shit that I'd add in my database. It's a bit distracting, especially when the children has more tagged images than the parent.

>>24547 If the issue you're having is that you just don't want to see the tags, you can hide them from being displayed with a button in the tag's context menu.

>>24515 Hardware issue causing kernel panics? Run a deep memory test. >>24536 Each version of hydrus uses a specific database scheme. When upgrading hydrus you can see the database gets upgraded through every version you skipped. I recommend against mixing versions.

>>24541 Yes, it was the visual duplicate processing in auto resolution. I once had a panic right in front of me after I did "review actions" in semi-auto after just adding a visually similar pairs and i've had it panic while I wasn't there when it was processing in the background. I've had visually similar processing paused since then.

>>24536 >>24552 >I recommend against mixing versions. It's perfectly fine to run two separate clients/databases with different versions though.

In 643, "furaffinity.net favorites page parser" gets the urls with its "content parsers" "urls - post page urls", but the urls are not used in the final (empty) result.

In the duplicates auto resolution rules, I wanted to add a comparator that checks that files A and B don't share any known urls. Is there some way to accomplish this with the kinds of comparators that currently exist? If not, would you mind adding a comparator like that, sort of like how the ICC profile value comparators work? I wanted to use that kind of comparator in one of my "riskier" rules as a way to avoid files that share a post url from being paired up by the rule, since I've found that files are much more likely to be alternates if they come from the same post.

I had a good week. I did the duplicate db tech thing I wanted, and I figured out an mpv test that may fix some long-term issues. The release should be as normal tomorrow.

I have my search distance for potential duplicates discovery set to 8. Is it safe to set this as the max search distance in auto-resolution rules? Since the "almost definitely visual duplicates" comparator should still ensure that files A and B are actually duplicates, I think that it should be safe, but I just want to make sure since it's not the default.

https://www.youtube.com/watch?v=uzg3DFsnH3I windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v645/Hydrus.Network.645.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v645/Hydrus.Network.645.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v645/Hydrus.Network.645.-.Linux.-.Executable.tar.zst I had a good week working on a mix of stuff. There's a new mpv test mode. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights For advanced users only: I figured out a wrapper that deletes mpv windows rather than re-using them. It doesn't seem to cause crashes any more. If you want to help me out, or if you often have a 'every other mpv window is mute'-style bug, please brace for a crash and try out options->media playback->TEST: Destroy/recreate mpv widgets instead of recycling them. Click a video and then unload it, try navigating from a video to an image, then video to video. Do you have any problems? Is anything fixed? I can't get it to crash, but if I scroll super fast, I can get a hang. 'review bandwidth use' now sorts its web domains so subdomains are below their parents. I'm renaming the advanced 'all local files' service to 'hydrus local file storage'. New users get it today, existing users are planned to get it in a couple weeks. If this breaks something for you, please shout at me. I'll do 'all my files' to something like 'all local file domains' soon too. The duplicates db tech improvement I wanted to do is done. You'll get a bit of work on db update this week, and your 'x pairs' numbers in the duplicates system will be better (they won't count some deleted stuff any more). next week I want to figure out fast duplicates search for very low hit-rate searches, like '5 pairs in 750,000'.

>>24480 >>24483 >>24506 Still experiencing tons of hanging on CachyOS since I switched from Windows to Linux. I run from source now and update using git_pull.sh. Experienced it on versions 642, 643, and 644, so no recent changes have remedied it. It seems to be happening more consistently though, as I used to have long periods free from it entirely. Most hanging occurs in the tag manger, with it hanging for 0.5 all the way to 5 or so seconds upon trying to load tag results, or trying to close the tag manager/apply the tags. It also crashed recently. I left it open and untouched for a few hours, and when I tried opening the window, everything but the info bar for the window and the window buttons was not rendering. Wasn't black either, it was empty and I could see other windows through it. After minimizing and maximizing a couple times to see if it clear up, it crashed. Instantly closed too, not a freeze where I had to manually kill the program. Since switching to CachyOS it's taken longer to initialing when restoring my tabs than on Windows, but after this crash it took much longer than usual. Curious to know if those logs I emailed the other day gave any insight into the issue? I'm considering seeing about running another install inside of WINE instead, but I'm not sure that will play nice with my db unless I put it inside the WINE prefix with it. Not sure about storing anything like that inside a WINE prefix at all. Updating to 645 now.

>>24573 Also, I must note, the hanging in the tag manager for tag searches happens even when searching the same query over and over. Been going though a lot of similar files that need tags that all start with the same word. While the very first search for that query can take the longest, repeated searches when the results I presume may still be in RAM can be instant on occasion but usually hang with variation now.

I've noticed in the last version, and this version, audio will continue playing after a file is deselected, until the end of the video is reached. This only happens if there is a very small length of the video left to be played, but can result in audio from two videos playing at once. It has not caused any more serious issues though.

After updating to 645 (from 643) some of the side panels like in the gallery download page have become about half the width of my screen, manually shrinking them causes them to cut off. I've seen this in the URL and gallery downloader pages, and the duplicates processor page, but with it changing tabs fixed it. An empty db doesn't seem to exhibit this behavior. Not sure if this is something I did accidentally or a slightly botched update.

In an auto resolution rule's "actions taken" tab, I'd appreciate if there was a way to open a pair that's in the list in a new page, kinda like how you can when you're in the filter. currently, I can't figure out any way to see a file that's been deleted (so I can check its metadata and such) since I can't open it to send it to a new page.

>>24546 Sorry for the trouble. The note merge options are really ugly and unintuitive. The logic behind the scenes is a pain too. I will have a look at it and try to make it more obvious what is supposed to happen, and then make sure I have unit tests to ensure that is actually happening, and I'll look at your exact report. >>24547 >>24548 Yeah. If I can get myself in order, I want to spend a bit of 2026 just revamping the 'manage tags' interface. It is too difficult to manage a bigass list of tags, and there should be more viewing/grouping/whatevering options to make dealing with siblings and parents not a pain in the ass. >>24564 Is this from this guy? https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/blob/master/Downloaders/furaffinity.net%20-%20version%202020-02-19%20-%20normal%20-%20AiO%20for%20Classic%20and%20ModernUI.png I'm afraid I don't make or maintain that, but did this happen to work in previous versions, and only now it is breaking in ~v643? I've been doing some URL handling changes in parsers recently, so let me know if I broke something. >>24565 Thanks, good idea. I'm hesitant to add too much 'dynamic metadata' comparator tech that examines things that can change, at least until we have a better grip or what works and doesn't here, but a hardcoded rule for 'no shared urls' is a good threading of the needle. >>24569 I'd say set it at 0 for now. My 'visual duplicates' comparator is way overtuned to never produce false positives, so I don't think it'll true positive for anything at distance 2+. You'll just be wasting CPU if you make the search encompass 'fairly similar' files. I'm working on this problem though and over the next months and maybe years I hope to make the 'visual duplicates' algorithm more capable at tuning out blurrier jpeg artifacts. I hope we can slowly expand the visual dupes search to encompass more stuff in future.

>>24573 >>24574 Your profiles were useful and I optimised some UI update stuff for the past couple of versions. I don't know how much was truly 'real' though, so let me know how v645 feels. I'm sorry to be lame, but if your client with its current sort of session and usage was ok in Windows but is now having trouble in this OS, it may more be that your OS isn't happy with the Wayland/X11 interpretation layer or compositor or Qt or disk access or some other complex Linux thing I'm not familiar with. The true solution here might be some conf, but I will keep working on optimisations, and I'm interested in fresh profiles in the latest code. I've noticed UI hitches in the past couple of months on my IRL client, where I'll click on a page tab and it won't show for 600ms kind of thing. I am not sure if it is new auto-resolution stuff or something else. I'm investigating it. >>24577 Please hit `options->media playback->LEGACY DEBUG: Use legacy mpv communication method` and restart the client--is it better? If so, leave that mode on and let me know your OS, hydrus install situation (source, AUR, built package?) and mpv version under `help->about`. >>24579 Sorry for the trouble here. It is probably the multi-column lists in those panels. In some random situations, I don't know how it happens or who for, my internal record of how wide one column should be, usually the rightmost one, goes crazy, like 135 characters, and then when I re-lay those pages down, they come up going 'oh, the user saved this column as being this wide, so I guess my minimum width should be 1100 pixels' or something similarly stupid. Please right-click the header of the affected lists and go like 'restore (list type) to default settings', and then resize your sidebar to what feels good and then restart the client. If this happens again, let me know if you remember what you may have done right before it happened. >>24580 Thanks--I'll make sure this gets done.

>>24582 >it may more be that your OS isn't happy with the Wayland/X11 interpretation layer or compositor or Qt or disk access or some other complex Linux thing I'm not familiar with That's probably it. I may just have to give up on it working smoothly because CachyOS doesn't like GUIs or something. It seems like if you want niche software to work on Linux, you'd need a deep understanding of the program, Linux, your distribution, your window manager, and Qt. Given everything has to go through the window manager and Qt regardless, I don't think even running it in WINE would help. >>24582 >Please hit `options->media playback->LEGACY DEBUG: Use legacy mpv communication method` and restart the client--is it better? This did not get rid of the audio continuing after deselect, but did make it so that the newly selected file could not begin playing until the audio from the previous file finished. OS:CachyOS Hydrus+MPV: Pic related

>>24583 Damn, ok, please keep me updated. I'm going to write some extra safety code for mpv this week that does stuff like 'only move to the next media when the current mpv window is safely "done"', which sounds like it will affect your situation. I moved to Linux Mint from Windows for my main desktop this year and found it a very pleasant migration experience outside of the normal Linux conf stuff you need for weird USB devices and so on. I would recommend it to anyone, including for hydrus.

>>24584 Maybe I'll switch to Mint. It was a lot of work getting everything I've got going now working on CachyOS though. CachyOS is the distro pushed as being pre-optimized for running games, besides Bazzite which is more aimed at dedicated portable gaming machine and is immutable. Looking around online, I think a lot of my issues are caused by Wayland, which has been forced recently upon KDE, which seems to be the standard DE for CachyOS. I think Mint's top DEs lack the issues of Wayland, and that's why everything runs smooth on your machine. At the very least, it would probably fix my drag and drop issues.

What can i do to filter/reject urls in the downloader BEFORE fetching them and using up bandwidth? for example I can disable archives in the import options on a downloader page to stop from importing .zip files, but hydrus will still download all the foo.zip urls first before rejecting them.

>>24585 Changing DEs so you can use X11 isn't that bad. Skim through here and find one you like: https://wiki.archlinux.org/title/Desktop_environment

I had an ok week working mostly on stability and optimisation. MPV windows are more reliable, and potential duplicate pair search is faster when the hitrate is low. The release should be as normal tomorrow.

>>24593 KDE still works on X11. It's just not focused on anymore. I still use Plasma on X11 in Fedora. >>24585 I've never even heard of CachyOS. I didn't read the whole chain, but assuming you're somewhat new to Linux, you should REALLY just stick to one of the more well known distros. Mint is actually a great choice IMO and my top pick for newcomers.

>>24599 CachyOS and Bazzite are the only ones presented as optimized for videogames, and Bazzite is immutable which sounded like a pain. CachyOS wasn't very difficult to setup. I had gotten things 95% of the way working how I want them to within a week, there's just that last 5% of shit that's annoying.

https://www.youtube.com/watch?v=9-m9a4TDN7I windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v646/Hydrus.Network.646.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v646/Hydrus.Network.646.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v646/Hydrus.Network.646.-.Linux.-.Executable.tar.zst I had an ok week mostly doing optimisation and stability cleanup. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights When potential duplicate pair searches happen--which mostly means when you see a duplicates page search count up the number of matching pairs or when a duplicate filter gathers pairs to show--I have a new optimisation for when the pair count is very low, like '5 in 750,000 pairs'. The incremental search should recognise it is getting a very low hitrate and, rather that iterating through all 750,000 to get the 5, change search strategy and get all 5 in one go. There's a little tuning magic in here, so I am interested to know when this succeeds and fails. Large GUI sessions will load in a less laggy way, and all pages now load at the same time, so smaller pages should populate faster. The mpv test last week was ok, but there were some hangs, either when transitioning too quickly through many mpv windows, or, on Linux, exiting certain media viewers with an mpv window in view. I have added more safety code to all mpv handling, test or no, and it seems to be much more stable now. If you tried the test last week, please give it another go and let me know what you get. next week The low hitrate potential duplicates search optimisation was the last 'big thing' of the duplicates auto-resolution work. I've now got about a dozen smaller jobs to clear out, and I'd like to get it done before the end of the year. I'll keep pushing on this.

>>24601 I don't know what in particular you did, but I think you just fixed all my hanging issues I've had since moving to CachyOS. Since just a short while ago jumping from 645 to 646 (both from source), I've experienced no more hanging whatsoever. I tagged 10 files in a row with zero delay in the tag manger autocompleting or closing the tag manager/applying tags, besides the very slight delay that's normal shortly after having just opened the client. All my pages also initialized much faster as well. The memory usage, I think, is also way down. It was using over 4GiB before, and now it's sitting at 1.3GiB with my usual pages, and I can only seem to push it up to 2.6GiB by loading a page with all my largest 1000 files and then quickly jumping around through 1-4GB videos. After closing those it'll then sit around 2.0-2.4GiB. Something else I had just noticed recently with my hanging issues is that the infobar at the bottom always said "very busy" no matter what I was or wasn't doing, which I'm certain had something to do with the delays. That constant very busy status is no longer present either. Hopefully it stays that way. I'm guessing something was broken when run through CachyOS and this update either fixed it, or optimized the hell out of it. Thanks a ton.

Some feedback for the command palette (ctrl+p). I have a 336 pages open, and there's pretty significant lag when it opens. This would be fine if the lag didn't interfere with typing. Often my first few keyboard inputs will get flipped around - for example, if I start typing "import" while it lags, it sometimes ends up as "pmiport". Also, it seems a bit strange that page names are elided in the command palette the same way they're elided as tabs. Surely it could show many more characters since there's lots of unused horizontal space.

>>24605 >I have a 336 pages open Jesus Christ, why?

>>24601 >youtube video That's so cool.

I just ran into a webm with multiple audio tracks. How do I change audio tracks in Hydrus? Videos are set to play using mpv, and that defaults to English.

Fedora 43 is officially removing Xorg support :/ Dev are you able to focus on fixing mpv support in wayland? Image viewing works, but mpv breaks the viewing window, outputs this error, and requires full program shutdown. on v646 >The cached device pixel ratio value was stale on window update. Please file a QTBUG which explains how to reproduce. >>24573 Is it hanging when it says "CPU Busy" in the lower right corner? Even on a decently fast CPU it will hang when loading lots of stuff. Telling ya use a more mainstream supported distro. I have had nothing but problems when I tried Arch. If you really want a super optimized distro for your CPU, try gentoo. I found this article recognizing its performance improvements, but also the extremely rough state of the OS. It also looks like the performance benefits are more suited towards benchmarks than real world applications. Still a cool distro and I hope others take notice of their more precise compilation settings. https://www.theregister.com/2024/07/23/cachyos_arch_linux/ https://www.phoronix.com/review/cachyos-ubuntu-2510-f43/5 >>24596 Thanks for improving MPV support.

>>24610 iirc you can still force hydrus to open in xwayland. no way they're dropping that too.

I need some advice as my database disk image seems to be malformed. Hydrus is running within a Win11 VM on an older HDD, chkdsk from host and within VM returns ok, Crystal Disk values (host) are also still good. I tried to troubleshoot using the included .txt file but checking with PRAGMA just returns with "ok" after a few seconds but my db is rather large. My VM crashed on me a few times with "Your device ran into a problem and needs to restart" while writing, the host should have enough power and the VM has enough RAM. Checking DB integrity within Hydrus says its running but i dont see any HDD activity from within the VM apart from a 500KB/s reading, i will let it do its thing for a while see if anything happens. I assume it is not a hardware issue im experiencing.

>>24609 Right click/Open/In external program Then in that video player select the desired audio track.

>>24614 >Hydrus is running within a Win11 VM Seriously. Why?

>>24615 So it can't be done within Hydrus? >>24610 >Is it hanging when it says "CPU Busy" in the lower right corner? Even on a decently fast CPU it will hang when loading lots of stuff. Telling ya use a more mainstream supported distro. I have had nothing but problems when I tried Arch. If you really want a super optimized distro for your CPU, try gentoo. My issue was resolved with the latest update. And yes, it said "very busy" in the bottom right all the time, despite hardly using any resoureces according to System Monitor. It no long does that though.

>>24602 >Hopefully it stays that way. >>24617 >My issue was resolved Scratch that, the problem has returned infrequently this morning. Tried closing the program and reopening it, since RAM usage had managed to go over 3GiB, and it's now initialising incredibly slow again. CPU usage doesn't even break 10% while this is happening, the status bar at the bottom does not constantly say "very busy "anymore", and memory is now sitting around 1GiB, but I'm still experiencing heavy hanging.

Can anyone help me fix the Nijie downloader? While the downloader can find posts from an artists page, it just spits out "Could not find a file or post URL to download!" when it tries to download the images.

I updated from v642 to v645. It looks like there was a change with the colors, where in the media viewer, notes and the little zoom and file number text in the bottom right are colored white instead of the faded blue-like color that they had before, even when the mouse isn't hovering over them. Is there a way to change it back to how it was before? I liked that the text was a faded color until I hovered the mouse over it. It looked nicer I'm using darkmode colors by the way, and I didn't touch any colors or style sheets between the updates.

>>24622 oh and just to be clear, I just updated again to v646 and the issue is still there

>>24592 Unfortunately I don't have smart filetype detection and filtering in my network jobs yet. I do plan this, but I'm not sure when it will happen. For now, it does this wasteful post-download check, so your best bet, if it is a big issue, is to scan your download list for obvious .zip URLs beforehand. >>24602 >>24617 >>24619 I discovered a years-old slowdown and memory leak at the very last minute on Wednesday. It could also cause very busy clients to hang certain async jobs, particularly single pages of initial session load. Some clients would be affected more than others. I will polish up my solution here a little more this week, but I'll take your report as progress. Since you are still getting lag sometimes (me too), we'll keep working on the next thing. Please explore the help->debug->profiling menu and see if you can capture a slow period. I just overhauled this profiling system a month or so ago, fixing many bugs, and while it is fairly good, I'm not terribly happy. I plan to explore using a new library for it in 2026 that will report multithreading stuff better. I have some good patterns and experience in reducing lag now, so the main task is to find the causes and then attack them. >>24605 Thanks, I will fix the elided page names. The switched characters is interesting. I didn't actually write this code, so I wonder if it does some out-of-order asynchronous text capture somehow. Can I ask you to try the 'ui' profile mode under help->debug->profiling? I'm not sure if that will capture the command palette, but let's see. If I can see what in the 336 pages is lagging it, I know where to work. >>24608 I stumbled on this a couple weeks ago: https://www.youtube.com/watch?v=CDN1P77Nm_Q It is just a bit on the cringe side of things, but I still liked it as this weird prog rock experiment and it got me looking at old trains a bit. I was struck at how large and bulkily aggressive the cross-continental work engines in use were, as opposed to so many of the pretty museum pieces. There's a couple big engines here: https://www.youtube.com/watch?v=dJfiFFMuOMI I dunno if 1897 was a critical year for black and white video, but there's a bunch of footage starting from that year so I'm guessing a guy invented a camera and took it on a tour. Here's an elevated train and what's got to be Teddy Roosevelt: https://www.youtube.com/watch?v=4LOysNLEqLI This appears to be a gold mine: https://www.youtube.com/@HuntleyFilmArchives/videos

>>24609 No support yet, but thanks for the example file. Opening externally to do anything clever is the best answer for now, but I can in theory dig this metadata out with ffmpeg or mpv natively and then present UI and a 'switch audio track' command to mpv, it'd just take some work. >>24610 I am not an expert, but as I understand, Wayland does not allow you to attach a window handle to a thing to be embedded, which is the central conceit behind my call to this mpv library: https://github.com/jaseg/python-mpv I've had a plan for a long time to explore another render solution where as I understand I basically spawn an OpenGL window with Qt and then behind the scenes ask the dll to draw to that render context in some way. I assume I'd have to figure out audio syncing separate to this. I don't know if the python-mpv library has a handle for this or if I am looking at something else or doing something myself. This would not be a small amount of work, and I don't know if the duct tape bullshit I am normally up to will handle it. There may be other ways to embed mpv into Qt in a way that Wayland is happy with, but I understand hydrus's Qt also spergs out in Wayland with multiple duplicated recursive main gui menubars and other such fun, so it is likely we have more bugs to deal with here. So, to answer your question properly, I am afraid I do not think I can come up with any nice patches any time soon. There may be solutions in Wayland, perhaps even nice ones, but I don't know when I will fit it in. I recognise that X11 is not in favour and Wayland and its new security and permission rules are probably the way forward, so I will have to deal with this, and I'm sure the Qt guys are working to improve their support too. It may be that hydrus on Wayland comes with a 'here is the XWayland stuff you have to apt in to get it to work' mini-guide in the running from source help for the next three years. Qt has an in-built media player, which I have as an 'EXPERIMENTAL' option under options->media playback. My prototype here is dogshit, and I didn't have much success with it, but I have heard the underlying player is better in newer Qt, so maybe I should revisit it and see what I can do. >>24614 When you say 'malformed', did you get that actual error with that specific word, and that's what lead you to the 'help my db is broke.txt' file? Normally, when you get a 'malformed' error, it means the database is scarred at the hard drive sector level, but it can happen if the intervening I/O is busted, for instance a joggy sata cable or a weak PSU, since a 'correct' or 'incorrect' blank sector read looks the same both ways to SQLite. If you are able to do a terminal integrity check and get 'ok' on all four files, that suggests the latter. It sounds like you have done all the correct stuff with chkdsk and crystaldiskinfo though, so it doesn't sound like you have a strict hardware issue. When you did the integrity check, you were doing that inside the VM, right? Not from outside but pointed at the same file? I'd assume that an integrity check would run into the same I/O problem that normal access would, but perhaps your issue is very random or triggered by random access versus a more contiguous pull or something. The VM 'needs to restart' during a write sounds like the main culprit here, but you'll have to figure out what the core problem is. My feeling is software. A bug in some write buffer tech. One thing I just thought of: Hydrus eats some mega temp space sometimes, so if your VM has a limit of like '512MB temp dir' in its conf settings somewhere, you might be getting a difficult to define error on a sustained write flush when the WAL journal gets committed. This wouldn't happen for a smallish client just doing some file imports, but if you have the PTR running and ten different file import queues of multiple 18MB pngs going, the journal and other temp usage can really pile up. Good luck, and let me know what you discover. If you do get an integrity_check problem and the normal clone stuff my .txt talks about doesn't fix it, let me know and we'll work on it together.

>>24622 >>24623 Thank you for this report. I cleaned up some of the 'media viewer background' text rendering a few weeks ago, and now everything there is selecting its style in better isolation and is drawing how I intended it. Is there any chance that there's another element in all that that still, and correctly, has that blue colour? Like it might be unnamespaced tags or a hyperlink or something? The bug that was happening before was that some things were setting the wider 'painter' instance to be 'text colour blue', and then the next drawing thing in my chain was sometimes forgetting to set back to 'text colour normal', so deviations were propagating to unusual places. I have fixed this now. So unfortunately what you see now is what I intend, with the colour in the background all following one option setting. This colour may differ from the normal text colour in the rest of the program (which I am fairly confident is what the 'on top' hover window has), but if it has not been defined in your stylesheet, I think it'll defer to the normal text colour. If you want to change those colours, you want to check under options->style for a stylesheet you prefer or install_dir/static/qss/readme.txt. If you decide to make your own edited QSS file, the specific QSS entry to edit is, I think, 'hmv_text', which you will see in some of the QSS files under a block like this: QWidget#HydrusMediaViewer { qproperty-hmv_background: #181825; qproperty-hmv_text: #cdd6f4; } If the QSS stylesheet you were using until now does not define that block, that may be why your text is falling back to the same brighter white. If your QSS does define that colour, and it is a nice blue hex code or whatever, and things are still rendering wrong, let me know! vid related is what I see when I set up what I expect to happen with a fully set stylesheet.

>>24627 >Is there any chance that there's another element in all that that still, and correctly, has that blue colour? Yeah. The top box frame (where the filesize and import time is shown) and the known url link text both have the same color from before. >If your QSS does define that colour, and it is a nice blue hex code or whatever, and things are still rendering wrong, let me know! I'm using the default_hydrus_dark stylesheet, and it defines those colors as QWidget#HydrusMediaViewer { qproperty-hmv_background: #343434; qproperty-hmv_text: #708090; } Those are the colors I expect, and for the top hover box and the links, they have that color as long as I don't have the mouse over it, like I want. The notes however, are always white text. I attached a pic so you can see what I mean. The pixiv link is the correct color, but the note text is white even though my mouse is popping up the box. I tried disabling that "override colors" setting to see if that would fix it, but no. It's still always white, even though the hydrus_dark QSS seems to set it as the blue color. >vid Yeah, that happens for the links and top box, like I mentioned, but not the notes or the bottom right zoom and file number text. Those are just always white.

>>24629 >even though my mouse is popping up the box I meant "isn't". My mouse is NOT popping up the box. my bad

>>24629 >>24630 Thanks. I was looking in the wrong place. I'll give the whole thing a pass, now it is rendering 'correctly', and make sure the 'correct' is something I intend. I'll adjust my dev stylesheet, too. I am a lightmode guy so everything in my world is black on white by default, so this stuff doesn't stand out to me when I'm working.

Is there a setting to disable the corner selector in the full media viewer? It hasn't been working for a while but no-one's had a bug about it so I'm wondering if I accidentally ticked a setting off. It works fine in the preview viewer. See vids, I can click this panel in the previewer, but it doesn't show the hover panel in the main viewer, it would sometime be offset down in the main viewer but now it doesn't show up at all.

>>24624 >Please explore the help->debug->profiling menu and see if you can capture a slow period. Caught it doing it about half the time as I was adding a few individual tags to 50 files on a case by case basis. 109 slow jobs, 531 fast jobs. Log sent by email as before.

>>24632 My corner menu in the fullscreen media viewer works fine, but yours looks different. No idea how you got all the info to display above the image like that or make the image move around when you hover over different info. Nor am I curious as to how, as I like mine the way it is. That was Astolfo that showed up on the bottom for a second, wasn't it? :^)

>>24634 Yeah, I love this pic

>>24635 I fuckin knew it. Ghey.

>>24624 >Can I ask you to try the 'ui' profile mode under help->debug->profiling? I'm not sure if that will capture the command palette, but let's see. If I can see what in the 336 pages is lagging it, I know where to work. It won't capture it. It says 0 slow jobs. No profiles created in the db folder.

>>24624 >I stumbled on this a couple weeks ago: Thanks.

>>24631 thanks. I hope you can figure out the issue and fix it. It's not like it's urgent or anything (like it's not breaking any functionality for me) but it is pretty annoying and distracting. I appreciate it

>>24616 i tried ubuntu 22 (?) and source install but had problems with hydrus gui and mpv so i went back to win11. >>24625 oh i misread the .txt, sorry, i checked every db file now. integrity_check did give me errors for master.db file. quick_check for mappings.db took too long so i aborted it for now, assuming its faulty, client and cache dbs are fine. I went and tried to fix the master.db but when it starts cloning it says "hashes..." and the cmd is gone, so im stuck there now. i am going to run a quick_check for mappings.db soon propably will take 2 days straight. Meanwhile i set up a fresh vm on another hdd with ubuntu 24 i once installed hydrus from source (v646) but i used to have problems with mpv fucking up my gui when i opened a video in the viewer and the gui was not responsive anymore, double&moved menu rows, i was not able to fix it. this time same problems again. i will try cloning it in ubuntu vm if it works my win11 vm is fucked. bottom line i had to do a clean up sometime soon anyway.

>>24277 Some anons are staying on X11 (especially now that there's an XLibre/X11Libre project updating Xorg now) and some of MPV doesn't work right if you aren't using Wayland. Could you add VLC support as an alternative to MPV? That way we have the freedom to sidestep MPV idiosyncrasies and even avail ourselves of VLC features.

>>24279 >https://www.youtube.com/watch?v=v6ng_bGozSo Great video but could you explain what the point/relevance of linking that was?

>>24643 Ubuntu kinda sucks now. Try bazzite. Fedora is getting shitty too but right now that's kinda all of open source. >>24625 Dang. The Xorg vs wayland stuff is understandable. >>24645 https://x11libre.net/#about >big tech sabotaging improvements on x11 sounds about right. I'm glad to see these projects that side step sabotage.

>>24646 >could you explain what the point/relevance of linking that was? Lurk moar newfag.

>>24645 >especially now that there's an XLibre/X11Libre project This is very promising. I'll keep an eye on it.

>>24648 I'm an oldfag who spent some years away at zzzchan because /tech/ was resembling /g/ too much. Just clue me in already instead of acting like a sekrit club fag.

>>24655 >sekrit club I can tell, you are not an anon.

>>24645 >some of MPV doesn't work right if you aren't using Wayland Haven't noticed anything about this. Which parts are you talking about?

>>24656 >unironically being a sekrit club fag >"I can tell, you are not an anon." I can tell you're one of those fucking newfags. The "no spoonfeeding" meme only arrived on 8chan after moot shut down 4/pol/ and the refugees started raping the board culture left and right. That 4channer wave was a big part of what ruined old /pol/, old /v/, and even old /tech/. And they were the ones who would rant about shit and then cry "no spoonfeeding" whenever people asked for sources/details/etc. I remember seeing that "no spoonfeeding" shit so much all of a sudden even though in the GG days not providing a source or rationale to back up your claim would get you shat on by everyone because we were all skeptical as shit having witnessed how much BS was being spread around unchallenged by supposed journalists and internet rumormongers so if you weren't willing to prove you weren't full of shit we all treated you like someone who was. You're one of those /g/ niggers pretending to be an oldfag even though old /tech/ had a very different board culture from you cunts but your demented newfaggot brains never did know the /tech/ board culture. Neck yourself, fool. Newfags like you are what made 8chan so cancerous that us oldfags started posting less and began moving on to greener pastures. >>24658 This: >>>/v/1919517

>>24659 >>24656 >>24655 >>24648 Take your uproductive retard fight elsewhere.

>>24659 >larping like one of us >pulling rank by claiming old faggotry Nigger, by posting Bill Nye you ousted yourself. If that establishment's shyster and propagandist is authoritative for you, you are beyond redemption. I bet you are a NPC covidian and believer of the Climate Hoax religion. Fucking unreal.

I had a good week working on a mix of stuff. There's fixes, quality of life, and some more duplicates tech. The release should be as normal tomorrow. >>24646 It is mostly just a fun habit, really, back from when I once made a new thread for each release and needed something to post with and just picked something neat that I'd seen around recently. If this week's release sucks, well, here's something else interesting from the culture. I've since realised that it also works as a very light gatekeep and shibboleth. If someone sees me successively post vids of some worm robot or DJ Sharpnel or pannenkoek2012 or Bach and thinks 'that's cool', they are probably one of us and may well enjoy hydrus. If they think 'this is weird nerd shit' and close the tab, they don't end up running into software they wouldn't like anyway.

Is there any way to extract information from imported URLs, especially after import, and turn them into tags? Like if I imported a video with URL https://www.youtube.com/watch?v=j0HEDcgAs1Q, it would automatically tag it as site:youtube, video_id:j0HEDcgAs1Q I assume hydl could do something like this, but only on import. I'd want to do this with already imported files, all with URLs.

>>24614 >>24625 >>24643 Problem solved for now, the master.db was bad. ".clone" did not work for me on win11 but ubuntu with the package for sqlite3. Took about 5-10hours for a 25gb file within VM on a HDD for reference. As for the .recover command... it seems artificially slowed because of the console output for every single action, i aborted after roughly 48hours and i was not even half done, about 40million lines/day (this one on host with ssd). Thanks for the pointers >>24647 Thanks i'll check it out

https://www.youtube.com/watch?v=OwqQD1sF-yo windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v647/Hydrus.Network.647.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v647/Hydrus.Network.647.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v647/Hydrus.Network.647.-.Linux.-.Executable.tar.zst I had a good week clearing a bunch of small jobs. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights In a taglist copy menu, if the subtag(s) have whitespace, you now get an option to copy with underscores. e.g. if you see 'blue eyes', you'll now get an additional choice to quickly copy 'blue_eyes'. Hope this makes it easier to paste tags back into a web search! Duplicate auto-resolution rules now have separate pause status and operation mode. It was a mess, trying to combine them. I've also added a pause/play button to the rule list in a normal duplicates page. There's also a new 'near-perfect jpegs vs pngs' suggested rule, and there's a new simple hardcoded comparator for 'A has higher jpeg quality than B'. Advanced: If you use multiple local file domains and move files around for processing, 'file import options' now have a checkbox to explicitly turn on/off putting files into the set import destination(s) if they are 'already in db'. This behaviour has proved annoying in the past, so it is now default off. next week I'm on a roll with duplicates auto-resolution, and there's only about a half-dozen jobs to clear. I'd like some sort of tag comparator tech.

>>24661 >larping as an oldfag when you cannot recognize older memes >going full nu/pol/ Just hang yourself already, you fake. You rapefugees are the cancer that ruined old 8chan.

>>24635 boywife

>>24663 You could do it with the API.

>>24666 >the command palette now displays and searches long page names without 'eliding...' Thank you!

Been getting this error message when trying to upload tags to the public repo for a week or two, at v646 and now v647.

>>24672 Now that i'm loooking into it, i see there is processing that needs to be completed in the public tag repo service. This used to happen automatically. Did a threshold get changed for when this gets processed?

>>24663 Assuming you already have a URL class for youtube, this parser will apply those tags if you do rightclick -> urls -> force metadata refresh. If you already have a youtube parser, you can transplant the logic from this one. It's just two rules.

East Coast Racer is pretty sick, nice find Hydev. If you like that song you may like the band Thank You Scientist. They don't have any real slow songs but the slow parts of their songs sound like ECR. I can't recommend Maps of Non-Existent Places since the mixing is ass (at least on the version I have), but anything after that is gold.

I'm on v646. just wanted to report that the new sparse search mode for potential dupes seems to speed up how quickly it finishes. It's a bit hard to remember, but I feel like it's finally back to around the speed it had a few months ago before you did that rework of potential duplicate searches when you were getting into the auto-resolution work. The only complaint I have is the minor strangeness of how Hydrus sort of locks up the page until it finishes when it switches to that sparse searching mode. It makes it look like Hydrus froze for a few seconds.

On linux ran from souce, bazzite anon from earlier. Using DISPLAY=:0 WAYLAND_DISPLAY= mpv generally works for me, but sometimes it lags massively or simply won't open from the clicking on thumbnail from the main window or going to the next video in the media viewer.

>>24680 okay now i'm not sure what to think as on a relaunch it's working fine even after scrolling over hundreds of videos not sure what's going on

>>24632 There's a bunch of checkboxes under options->media viewer hovers to turn things off. I'm sorry for the clutter and bad positioning there; most of my media viewer stuff is still running on hardcoded stuff that I hacked together years ago. I'm planning a complete revamp to have a proper Qt layout, but when the media viewer gets too thin, my code has bad recovery and your top-right hover is going to offset much lower, like you've seen. I'm not sure about it not showing up at all, but maybe my code in a really bad spot puts it offscreen or something. I'll fix it eventually and hopefully clear out some flicker along the way. The bad 'solution' for now is to make the window wider. >>24637 Ok, thanks, that's good information too. I've said in some other places I have a plan to revamp my profiling code again. Sometime in the new year I hope to have a fairly universal solution that will, fingers crossed, capture everything regardless of the source and display it in a sane way. >>24643 >>24664 Great, I am glad your db is better. Let me know if you have any more problems. It is a shame you had trouble with Ubuntu--I build the release on github Ubuntu, so I'd have thought it would be fine. Maybe it is another newer wayland/x11 thing. >>24645 I did some research into rendering mpv to an OpenGL context and it seems easier than I feared. I'm going to play with it in 2026. After that, I'll likely do some work with the Qt media player. I'm not averse to a VLC embed or whatever in the same vein, but the trick that makes it possible for mpv is this library which does all the heavy lifting: https://pypi.org/project/python-mpv/ I know there are some vlc wrappers that make it fairly pleasant to embed in various other UIs, but I simply haven't tried it for python Qt so I don't know how actually doable or performative it is. >>24663 Not yet inside hydrus, but a future version of the objects and panels you see in the sidecar import/export UI is going to handle arbitrary metadata-to-metadata translations within the client. You'll eventually be able to suck in an URL and convert it to a tag or whatever. Otherwise, some hacks like this >>24676 are probably an appropriate solution for now. Most sites deliver the same ids and stuff of their URL in the content, so there's usually a way to pull the '123456' id or whatever with a normal parser if you want to put the effort in, or just attaching a static 'site:youtube' tag with something that parses something arbitrary but stable and regex replaces it with 'site:youtube' etc..

>>24672 >>24673 We've had this test for a while, I guess a year or more. You need to be synced to within two weeks ago to upload more stuff to a tag repo. We found that many clients were committing old data over and over, or petitioning data that was already cleared, simply because they hadn't caught up. I assume you are looking at your progress bars in services->review services? Is there any reason you might have dropped behind? Maybe not run the client for a couple weeks? I know there have been some gonzo ~10 million row update days recently, so maybe you overflowed trying to catch up, but this error rarely happens. Any chance you are/were paused under services->pause->repo sync? In any case, the solution here is to let your processing catch up. That should happen naturally, if you have not had much chance recently, but if you haven't changed your routine, it sounds like your client is processing slow. If you think you are running slow, can you check your 'db_dir/client - date.log' file and ctrl+f for "definitions at" and "content rows at"? You should have a bunch of status lines about how fast your processing is going. A decent machine with a good NVME should normally see 1-5k rows/s when fully synced. If you are regularly seeing 100 rows/s processing speed, it might be something is running bad, and I'd be interested in investigating it. >>24677 Thanks, I'll check them out! >>24678 Great stuff, that's exactly what I was aiming for. Same on my IRL client when I search for a single creator or some system:hashes. Yeah, I want to explore having a nicer label there, rather than just pausing on the '4,000/800,000...' line. It is a surprisingly tricky situation. Unfortunately, the guy I specifically focused on writing that mode for gets terrible performance and 30s+ search times, so I'm not amazingly happy. What sort of searches are you doing where you see this guy kick in? Is it exact pixel dupes, one or two searches, what max distance? >>24680 >>24681 I recently added 'LEGACY DEBUG: legacy mpv communication method' and 'TEST: destroy/recreate mpv windows...' checkboxes to options->media playback, and in the near future I'll be adding some stuff about recycling mpv windows within a particular media viewer instance. You might like to try flipping those on and off to see if you can improve or worsen your mpv behaviour here. Best to restart the client fully when changing one of these values to get a truly fair test.

hi. Hydrus was unable to make a thumbnail for this video. the error in the log said to send it to you so here it is. (nsfw) I can't think of any other relevant info other than that I'm on version 646 and running from source

>>24686 Thank you for this report. Can you hit help->about and tell my your ffmpeg version? This imports and renders ok for me. If your package manager has a newer version of ffmpeg, that may be the best solution here, but I'm interested to know if you cannot for some reason update, or if a new version of ffmpeg is causing the issue.

Just realized I imported a hundred of different artists with >creator:: instead of >creator: is there maybe a plain json or txt somewhere i can batch edit to fix this?

>>24684 >There's a bunch of checkboxes under options->media viewer hovers to turn things off Thanks, I turned "Do not pop-in top-right hover-window on mouseover" and now it does show the pop-up. I guess I thought that disabling it would make the background version clickable and then promptly forgot about it. Gotta stop tweaking stuff when tired or something.

>>24692 If you still have the files unimported you can reimport them with creator and then delete every instance of creator:. If you deleted them on import you could also export all with the creator: namespace as the filename and then reimport them that way, e.g. creator::asanagi gets exported as asanagi[spacer][hash] or something and reimported as creator:asanagi. If you use a consistent method to separate namespaces like a string of #!+$!@ (aforementioned spacer) you can just regex the name up to that part and discard the rest. Hydrus will see it has the same hash and merge the tags together. There's no way to mass edit namespaces yet unfortunately. Don't delete the files while exporting.

>>24694 I guess I'll export and re-import. Kinda fucked though because I deleted them off the drive since it was around 60k images I've been keeping on Pixivutil for a minute now. Thanks.

>>24685 >>24673 >In any case, the solution here is to let your processing catch up. That should happen naturally, if you have not had much chance recently, but if you haven't changed your routine, it sounds like your client is processing slow. If you think you are running slow, can you check your 'db_dir/client - date.log' file and ctrl+f for "definitions at" and "content rows at"? You should have a bunch of status lines about how fast your processing is going. A decent machine with a good NVME should normally see 1-5k rows/s when fully synced. If you are regularly seeing 100 rows/s processing speed, it might be something is running bad, and I'd be interested in investigating it. Not sure what happened with my client, looking through the logs, i see definition processing at ~1-2K rows / second up to 10/24. Then no processing until 11/13 when I kicked it off in services -> review services -> processing sync. Since 11/13, it's been running daily again. I'll keep an eye on it and now no where to look. Thanks for all you do, devanon. I probably only use about 2% of what it's capable of, but it's been a constant companion on my workstation for almost 10 years now. Had to rebuild my win machine after a ransomware hit in 2018 (poor security hygiene, I had RDP open at home, don't do that). I had been using hydrus for at least 2-3 years before that. I am a proud pay-pig and chip in a few bucks every month via patreon. I do hope they aren't skimming too much off the top. </blog>

How do I clear out "deleted mappings" from a file? I don't want every stupid typo I make to hang around and taunt me forever.

>>24680 >>24681 Okay, now I can't launch Hydrus with the DISPLAY=:0 WAYLAND_DISPLAY= variable. Getting XCB firing isn't working cause of qt.qpa.plugin: From 6.5.0, xcb-cursor0 or libxcb-cursor0 is needed to load the Qt xcb platform plugin. qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem. thing even though I already installed the equivalent packages for fedora (I'm using a fedora distrobox for hydrus) and had this working before No clue what changed since I last used hydrus with the above setting yesterday

>>24694 >There's no way to mass edit namespaces yet unfortunately Different anon here. I'll keep praying for it.

I had a pretty good week adding more duplicates auto-resolution tech, fixing a couple bugs, and smoothing out some maintenance scheduling. The release should be as normal tomorrow.

>>24698 tried making hydrus in a linux mint distro box, can't get it running without using wayland at all, thowing up the same error as here can't seem to get xwayland to fire though I was able able to get (i went with mint as it was easy to use hydev's commands from the help) At a loss here, shit sux

>>24705 tried rebasing to an older bazzite image from when I could boot with DISPLAY=:0 WAYLAND_DISPLAY= since I was able to boot with it up until fucking sunday Didn't work

https://www.youtube.com/watch?v=YvhFbRbPeko windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v648/Hydrus.Network.648.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v648/Hydrus.Network.648.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v648/Hydrus.Network.648.-.Linux.-.Executable.tar.zst I had a pretty good week fixing some things and working on more duplicates tech. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html highlights Just a quick note--I have disabled the Animated JpegXL detection, since we discovered that, for some JXLs, the ffmpeg call to detect animation was getting stuck in an infinite loop. I'll fix it for next week! After I unsilenced the error reporting for certain slow jobs last week, we saw that one of the menu-population routines had been having a problem all this time, and in fact it was a source of program instability. Sorry if you saw an ugly long traceback after quickly right-clicking some thumbnails last week--the bug is fixed. I tweaked system:duration and system:framerate a little. Duration now allows '=' and '!=', if you want to search it down to the millisecond. Framerate no longer allows '<=' or '>=', since the calculation behind the scenes is naturally imprecise at the moment (what you see as 30fps might actually be 30.003fps). Framerate UI has a couple notes about this. I'm thinking about reworking how framerate is calculated behind the scenes so we don't have to deal with this. I suspect it will be 'if fps is lower than 10, round to nearest two significant figures, otherwise go to nearest integer'. Let me know if you have opinions about this! Duplicates auto-resolution has many new 'A or B' comparators: tags, namespaces, wildcards, system:number of tags, system:duration, system:framerate, system:number of frames, and system:has audio/forced filetype/transparency. The autocomplete dropdown where you enter all this is cleaned up and collapsed into the same stub entries (like 'system:file properties') you see in a normal file search. There's also 'system:framerate' for 'A vs B' comparators. Some users are playing around with video in auto-resolution, so we'll see how it goes. next week I want to jump on this animated JXL issue and harden up all my ffmpeg calls. There should be more stuff that says 'hey if ffmpeg took more than x seconds, consider it stuck and kill it' etc.. With all the comparators this week, I am happy to say I am running out of work to do for auto-resolution. Next thing is efficient counting updates so we don't see so many slow re-counts. If I can get that done, then it'll be modified date merging and a help document brush-up and we are done. I am confident I can clear it by the end of the year.

>>24687 I checked and it looks like ffmpeg is out of date for me so you're probably right that that's the issue. I'll update it soon and hopefully it'll be working fine for me then. I was about to write that I think it'd be a good idea to add a suggestion to check if ffmpeg is out of date into that log message, but it looks like you beat me to it for v648

>>24714 >>24687 oh and in case you care about my actual ffmpeg version, it's 7.1.2

>>24705 >>24706 Ok, somehow got X11 working again on my Fedora setup? Not really sure how, saw a post somewhere mentioning libxkbcommon, got it nd some related packages and it worked after, but couldn't replicate with my Mint setup, so I can only assume there was some little change I missed. Glad to have it working at least.

>>24713 How about showing the video codec?

>>24474 Same issue, except I don't know where to change all the downloader components so I hadn't even gotten that far. The latest Kemono downloader is from 5 months ago, and it says kemono.su still. I imagine it might take more than just changing the url in downloader components to get it working.

Anyone know how to get the gelbooru downloader to work properly? It misses images. I tried logging in, that didn't fix it. Someone said sending HTTP headers? How can I do that? Example tag: michiyoshi >search with filters: 4 results >search without filters: 42 results >gallery download in hydrus: 4 results

>>24474 >>24724 There's some instructions to get things kind of working half-assed for Kemono at https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/issues/192 You'll need to do a lot of manual changes. And even then, parsing galleries and updates don't work automatically. But this will at least let you import individual posts manually. Combine that with something like Link Gopher to mass grab post URLs and it's mostly tolerable. https://sites.google.com/site/linkgopher/ But with it having been 3 months without a fix for one of the largest sites... I'm not getting my hopes up and have been trying (and failing) to figure out how to parse Gallery-DL's json metadata into something importable.

>>24713 >>24704 >more duplicates auto-resolution tech Always love that, thanks.

>>24631 >>24640 (me) I just updated from v646 to v647 and just wanted to say that it looks like you fixed it. Thanks!

>>24685 >What sort of searches are you doing where you see this guy kick in? Is it exact pixel dupes, one or two searches, what max distance? The searches are often involving a lot file properties like "has duration" or "filesize >= X" or something like that. It'll often be a dupe pair search where File One and File Two have different file searches, and it'll sometimes involve namespace inclusions (namespace:*) or exclusions (-namespace:*) as well as exclusions of specific tags and url predicates like for certain domains. For exact pixel dupes, I basically always have this set to "must not be pixel dupes" for all of my dupe searches. The max distance will most of the time be 0, but sometimes I set it to 8 for searches where I wanna make sure that everything is cleared out. 8 is the max distance that I have Hydrus set to search at.

Dumb question but is it safe to switch from .zip install to .exe on Windows? I used to use the .exe install but switched to zip install for reasons I forgot and now I want to switch back as its way better at warning me for things before it installs them.

>>24692 >>24694 >>24695 >>24699 One day I'll have this nicer. Check the Client API if you have a very big job here: https://hydrusnetwork.github.io/hydrus/client_api.html It takes some work to set up, but you can do something clever like 'every time there is an x tag, remove it and replace with regex( x ) tag' with a python script of like thirty lines of code. >>24696 Thanks, interesting. Let me know if it keeps happening, and we'll look closer. I'm glad you like my software! I appreciate the support, and I'm happy I can help a guy manage a whole load of stuff a bit easier. >>24697 You know, I don't know if there is a nice and easy way to do it in the normal UI. There should be, since the content pipeline supports it. The completely ass-backwards way, if you really want to get rid of something, is: - MAKE A BACKUP - open the file in manage tags - hit cog icon -> migrate tags for these files - source is the tag service, 'deleted' tags. filter for the lone tag if you need it, otherwise all deleted tags - dest is the same tag service, action is 'clear deleted record' - fire that job off - see if it worked. if it didn't restart the client and check again - let me know how it went I just tried it and it worked. But I've been doing some migrate tags stuff myself recently, and some of it is acting funky. I'm going to write some more unit tests to harden it up. >>24698 >>24705 >>24706 >>24716 Sorry to hear about your trouble. If you can think of anything I can add to the 'running from source' help regarding wayland or fedora or whatever specifically, I'm interested in the list of 'libxkbcommon' and whatever else. I can't offer much expertise, but that error about xcb-cursor0 does mention it is for Qt 6.5.0. It is an ugly answer, but I wonder if you can downgrade your Qt to <6.5? I did a little research on the 'no xcb-cursor0 despite yes having it' question and got: - might be you have the dev versions of the libraries, not the normal - might be you need more than listed (what you saw and had success with) - might be some mickey-mouse i686 vs 64 bit version of the libraries You know better than me what of this sounds plausible. Something you can _try_ is to locate the 'libqxcb.so' file in your venv, should be something like 'venv/Lib/site-packages/PySide6/Qt/plugins/platforms/libqxcb.so', do 'ldd' on that and see what it lists. Anything 'not found'?

>>24714 >>24715 Great, glad it looks that simple. Let me know if you still have any problems. I was astounded that my stock Linux Mint install did not come with ffmpeg by default. Now I have it, it doesn't seem like package updater updates it all that often. I don't know much about licensing fights, but do you know if there is some reason it isn't available by default everywhere and updated all the time? It is such an excellent thing to have on your PATH, I don't know why the problem isn't just auto-solved without user input. >>24719 I hope to have something like this in the future. ffmpeg is reliable at providing this, so if and when I have some 'EXIF and other arbitrary file metadata gubbins' storage and search, discovered codecs is something I could cram in here. >>24728 Try here: https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Downloaders/Gelbooru >>24735 Great! >>24736 Thanks. You do a wide variety of searches, so if you notice things working crazy slow, I'm interested to know if it always happens with a x predicate or whatever. I suspect the search distance may be the real killer here. It grows superlinearly in some way, so maybe I should drape this thing in a warning label or in the help docs somewhere to say 'stick to 0 when you can, don't go above 8', like you've (correctly) intuited. >>24737 Yeah. Make a backup beforehand to be safe, though. In the install step, just tell it to use your existing path. It may get pissy about overwriting files or being in a location that has stuff in it already, but you are good, you can just click 'ok' to overwrite. Once it is settled back in and added/updated any install metadata, subsequent exe installs should know to point to that location and be painless. If it goes wrong, roll back to your backup. The critical part, as always, is your db dir. If you are feeling nervous because you cannot make a backup, just install to a new location and then drag and drop your db dir from the old location to the new, and the first time you boot, it'll talk to your now-migrated db and you'll be running like normal.

Is there some way to automatically re-import tags for already downloaded subscribed images? And I mean automatically, I know how to do it manually just fine. The problem is that e621 jannoids are becoming increasingly more retarded and deleting art within days of it being uploaded for arbitrary reasons, so I've been forced to set my e621 subscription to check every day. This works fine, but frequently leaves the files undertagged, and it'd be nice to specify a "recheck tags and reimport after N days" option. In a similar vein, e621 generally doesn't delete everything about a file. They keep a "deleted" page that still features all the metadata and tags related to a file. It'd be nice to know a file has been deleted from a specific booru. It's mostly a parser problem that I could solve myself, except for the automatic after-the-fact checking. I can manually re-import everything and have the parsing conditionally add a "meta:deleted on X" tag locally, but not automatically. >>24716 >libxkbcommon That should be completely unrelated, it was my "solution" to a scrolling issue in an otherwise completely functional running from source hydrus install. I haven't really updated the virtual environment since, but the issue of sticky UI scrolling while moving the mouse cursor has persisted since. It's better than not being able to scroll at all, and it might resolve itself if I update the venv, but it might also brick itself, knowing venv.

There isn't any way to make a downloader that sends a POST request with a specific body (parsed from a preceding URL) to a JSON API, is there? All the downloaders I see have URL components as variables, while the one I want to make is a generic image_downloader.json endpoint that takes the request body in JSON form and returns an image JSON.

I had a great week fixing UI bugs, improving performance, and making a clever new way to search ratings. There's also a heap of neat UI features from a user. The release should be as normal tomorrow.

https://www.youtube.com/watch?v=_oilmHSDCRE windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v649/Hydrus.Network.649.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v649/Hydrus.Network.649.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v649/Hydrus.Network.649.-.Linux.-.Executable.tar.zst I had a great week working on bug fixes and performance improvements. There's also a heap of neat UI features from a user. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html ui features The user who has been adding rating and command paletted features has a bunch more. There's a lot, so check the changelog for full details. For ratings, we now have a really nice rating preview widget, for options->ratings where you can change the size, and services->manage services where you change shape and colour, that previews your changes live. The command palette can now show your page history and favourite searches. There's also options to limit the number of search results, for reducing lag in big clients. There's clever new zoom and image-centering options. And all page of pages now get a little ' ↓' name suffix. You can change this text, or hide it completely, under options->pages. clever ratings search I figured out a new predicate under system:rating, for when you have more than one rating service, that allows you to do 'any of x services are rated/not rated', 'all of x services are rated/not rated', and, for convenience, 'only x services of y services are rated/not rated'. If you do a lot of rating searches, check it out and let me know how it feels. There's some experimental new tech here, and the logic can get tricky, so it may need a revisit. better external program calls I greatly cleaned up how hydrus calls external programs like ffmpeg. Errors are caught better, environments are set up nicer, old handles and zombie processes are cleaned up more reliably, and any call that takes longer than x seconds is now terminated and cleaned up with an appropriate error hydrus-side. We had an unpleasant surprise last week when certain JpegXLs were causing ffmpeg to get into an infinite loop on a particular 'is animated?' test. The backstop ensures this now cannot happen again. I also instructed ffmpeg to bail out as soon as it encounters any error from now on, so the loop shouldn't happen again in any case. I don't know how prevalent small errors are on normal files, so let me know if certain types of videos stop getting thumbnails or something! better prefetching The media viewer is much cleverer about pre-rendering the previous and next items. It schedules things smarter, handles memory bumps from larger files better, and experiences less CPU-wasting churn when things are tight. Lots of details in the changelog if you care about this a lot. next week I want to focus on faster duplicates auto-resolution count tech so I can be sure this project is cleared by the end of the year.

Do you want users to report false-positives that they find for "very probably" visual dupes?

>>24759 Yes please. I am less concerned with the 'very probably' confidence, so I don't need a hundred all in the same vein, but I am happy to take examples that I can fold into my tuning test suites as IRL data.

Chromium recently announced they are adding support for JPEG-XL (.jxl) files. Seeing as most other large companies and CDNs (Meta, Apple, Adobe, Cloudinary) support this change I assume it's only a matter of time until it becomes the de facto standard. As such, what would be the easiest way to convert my current png/jpeg files into jxl without completely destroying my database? Thanks.

>>24761 Yeah, I am very enthusiastic about their decision. JpegXL winning the war is my ideal outcome, so let's see what happens. My advice for your problem is to wait. We don't have much experience doing conversions to it yet, and JpegXL has a lot of interesting new levers and dials behind the scenes. Most of the libraries like ffmpeg and imagemagick have limited support too. One thing that probably is safe, although perhaps not ideal for hydrus stuff since a file conversion for the sake of it is probably a pain more than anything, is that JpegXL can apparently hold a jpeg inside it losslessly, i.e. completely recoverable, but maybe it still saves some space? Ultimately, I expect to write tech for hydrus that will handle various sorts of file conversion that'll farm the work out to an external executable like imagemagick, with a set of params we've figured out work well, and will then do automatic duplication merge stuff on the hydrus side with the result. I hope to solve much of this problem for you in some years, and we'll know what is good in a few years too, and hopefully which format won, so I think it isn't time yet. If you are determined to experiment, I recommend imagemagick and then letting the new auto-resolution tech handle the dedupe for you.

>>24762 Alright, I think I'll try imagemagick on the files that I archive locally for my personal entertainment and keep the rest as is until Chrome (and by extension the imageboards) decide to make the change. Thanks!

Bug report. I'm having troubles trying to import a video with japanese characters in the metadata.

>>24764 this file worked fine for me in v648, so it's probably a regression unless the metadata was stripped when you uploaded it here

>>24713 >648 On the duplicate auto-resolution tab, the "pause" icon always shows pause.

>>24767 The metadata is still there but it looks like that's not the problem because I edited using VLC, but the file still fails to import. F

If Hydrus opens and a few different subscriptions are due to run, what order do they run in?

For subscription checker options, would you mind adding an option to set an interval modifier for the reactive checking check frequency? Meaning for example that you could set the interval modifier to 90% and Hydrus will check after 90% of the ordinary time that it would check has passed (so instead of checking after 10 days it would check after 9). I've found that Hydrus's reactive checking tends to be a little too slow for me a lot of the time, even when I have it set to "1 file per check". I don't want to set a low hard cap like 30 days because I have many queries, so I like that Hydrus checks differently for queries with different new-file rates. I just want a way to add a "multiplier" of sorts to each interval that it chooses, so that it can check a little more (or in some cases less) often than it normally does. If you're okay with adding an option like this, then I imagine that it would be set per subscription, so I could have a set of queries with the normal checking, and then another subscription for the queries I want to use an interval modifier for.

>>24778 afaik it's purposefully in a random order

I can't find a way to set file import options for a gallery url class. I see a way to set tag and note options under `network --> downloaders` but file options is missing from there. I wanted to set a certain gallery downloader to have different file importing options than my global default. Is there a way to do this, or do I just have to set custom options every time I download a query for that downloader? I just wanna make sure I'm not missing something

>>24753 I could have sworn I had some sort of 'only hit x post urls y days after they are added' mode somewhere, and most likely in subscriptions, but I can't find it now and looking at the code it would be a pain in the ass to do, so I guess I'm imagining that it actually happened. I was going to suggest you set up a second sub that checked things seven days later, with forced tag parsing, but I don't think it is so neatly possible. I think your best bet for now is to have a second subscription nonetheless, where it has the same queries as your primary sub but you set the 'tag import options' to 'fetch page fetch even if hash/url recognised...'. Then you have this second sub paused most of the time. Once a week, you unpause it. It works and runs fast since it knows it has those files already and is just grabbing tags on this second run. On average, you'll get 3.5 days later than publish time. Extend it to every two weeks if you want 7 days average. Not perfect, but it'll do as a hack. I just remembered that e621 include all their data in the gallery hit, too, so this'll be really super fast. >>24754 I'm sorry but the current downloader engine can't handle that yet! Try gallery-dl or hydownloader for clever stuff for now: https://github.com/hyuugi/hydownloader >>24764 Thank you for this report and the file. I'm afraid this imports for me ok here on my dev machine, which has ffmpeg 8.0. Since you are on 7.x, I am guessing that is the problem. Pic related is my output, it looks like your file is a slightly funky AVC1 (newer codec), so I can see how an earlier ffmpeg might not be happy with it. I don't see any unicode metadata in this view, but it may be there's some in a different call I make that is failing later, but I'll be that would be due to ffmpeg version too. Can you open a terminal on a directory with this file and do 'ffmpeg -i filename.mp4'? You'll get this same output, hopefully, and I expect it'll say 'no video stream found' or something useful. If you can, please try and update ffmpeg with your package manager. I don't know why, but a lot of Linux seems to just have old versions, or there's a licence dispute I'm ignorant to. If you can't update ffmpeg, if you can download a newer executable from like here https://github.com/BtbN/FFmpeg-Builds , just put 'ffmpeg' in your hydrus_install_dir/bin directory and after a program restart hopefully your help->about will show the newer version. Let me know how you get on!

>>24768 Thanks, yeah. That guy is actually a stupider 'flip pause/resume' button (since it can action on multiple things at once) and I realised I had no icon for that yet. I'll make a nicer pause/play icon to make it a little clearer. >>24778 >>24781 Check options->downloading. You can have it run random, which I generally recommend if you don't mind, since that smooths out a bunch of bandwidth and checker timing bumps, or alphabetical. Now I look, the alphabetical is pretty bad (strict lexicographic), so I'll improve that to my 'human a-z' that handles '3<11' and stuff better for next week. >>24780 I was going to say a rare 'no, sorry, that is overengineered', but what I'll do is make that 'n file per check' thing support decimals, so you'll be able to set 0.9. >>24782 Sorry, can't set file import options for a particular url class yet. The whole system is way overdue for an overhaul. I'll try and make sure you can assign file import options to url classes when I get to that--please remind me if it seems like I never got to it.

>>24785 >what I'll do is make that 'n file per check' thing support decimals, so you'll be able to set 0.9. that's completely fine. it's actually the first solution I thought of, but I don't know how reactive checking is implemented, so I didn't know if fractions of files even made sense logically. I brought up the interval modifier idea because it would just change the final time, so how checking was calculated wouldn't matter. Situations like this is why I always explain why I want a feature when making a feature request. anyway thanks! I'd love to see it some time.

Still on Hydrus 646 from source. Not sure if this is a Hydrus issue or the file itself is borked. It's an .mp4 without audio, but every time it begins playing, including pausing and playing from the middle of the video, my speakers/headphones make a clicking noise. When this first happened, it broke the audio on an rpgmaker game I had open.

>>24788 It isn't making the popping noise when played in my browser or when opened in an external media player.

>>24783 Reporting on the funky .MP4 file refusing to import. >Can you open a terminal on a directory with this file and do 'ffmpeg -i filename.mp4'? I did. See pic #1. >download a newer executable from like here https://github.com/BtbN/FFmpeg-Builds , just put 'ffmpeg' in your hydrus_install_dir/bin directory My package manager won't allow to upgrade to ffmpeg 8, so did as you said dropping the executable into the Hydrus' directory. The upgrade was a success, but unexpectedly the MP4 file still refused to import. See pic #2. Then I took the matter into my own hands and transcoded the damn file using the "Covert/Save" feature of VLC. See pic#3. The new "fixed" file imported successfully. See pic #4. Thanks devanon. I learned a couple of things.

>>24792 By the way, the terminal's output on the new "fixed" file is as shown in the pic.

>>24785 + >>24781 >Check options->downloading. You can have it run random, which I generally recommend if you don't mind, since that smooths out a bunch of bandwidth and checker timing bumps, or alphabetical. Thanks dev. I'm just using this to make my artist searches run before concept searches, it helps keep the import times in order when an artist releases a comic/series where only some files match a concept search. This option to run alphabetically and add a special leading character to the subscriptions I want running first/last is a good solution.

>>24713 >648 Hydrus Companion for Chromium worked for me in, I think, November, but now it says something like "Network error (Hydrus not running?)"

>>24783 Neither gallery-dl nor hydownloader work for my case because gallery-dl does not support the site and the very specific use case I have for it, and hydownloader fundamentally relies on gallery-dl. That I know, I'd have to compile my own version of gallery-dl with my own extractor just to save maybe a minute or two while bulk-downloading images. It'd probably be simpler to make the changes I want to Hydrus itself than to go through an extended dependency rigamarole, but again, spending hours coding just to save a few minutes of manually regexing and pasting image links for myself and only myself is ridiculous. Thanks, I'll wait for the eventual API parsing reconstruction.

While most new releases do seem to prefer .cbz, I still have a collection of old .cbr files. I could repack them, but... I was hoping adding rar support wouldn't be too much work if it could use the same algorithms. Calibre, also a python program, uses the python library unrardll for .cbr support. I'm really hoping to leave Calibre behind asap because it eagerly makes changes to imported files! Bleh!

I had a great week finishing up the duplicates database tech I had been planning. The duplicates and duplicates auto-resolution systems are much faster when reacting to file status changes, and a number of annoyances are ironed out. I also did a scatter of small misc work, but it is mostly duplicates stuff this week. The release should be as normal tomorrow.

>zooming in on pictures in preview looks like this is this user error or a deliberate choice?

>>24815 >zooming I don't follow. I don't see anything abnormal.

>>24816 the overlayed qView window is the same image also with zoom, but the hydrus media viewer specifically has extreme artifacting. fine for tumbnails and maybe the preview window but seems completely unacceptable for even basic browsing of especially pixel art when its the full media viewer.

I'm trying to import a file with a json sidecar but I get this error when clicking the "edit parsing formula" button: v649, linux, source TypeError unhashable type: 'list' File "/opt/hydrus/hydrus/client/gui/widgets/ClientGUICommon.py", line 357, in EventButton self._func( *self._args, **self._kwargs ) ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/hydrus/hydrus/client/gui/metadata/ClientGUIMetadataMigrationImporters.py", line 242, in _EditJSONParsingFormula panel = ClientGUIParsingFormulae.EditJSONFormulaPanel( dlg, collapse_newlines, self._json_parsing_formula, test_data ) File "/opt/hydrus/hydrus/client/gui/parsing/ClientGUIParsingFormulae.py", line 1064, in __init__ self._parse_rules.SetData( parse_rules ) ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^ File "/opt/hydrus/hydrus/client/gui/lists/ClientGUIListBoxes.py", line 1424, in SetData self._listbox.SelectData( selected_datas ) ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ File "/opt/hydrus/hydrus/client/gui/lists/ClientGUIListBoxes.py", line 299, in SelectData list_widget_item.setSelected( data in datas ) ^^^^^^^^^^^^^

>>24818 Thank you--another guy reported this exact same bug to me earlier this week and I just couldn't figure it out. His situation was after creating a new content parser, switching its formula to JSON, and then trying to edit. It all works fine for me and I can't figure out where this spare list is coming from. Today's v650 will skip over the error, but I expect it'll now cause a new one further into the panel since it sounds like we have some weird data getting in somehow. Can you click the 'export' button on that edit formula panel that fails and then paste the JSON here? For instance, when I was doing his work with an empty formula, I exported this: [31, 4, [[[0, [51, 1, [0, "posts", null, null, "posts"]]]], 0, "", [84, 1, [26, 3, []]]]]

>>24817 You're talking about the lanczos interpolation scaler. It often creates these ugly ring artifacts. In options --> media playback, you can switch the media viewer to use bilinear or nearest neighbor instead. For pixel art you want nearest neighbor. Side note, but I wish Hydrus had more options for scalers. I don't like Lanczos and bilinear is blurry, but I wish it had something better than just nearest neighbor. the ewa_lanczossharp filter from MPV is pretty good, and it'd be cool if Hydrus supported that.

>>24810 unrar is a bitch to work with. It uses COM files. I'm working on a new comic reader, and actually got Ai (Grok) to help me with it. My program was able to read rar files in the end, but now I find it can't read RAR5 files. So, another step to be dealt with. Unrar is just really hard to work with.

>>24810 If you're looking for a comic reader that can read .rar files, I suggest Comic Rack. I'm modeling my new program after it, because it can't do the one thing that I really want. And that is have unlimited stacks. Cyolito once told me he couldn't do it, because he had programmed himself into a corner. So, I'm attempting to get at least a basic viewer that can do that.

>>24823 >and actually got Ai (Grok) to help me with it

https://www.youtube.com/watch?v=vnIxlW2lwgg windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v650/Hydrus.Network.650.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v650/Hydrus.Network.650.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v650/Hydrus.Network.650.-.Linux.-.Executable.tar.zst I had a great week finishing the last of my duplicate database tech updates. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html search undo I forgot to mention last week that the user who sent in all the UI updates also started an undo system for adding/removing predicates in normal search pages. If you remove a complicated predicate by accident, try hitting the 'undo' menu up top--you should be able to recover it now! duplicates I hashed out a ton of clever database updates this week for duplicates. The tl;dr: is that duplicates and duplicates auto-resolution now count faster and stop doing some annoying stuff. I am now done with pretty much everything I wanted to do here, so I am going to polish the help and finally launch this system for all users properly next week. If you have been waiting to try this out when it was nice to use, please watch this space. If you have a bunch of big auto-resolution rules, they are going to do a bit of work on update this week. Shouldn't take more than a few seconds for each rule. Specifically for the work, this week I made all the incremental search/count stuff update smartly on any duplicate pair change instead of just resetting to be regenerated from scratch. So, when you have a duplicate pair panel that says '10,000 pairs searched; 523 match', resolving a duplicate or having a file import now just quickly adjusts the count by a tiny amount rather than loading up the whole thing and counting again. You can now watch the duplicates page while plenty of imports are going on and it is no longer a refresh-fest. The 'initialising' step you will have seen a bunch before now also lags the first time you do a search. Furthermore, the duplicates system now does these updates when files move between local file domains or into/out of the trash. So, moving files around in complicated search domains, or simply just deleting a file, will now trigger an instant quick count update where it matters. And, on top of that, auto-resolution rules are now plugged into this local file domain tech. An auto-resolution rule is now careful to no longer track pairs where one file leaves its search domain (e.g. entering the trash), and it does the same incremental count update. A bunch of 'why is this pair still in this auto-resolution queue/count even though it is deleted' stuff is now fixed. I also re-did some refresh stuff in the 'review pending actions' panel, so that any time you close the duplicate filter on that guy, it will now refresh the queue properly. next week Final polish and a help pass for duplicates auto-resolution, so I can launch it properly.

>>24826 >I forgot to mention last week that the user who sent in all the UI updates also started an undo system for adding/removing predicates in normal search pages. If you remove a complicated predicate by accident, try hitting the 'undo' menu up top--you should be able to recover it now! Super fucking cool.

>>24825 Lol! Grok ( and ChatGPT) are actually really good at programming, especially when it comes to C++ and Qt. You just have to keep talking to the Ai to refine the code that has been put out, pointing out the changes that you want. I could have never written what I have so far without it. Especially the COM file part for unrar.dll.

Can someone please post their http header config for hydrus? The entry under network>data> manage http headers> gelbooru header popup I found them in firefox. Pretty sure "fringeBenefits=yup" is the header I need, but it's not working no matter how I try to format the entry in hydrus. >>24828 That's cool but I always fear the idiocracy future where AI enabled jeets have no idea what they are working on and all the highly skilled professionals (like dev-sama) are pushed out of the market lol.

>>24829 I don't see a header for Gelbooru, and I do download from them. I only have headers for Sankaku and Pixiv. So glad I didn't pursue my CS degree beyond Associate. Went with Medical Technology, but ai took them both over in the end. I can't see a future with human programmers. Ai will be FAR beyond them in knowledge.

>>24829 >>24832 I worded that poorly. Can someone post their gelbooru http header config?

>>24829 >>24833 I have my benefits in the cookies. I checked my HTTP headers, nothing for gelbooru.

>>24818 >>24820 I tried with an empty DB in v649, and it also gives the error. Then trying v650, I can now open the formula editor. empty parser exports as: [109, 3, [[26, 3, [[2, [114, 3, [[84, 1, [26, 3, []]], true, "", [55, 2, [[], "my_image.jpg.json"]], [31, 4, [[], 0, "", [84, 1, [26, 3, []]]]]]]]]], [84, 1, [26, 3, [[2, [99, 1, [2, true, null]]]]]], [115, 1, "6c6f63616c2074616773"]]] I tried two different jsons, but I don't think it's detecting the raw data properly. Instead of the expected object I've seen it empty (vid) and with '0' as 'raw data', nothing like the files I put in. If I give it a malformed json, the 'raw data' is a JSONDecode Error, but it does display the expected json after the error text...

>>24826 So I updated from 647 to 650, but the upgrading screen was taking longer than it should have, I SIGTERM'd hydrus and restarted it and it seems fine, there's nothing wrong in the logs. Anything I should look at to make sure I didn't subtly break something?

Are there any working nitter and/or bluesky downloaders? The nitter one in the cuddlebear repo seems to silently block Hydrus, and trying a few other instances they either cuckflare block or think Hydrus is an AI scraper and block it. Or maybe someone here has a gallery-dl config that can make a json sidecar with all the metadata Hydrus supports, that would be great.

Is AI tagging viable yet?

>>24834 You're able to download all site content? I'll try importing cookies

>>24844 I messed with the WD14 tagger in comfyUI and did image > tagger > text to image and its pretty accurate. The tuning is in the tagger strength. Image upscale is another good use of AI, but it would obviously double your DB size.

Is there a hydrus companion alternative? It has a bad memory leak in firefox.

>>24841 Hydownloader can grab twitter & bluesky. >>24844 https://github.com/Garbevoir/wd-e621-hydrus-tagger I use this, it's ok, it's not great for weird things like 4chan memes but it's pretty good for stuff you'd find on a booru. It was able to tell pic1 was aged down Kiara but it thinks pic3 is a Pokemon creature. Still beats having no tags.

>>24849 Just a heads up, there's been a v3 and a v3 Large of WD v1.4 ViT that's been out for quite a while now. Both are better than v2 imo (especially v3 large). https://huggingface.co/SmilingWolf/wd-vit-large-tagger-v3 https://huggingface.co/SmilingWolf/wd-vit-tagger-v3 I've been using this for Automatic1111 which should auto download those newer models (and older one) for anyone that wants a GUI: https://github.com/67372a/stable-diffusion-webui-wd14-tagger

why has hydrus stopped downloading all images from a blueksy link? it only downloads the first one for me now

>>24850 Nice, what specs do I need for the large? I run AMD so I don't get CUDA acceleration, if that matters.

>>24852 Not sure tbh, but it should not be very demanding. I've got a 12GB RTX 3060 which is overkill for such small models. v2 and v3 are both about the same size at 378MB, so you should have no issues running standard v3 at about the same speed since you're already running v2. v3 Large is a little over 3x the size at 1.26GB. That's still quite small as far as models go. It will be slower to process, but the trade off for better tags is worth it. You can probably run it no problem. v3 Large took an average of 4.25 seconds per image over 2,973 images (or 3.5 hours) on my machine earlier today. Pic related.

>>24852 >>24853 D'oh! Meant to add this. Here's how v3 large tagged that Kiara pic. Unfortunately, I'm not aware of any models that handle IRL photos very well. They're all trained off boorus and similar sites because the images are already part of a huge reliably tagged dataset.

>>24852 >I run AMD so I don't get CUDA acceleration you technically do through rocm but it's not that great

Mr. President, a second * -> clothing:* sibling has hit the PTR.

Do you guys use hydrus for all of your image files or just chinese cartoon drawings?

>>24857 All image files, video files, and even some documents and zips that aren't ever edited. At some point, I'll stick all my flash files in there too.

Put your receipts in hydrus right now, Anon, then we'll talk about your washing machines manual.

>>24788 Thank you for this report. I am not an expert at this, but I know that some Linux sound systems/driver versions don't like the mpv embed and its default settings for whatever reason, and this can cause the pops and so on as some part of the sound driver clicks on and some filter initialises etc... Please check the 'mpv.conf' in your db directory. If any of the audio normalisation stuff is on there, comment it out. I know some users have had luck with one or both of these lines added, too: ao=wasapi audio-fallback-to-null=yes There's a huge master doc of all this here: https://mpv.io/manual/master/#options-audio-exclusive , which you can ctrl+f if you like. An AI chatbot knows more about mpv settings and whatever your audio system is better than I do, though, so it might be worth asking one. If you find a good combination that fixes the issue, please let me know and I'll fold it into our help for the next guy! >>24792 >>24793 Thank you, this is super interesting. Can I ask you to zip that file and post it again? I think I'm correct that 8chan strips some metadata from images, so I wonder if it did the same to this video, such that I was testing a clean version of the video and thus had no problems. If you can zip it up, we'll guarantee what I test is byte-accurate to what you had. Or if you can point me to the same URL source. >>24803 I'm afraid I don't write the companion, so I can't offer a lot of help, but if you open up the client api tab in services->review services, and you hit 'open client api base url', do you get a clean landing page in your web browser? Should be an ASCII lady and some version info. If your web browser can see the API landing page ok, no weird certificate errors or anything, I'd imagine the add-on can see it too, and vice versa. >>24805 Damn, that's a shame. I cannot promise POST support any time soon on my side of things, so my guess is the most pragmatic solution here would be to figure out an entirely custom 50-line python script that just did the downloading how you wanted to a folder somewhere, which you then import. I'd like to figure out an 'executable manager' for 2026, which will open some doors here, too. We might be farming out URLs to external exes/scripts before long. >>24810 >>24823 Thanks, that's interesting. I never knew Calibre was python. I have it on Linux, I wonder how (if?) it handles rars for multiplat? I'll look into it. You probably know, but I support zip so easily because python supports it out of the box. I can extend to cbr/cb7 super easy if we identify nice pypi packages for it; I haven't looked properly though. EDIT: Just did some research, this may be the ticket: https://pypi.org/project/archivefile/

>>24815 >>24816 >>24817 >>24822 I was talking with some guys the other week about this broad subject, and I think one of the next pushes for the media viewer is going to be it understanding that images can have multiple ways of rendering, and then I'll add in-viewer controls to change the current interpolation method live, including shortcuts. I don't like how this is buried in the options dialog and really want finer controls here, and maybe in a couple of iterations we could even have my new MetadataConditional object apply a rule like 'if image is a png with "pixel art" tag, use nearest-neighbour'. As for the available algorithms, we use OpenCV to do image resizing for human view right now, and I just copy what they offer: https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga47a974309e9102f5f08231edc7e7529d https://docs.opencv.org/3.4/da/d54/group__imgproc__transform.html#ga5bb5a1fea74ea38e1a5445ca803ff121 We'll likely migrate this stuff towards Pillow, which seems to be roughly the same suite: https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Image.resize https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.Resampling We _could_ write support for a new interpolation manually, but it'd be a bunch of work and I imagine performance would be pretty trash since it'd be pure python, and my shitty code, and anything we wanted to add would be clever stuff anyway. That's not necessarily a reason not to do it, but I can't promise much. >>24835 Thank you! I will investigate this carefully this week. >>24839 Yeah you should be good. It was probably it updating the auto-resolution rules, the stuff in your log there where it says 'during file domain resync...'. Did the splash window say something like 'updating db to v650' and 'making auto-resolution rules smarter...'? For my IRL situation, a 500,000 duplicate-pair client with 5 rules or so on snappy hardware, it was about 45 seconds of work. Looks like yours was taking much longer, about 45 seconds each. I often write more detailed 'updating rules 4/5' kind of progress when I expect updates to take a long time, but most users were also fairly quick--do your rules have like 3 million pairs each, by any chance? It looks like the SIGTERM triggered a clean 'fast shutdown' and everything was saved ok, even if it didn't clean up the durable temp database nicely. You might like to go into your 'auto-resolution' duplicate files tab and hit the cog icon button and do 'maintenance: resync rule file domains'. It'll take 5 minutes again, but let's double-check all that applied ok and nothing was missed. Just for interest, generally speaking, a SIGTERM should never break anything in hydrus. I've never seen it done during startup like that, but I'm happy it ate the signal and seems to have dumped out as soon as it started. Interesting that it never printed the 'updated to v650' message. Anyway, after SIGTERM, it'll wind itself up and it might be rude about some updated window width saving or something, but things are ok. A SIGKILL is basically the same as a crash, and it won't ever malform your db, but it can, if done in a very busy period, cause some desync between the four database files that can cause some headaches, but I've got repair code for most of those situations now. >>24844 >>24846 >>24849 BTW as a side thing here, I'm going to work on more integration here in 2026. To answer the first question, my overall view is 'this tech absolutely works, it is broadly magical and cool, now we just have to integrate it nicely'. I've got a project started to play around with CLIP in 2026, which is basically AI searching using models like this. I am planning not to import pytorch to hydrus, nor do any AI computation on our process, but with my new exe manager, we'll be asking external exes/scripts for CLIP vectors and tag suggestions, and it is the responses that I'll be dealing with. Turns out there is a nice SQLite thing we can add that allows for fast CLIP vector search, so we might just have 'show me images like this' soon. I remain very enthusiastic about all this, and now it is time for me to do lots of work to make it not so much of a pain in the ass to get it into hydrus. I am also keen, if we can get nice data suck working, to then very hesitantly play around with maintenance workflows for training new models too. The grand dream of the PTR has been to train user-specific models on our interesting corpuses of data, on our own CPUs, and it seems like we are turning the corner of this being viable.

>>24851 I haven't changed much recently with the downloader engine, so I think probably bluesky probably changed something their end and whoever made the hydrus bluesky downloader needs to update their parser. >>24857 I use it for pretty much everything except comics/manga and longer-form video like anime or films or whatever. Basically anything that is an individual file that doesn't have a canonical title that works really well as a filename. There's some other exceptions though. I've got a ton of short vids, 3-10 minute vtuber clips and stuff, that I'm debating what to do with. They've been piling up in a 'process these later' folder for years, and I'm not getting around to it. It is the same problem I had with images, years ago, so I wonder if I'll just drop these things in hydrus eventually, and if in eight years some video ai tagger sorts them all for me.

>>24863 >>24858 How do you go about tagging reaction images that aren't simple emotions?

>>24867 What do you mean? You want me to use your pics as examples?

>>24867 >>24868 I already have three of these tagged in by database too.

>>24868 I mean stuff with text I guess. Not angry miku, disgusted loli, etc. >What do you mean? You want me to use your pics as examples? Sure.

>>24870 >Pic 1 Besides the biggest superficial elements like, <1boy, solo, afro, mustache, sunglasses, race:black, text:nigger (because it helps to seperate heavily politically incorrect stuff like this) I could broadly categorize that as <emotion:approval. >Pic 2 <meme:tldr >Pics 3 through 5 Broadly they can all get the tag "emotion:disaproval". Specifically for each one I'd put down for each >Pic 3 <meme:autism, imageboards, meme:how could this be happening to me >Pic 4 <insult:kill yourself (kys), wink >Pic 5 <Kemono:sheepgirl, meme:were it not for the laws of this land, pointing weapon at viewer. I'm quite autistic, so I'd tag much more than this, but these are the bits that stand out most to me. For template text memes that have had many iterations likes pics 3 and 5, I find labeling the meme by some core snippet of text that appears in most or all versions tends to be a good way to find it again. I do all my tagging manually though, so I'm probably not the best example to follow. I also never use unnamespaced tags, and have just ommitted some namespaces here for simplification.

>>24870 Oh, I also have pic 1 tagged as, >origin:flash game, origin:source material, pixelated, dithering

How do I group images? I have a ton of short unsorted pixiv and twitter comics and it would be nice if I could link them together.

>>24857 digikam seems better for non pre-tagged image content

>>24867 >How do you go about tagging reaction images that aren't simple emotions? NPC detected.

>>24873 >How do I group images? Check my system out. It is not the easiest one, but works wonders for me. For example searching for : item number:017,412 will fetch all the items belonging to it: "item number:017,412-1" and "item number:017,412-2".

>>24867 >How do you go about tagging reaction images that aren't simple emotions? Check this out.

>>24861 >Can I ask you to zip that file and post it again? This site won't allow me to post zip files, but the video URL's is https://www.youtube.com/watch?v=78Rlj5gTG0Y By the way, I just downloaded it and ran "ffmpeg -i" and the Japanese metadata is indeed there, also I tried to import it but it failed again.

>>24857 Yeah it was "stuck" on "making auto-resolution rules smarter". Most of my rules are pixel match duplicates, most didn't match the search though, not sure if that matters. Yeah, I know SIGTERM is the "asking nicely" part of shutting down a program, I always try it before a kill signal. Glad nothing should be broken but I'm running the resync right now just to be safe.

>>24329 late but have you tried deleting the rule34.xxx parser and using the gelbooru 0.2.0 file page and gelbooru 0.2.x gallery page parser for the r34 file and gallery page url classes? i was having the same issue but this seemingly fixes it

I'm about to just make a second file service and put well-tagged booru images in one and untagged junk in another. I have literally thousands of barely tagged images from Pixiv... I know about AI tagging but unless it has changed in the past few months having lots of wrong tags pisses me off more than a handful of correct ones.

>>24911 It depends on how detailed you want to do your tagging. If you set the thresholds fairly high waifu diffusion has pretty high accuracy. Not perfect but good enough. With the Automatic1111 tagger in batch tagging mode you can actually go in and manually curate the tags for each individual image. Tags are auto-selected according to the thresholds you have set and you just toggle them on/off as desired. Really just depends on how much work you want to put into it. Pic related of AI tag curation. I keep a fairly low threshold because I'd rather have a bad tag than miss a good tag. What's selected here is what my result threshold finds. There's been no manual changes. Personally, I don't curate by hand very often, the raw results for WD 1.4 v3 Large and Z3D-E621-Convnext are sufficient for my needs. I just straight export the tags and import into hydrus. I've been using this one for a couple years now. I keep Automatic1111 around specifically for this. https://github.com/67372a/stable-diffusion-webui-wd14-tagger Also of note, I have my parents and siblings for my AI Tags service synced with the PTR. So many of these tags will get sibling'd away to PTR standard tags.It helps a lot with consistency.

Is there a way I can check which tag in a given namespace has the most files? Like the most populated character, series tags, etc.

I had a great week polishing off the duplicates auto-resolution system for a v1.0 that I am happy for all users to try out. There's also some bug fixes and quality of life work, and neat new options search tech. The release should be as normal tomorrow.

>>24917 search for the namespace and scroll through the page tags

https://www.youtube.com/watch?v=Z0wL6DXIvoc If you are into New Vegas, I really recommend the vid this week, just in case you missed it. Essentially an "Astartes" situation for Fallout that just released. Skip it if you dislike violence. windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v651/Hydrus.Network.651.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v651/Hydrus.Network.651.-.Windows.-.Installer.exe linux tar.zst: https://github.com/hydrusnetwork/hydrus/releases/download/v651/Hydrus.Network.651.-.Linux.-.Executable.tar.zst I had a great week finishing up the duplicates auto-resolution system for everyone to try out. There's also a cool new options-search from a user. Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html non-auto-resolution The user who has been sending in UI features has sent some more cool stuff in. Most importantly, the options dialog now has a search box! It searches all the text strings in the dialog, and when you select one, it'll take you to that page and highlight the text. It has to be a little hacky because the underlying dialog is a little jank, but this is generally really neat and a great practical solution. Also, the command palette has home/end, page up/down, and wraparound tech, and favourite search results now open a new page by default. duplicates auto-resolution So, I have been working on this system for most of the year, and it is now feature-complete for a v1.0 launch. I am happy for all users to try it out. Hydrus now has the ability to recognise and automatically clear out common types of duplicate files. The rules that make this work are completely user-customisable, and can get very complicated, or you can just follow a quick 'tl;dr' guide and be set up in three minutes and never touch it again. If you have never looked at the duplicates system in hydrus at all, please have a look at the updated basic help before you try for the auto-resolution: https://hydrusnetwork.github.io/hydrus/duplicates.html If you want to check out auto-resolution, please start at the updated help here: https://hydrusnetwork.github.io/hydrus/advanced_duplicates_auto_resolution.html And let me know how you get on! There's lots I can still do here, and I'll be folding it into normal occasional weekly work hereon. next week I have two more work weeks in the year, and I'd like to just do some simple cleanup work so I can take Christmas off and not accidentally leave something broken. Now I'm done with a huge project, I similarly want to broaden my focus for a bit. I expect to keep it to misc small jobs for the first three or four months of 2026, rather than jumping straight into the next big expansion.

>>24933 >video Based.

Would it be possible to add a File Properties: Has Alpha Channel option? I know there is a File Properties: Has Transparency predicate. It's not good enough. It's heuristic. I want something dead simple that only checks if a non-trivial alpha channel exists in the image. Why? Because people have been posting AI-generated images that, instead of embedding the metadata as EXIF in .jpeg or in extTEXT within the .png, they encode it as 254 binary pixels in the alpha channel of the image. This bypasses cuckchan's metadata stripping, as well as here. It, however, doesn't show up with File Properties: Has Transparency. I added an image example. It has no EXT metadata, yet can be read by Stable Diffusion versions (specifically reForge). These images are particularly annoying to tag because the only way to figure out whether they do, in fact, have metadata is to drag them into Stable Diffusion one by one. Dumbly checking for the presence of an alpha channel would help a lot.

When tag presentation shows extremely long tags, it's useless. There should be a way to limit the length of some of them.

I seem to have lost thumbnails somewhere, a lot of them. How can I tell Hydrus to scan through all the thumbnails, and regenerate ones that are missing?

Once every 2 days or so, I get this error message in the log: v651, 2025-12-11 17:42:51: Traceback (most recent call last): File "/home/user/hydrus/hydrus git/venv/lib64/python3.13/site-packages/twisted/python/threadpool.py", line 269, in inContext result = inContext.theWork() # type: ignore[attr-defined] File "/home/user/hydrus/hydrus git/venv/lib64/python3.13/site-packages/twisted/python/threadpool.py", line 285, in <lambda> inContext.theWork = lambda: context.call( # type: ignore[attr-defined] File "/home/user/hydrus/hydrus git/venv/lib64/python3.13/site-packages/twisted/python/context.py", line 117, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/home/user/hydrus/hydrus git/venv/lib64/python3.13/site-packages/twisted/python/context.py", line 82, in callWithContext return func(*args, **kw) File "/home/user/hydrus/hydrus git/hydrus/client/networking/api/ClientLocalServerResourcesAddURLs.py", line 140, in _threadDoGETJob url_statuses = CG.client_controller.Read( 'url_statuses', normalised_url ) File "/home/user/hydrus/hydrus git/hydrus/core/HydrusController.py", line 696, in Read return self._Read( action, *args, **kwargs ) File "/home/user/hydrus/hydrus git/hydrus/core/HydrusController.py", line 214, in _Read result = self.db.Read( action, *args, **kwargs ) File "/home/user/hydrus/hydrus git/hydrus/core/HydrusDB.py", line 1269, in Read return job.GetResult() File "/home/user/hydrus/hydrus git/hydrus/core/HydrusDBBase.py", line 343, in GetResult raise e hydrus.core.HydrusExceptions.DBException: URLClassException: Invalid IPv6 URL Database Traceback (most recent call last): File "/home/user/hydrus/hydrus git/hydrus/client/networking/ClientNetworkingFunctions.py", line 565, in ParseURL return urllib.parse.urlparse( url ) ~~~~~~~~~~~~~~~~~~~~~^^^^^^^ File "/usr/lib64/python3.13/urllib/parse.py", line 395, in urlparse splitresult = urlsplit(url, scheme, allow_fragments) File "/usr/lib64/python3.13/urllib/parse.py", line 514, in urlsplit raise ValueError("Invalid IPv6 URL") ValueError: Invalid IPv6 URL During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/user/hydrus/hydrus git/hydrus/core/HydrusDB.py", line 877, in _ProcessJob result = self._Read( action, *args, **kwargs ) File "/home/user/hydrus/hydrus git/hydrus/core/HydrusDB.py", line 952, in _Read return self._read_commands_to_methods[ action ]( *args, **kwargs ) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "/home/user/hydrus/hydrus git/hydrus/client/db/ClientDBFilesMetadataRich.py", line 166, in GetURLStatuses search_urls = ClientNetworkingFunctions.GetSearchURLs( url ) File "/home/user/hydrus/hydrus git/hydrus/client/networking/ClientNetworkingFunctions.py", line 415, in GetSearchURLs p = ParseURL( url ) File "/home/user/hydrus/hydrus git/hydrus/client/networking/ClientNetworkingFunctions.py", line 569, in ParseURL raise HydrusExceptions.URLClassException( str( e ) ) hydrus.core.HydrusExceptions.URLClassException: Invalid IPv6 URL I don't know what it means, and I haven't noticed anything going wrong in Hydrus, but it might be important so I thought I'd show it and hope that you know what's happening.

>>24867 Just as general tagging philosophy, I will link my tagging help: https://hydrusnetwork.github.io/hydrus/getting_started_more_tags.html#tags_are_for_searching_not_describing 1) Don't try to be perfect. 2) Only add those tags you actually use in searches. It is tempting to describe images, but tags are, generally, not for describing--they are for searching. If you have reaction images that you want to search and differentiate by emotion, then I think doing 'reaction image' + 'happy/sad/whatever' is the best. 'smug' is particularly useful. You are talking about images that are more complicated than that, so I'd say think about how you mean to post them. If you want to differentiate anime vs real life or some fandom or show or whatever, then that's what is worth tagging. If you have a specific meme in mind, tag with that. You don't need so many tags that each file is individually specifiable: if you can cut your search pool down to 50 or 150 things, that's all you need to find 'the thing'. I find that for reaction images, I'll rarely go above three tags. A series/character tag, something like 'reaction image', and maybe one noun for the topic. Although it is tricky to navigate, doing 'meme:(canonical name)' can help shape searches too. 'meme:i ain't reading all that', if that is roughly the key phrase you think of when thinking of that, is ideal for searching that thing and all its variants. If you are tagging on the PTR, then obviously stay away from any emotion tagging that is too subjective, and try to get any 'meme:' names to harmonise. Siblings can also help if a meme has multiple legit names. >>24873 Not many excellent answers right now, but I hope to expand the 'alternates' system significantly, probably in 2027, to have unsorted or sorted 'pool' ability, 'two-page comic' ability, 'these are four WIPs in order', maybe even some dynamic paged chapter construction, etc.. I talk more about this in the revamped duplicates help, just ctrl+f for 'alternates', which is the current 'holding area' for all non-duplicate complicated file relationships: https://hydrusnetwork.github.io/hydrus/duplicates.html >>24885 Thanks. I downloaded it with yt-dlp with several different options but did not get the japanese metadata, so I'm doing something wrong. If you are willing, I am willing to keep working on this. Can you pipe the 'ffmpeg -i' output to a text file and post/pastebin that? I'll run it through my ffmpeg parser manually. >>24917 >>24929 Yeah, we don't have a nice way to inspect/browse tags in hydrus yet. I just never made a 'tag wiki' or anything. I'd recommend doing it with wildcard tag searches like 'creator:*', but don't run the search--just browse the autocomplete dropdown alone. You can also do it in a 'manage tags' dialog. Make sure you are allowed to do wildcard searches (they can be super super slow!) under tags->manage tag display and search and then enter your 'creator:*' in, and the highest counts will appear at the top. Might want to change from 'all known tags' to 'my tags' or whatever if you sync with the wildcard-laggy PTR. You can also just enter '*' for the mega search, and you'll probably have '1girl' and its friends at the top.

>>24937 I will be doing something like this when I get around to expanding the file properties system. I want to get EXIF and fifteen other things crammed in there, and rather than keep hardcoding a new database table every time, I want to have one generalised system with enums or whatever to handle everything with a nice unified UI and all that. For now, if you are ok not having the clever 'has transparency' logic, you can change its sensitivity under 'consider a file as having transparency when' under options>media playback. Set it to 'it has a transparency channel', and then any new file will obey this. Any existing file will need to get a database->file maintenance->manage scheduled jobs run with the 'determine if the file has transparency' check. This could be a big job, so I'd recommend trying to carefully shape the search that queues up the file maintenance job to only be like 'pngs without transparency imported within the last three months' or whatever will cover your stuff here. >>24940 Thanks, good point. I'll have a think about this. The technical storage limit is I think 1024 characters, which is just a failsafe. I agree that when tags overspill the UI there aren't great options. >>24941 database->file maintenance->manage scheduled jobs. Try a system:everything search and the 'regenerate thumbnail if incorrect size' job, which will quickly pass over good thumbs but fill in your holes. It will take a long time to finish. I cannot tell if you have figured out why this happened. How do you know it happened--are your thumbnails loading very slowly in the normal search view as they are regenned from source? Did you lose whole folders of thumbnails, or just a random selection of thumb files within them? Hydrus won't ever just delete a bunch of files like this, so it would be an external cause. If you are uncertain why your thumbnails disappeared, then I strongly recommend you check that before doing any repair. If your hard drive is damaged etc.., then that's your top priority. >>24942 Thanks. It looks like you have a program passing the Client API some URLs to import and some have whack formatting. It might be they are legit IPv6 URLs that I am parsing wrong, it might be they are malformed URLs, or it might be they are just random crazy text that my thing is interpreting as bad IPv6 URLs. I'll update my error handling code to specify what actually parsed wrong so we can better track this down. Please have a think about what you have talking to the Client API and see if you can figure out if it is recognising any failures on its end, and what it could be passing that would parse wrong.

>>24849 >Hydownloader can grab twitter & bluesky. It's based on gallery-dl though, which can't download from Twitter without a login

>>24944 (24941) Thanks! My really old computer probably messed up somewhere under load. I've restored a backup of the database, killed off orphan files and records, and have noticed that I'm missing a lot of thumbnails. So I figured I can just regenerate them. Will this work for imported files as well ( not from boorus, no URLs) ?

>>24697 >How do I clear out "deleted mappings" from a file? >>24741 >The completely ass-backwards way, if you really want to get rid of something, is: >- MAKE A BACKUP >- open the file in manage tags >- hit cog icon -> migrate tags for these files >- source is the tag service, 'deleted' tags. filter for the lone tag if you need it, otherwise all deleted tags >- dest is the same tag service, action is 'clear deleted record' >- fire that job off >- see if it worked. if it didn't restart the client and check again >- let me know how it went It works flawlessly. Super useful. Thank you

Maybe a stupid idea, but is there a way to use a game controller for shortcuts? The shortcut dialog doesn't seem to pick up controllers, might have to try remapping controller inputs to key presses.

>>24943 >Thanks. I downloaded it with yt-dlp with several different options but did not get the japanese metadata That's odd, I use a GUI for yt-dlp called "Video Downloader" with the setting to download 1080p quality if available. https://github.com/Unrud/video-downloader https://flathub.org/en/apps/com.github.unrud.VideoDownloader >Can you pipe the 'ffmpeg -i' output to a text file and post/pastebin that? Sure. The video's URL is the same: https://www.youtube.com/watch?v=78Rlj5gTG0Y and I confirm it is not importing, see pic #1 The terminal's output is: https://pastebin.com/xgNnmg98 ----- Also I found another MP4 1080p refusing to import, see pic #2. This time with Spanish characters in the metadata: https://www.youtube.com/watch?v=KQvfFqLkM-0 I'm posting the terminal's output in two chunks because of Pastebin's filter refusal to upload, see pic #3. The Pastebin's links are: https://pastebin.com/wpQ38KL2 and https://pastebin.com/QW9UuWRP

In the folder Hydrus Network\db\missing_and_invalid_files, what are these files and are they needed? I have about 200k of them.