/hydrus/ - Hydrus Network

Bug reports, feature requests, and other discussion for the hydrus network.

Posting mode: Reply

Check to confirm you're not a robot
Name
Email
Subject
Comment
Password
Drawing x size canvas
File(s)

Board Rules

Max file size: 350.00 MB

Max files: 5

Max message length: 4096

Manage Board | Moderate Thread

Return | Magrathea | Catalog | Bottom

Expand All Images


Version 514 Anonymous Board owner 01/26/2023 (Thu) 00:11 Id: 51ce74 [Preview] No. 1453
https://youtube.com/watch?v=PjfkAuUUq5w [Embed]
windows
zip: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Extract.only.zip
exe: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Windows.-.Installer.exe
macOS
app: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.macOS.-.App.dmg
linux
tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v514/Hydrus.Network.514.-.Linux.-.Executable.tar.gz

I had a good couple of weeks doing a variety of work. The changelog is long, mostly fixes and improvements that don't change much, but if you use the twitter downloader or the Client API, make sure to read before you update.

Full changelog: https://hydrusnetwork.github.io/hydrus/changelog.html

twitter

Twitter changed their policies regarding third-party interfaces recently, and it seems as part of that they took down the 'syndication' API our nice twitter downloaders were using. None of them work any more!

A user has kindly figured out a patch, which today's update will apply. It will throw up a yes/no dialog asking whether to do this, which almost everyone will want to click 'yes' on.

Unfortunately, this downloader can only get the first 0-20 tweets in the latest 20 tweets and retweets an account makes!! That's it! When you update, please check your twitter subs to make sure they moved to this new downloader correctly, and you might like to tell them to check more frequently.

I expect twitter to continue in this direction. I don't know if nitter or any other mirrors will be a viable alternative. If any of your twitter subs get posted in any normal booru, I strongly recommend you pursue those instead!

Client API

I am retiring some old data structures from the Client API today. If you use hyshare, please make sure you update to 0.11.2 or newer. If you use hyextract, please update to 0.4.1. If you use Hydrus Companion, I think you'll be good with the latest release for everything. The main changes on my end are to do with tag-viewing and tag-editing, so if you do a lot of that, make sure you have a backup and do a test of everything after you update.

If you have custom scripts, check the changelog. Mostly services are going to be referred to with service_keys in future, so I am removing 'service_name' structures and only supporting 'service_name' arguments in a legacy state. Please move to the /get_service(s) calls to figure out the service keys you need to use, as I will slowly retire the last legacy support later this year. Note that all users can now copy their service keys from review services.

As a side thing, I also rewrote how v413's /set_file_relationships call works. The old 'list of lists' parameter should still work, but a user reminded me that an Object with proper names would be a lot easier to extend in future, so I just rewrote it. This call is sufficiently new and advanced-user-only that I'm ok changing so quick, but if you set something up for it already, check the docs!


Anonymous Board owner 01/26/2023 (Thu) 00:12 Id: 51ce74 [Preview] No.1454 del
related tags

EDIT: In final testing, this stuff was working slow IRL for me, so I reduced its search space for today. You are going to see a lot of 'search domain is too large' on the PTR. I'll keep working!

Thanks to a user who sent in a great proof of concept, the 'related tags' column (which shows in manage tags and you can turn on under options->tag suggestions) has some new buttons to test. I called them 'new 1' and 'new 2'. The first works fast but may have limited results on repositories, the second--which only shows on repositories--can only search small-count tags but gives deep results.

Secondly, if you select some tags in the main tags list, the 'related tags' search will now only search those tags! If you can't remember a character name, then select a 'series' tag and click for a related tags search, and you'll get a bunch of them!

Thirdly, these new search buttons now work on manage tags launched from multiple thumbnails!

Please give this a go and let me know how you get on with IRL data. The quality of the new search results is better than my old routines, but I need to do a bunch more tuning and optimise their search. Are there too many recommendations (is the tolerance too high?)? Is the sorting good (is the stuff at the top relevant or often just noise?)? Which source tags (e.g. 'page:x') give the worst suggestions?

I'm sufficiently happy with the new performance, particularly with the tag selection, that once this is working nice, I think I'll make it default-on for everyone. Lots of people missed this system, and I always thought it was a bit jank.

other highlights

I wrote some sidecar import/export help here! https://hydrusnetwork.github.io/hydrus/advanced_sidecars.html

I optimised the how the thumbnail cache and downloader import queues do their normal work, particularly when they get tens of thousands of items. Big clients, which may have very busy versions of these objects, should be a little less laggy. Let me know if you notice any difference!

Downloaders that get from multiple sites in one query (aka NGUGs) now do their searches one site at a time, not interleaved (e.g. they now search sites A and B as AAABBBB, rather than ABABABB)!

next week

I've been semi-ill this past two weeks with some long-running something, and I pushed myself a bit hard getting all this ready in time. Next week will be some simple code cleanup, nothing too clever!


Anonymous 01/27/2023 (Fri) 23:47 Id: b8463f [Preview] No.1455 del
>>1453
twitter gay, thanks for fix
I suppose this means my nitter subs are broken too?

On a different topic, does Hydrus do anything with DeepBooru or are there any extensions for it you know of that I can maybe use to try some autotagging?


Anonymous Board owner 01/29/2023 (Sun) 02:24 Id: 9a2945 [Preview] No.1456 del
>>1455
Is DeepBooru the same as Deep Danbooru, or the same basic model, so you could switch it in place? Hydrus-dd offers several ways to plug into Deep Danbooru, here:

https://gitgud.io/koto/hydrus-dd

I was talking with someone else today about how I'd like to do more with this sort of tech in future, just making it easier to work with the sidecars you need or the Client API whatever. I'm sure image-to-text is going to benefit from all this text-to-image tech that's now floating around, especially now it is becoming reasonable to train your own models to do stuff. Hydrus as a big metadata-rich archive seems like it plugs into that tech pretty well!


Simple Release Tomorrow! Anonymous Board owner 02/01/2023 (Wed) 05:23 Id: 14c950 [Preview] No.1457 del
I unfortunately had a light week. The new 'related tags' search is polished and turned on by default for all users, and I fixed some small bugs.

The release should be as normal tomorrow.



Top | Catalog | Post a reply | Magrathea | Return