the rest

The archive/delete filter gets a couple of workflow changes: first, if you finish a filter and there is more than one possible local file service to delete from, those 'commit' buttons are now disabled for 1.2 seconds. This is to catch you from spamming 'enter' through this dialog when it is suddenly different (I've done this myself more than once). Second, if you hate the idea of these buttons being disabled, and you always want to delete from all local file services anyway, please hit the new 'when finishing filtering, always delete from all possible domains' checkbox under options->files and trash, which lets you always have a simple 'commit' dialog that only shows 'delete from all local file services'.

The client now tries to load truncated images by default. The damaged images it now allows might be missing one pixel in the bottom right, or have a few lines of grey at the bottom, or might appear fine but just have some crazy metadata, but they won't, fingers crossed, fail with a 'malformed image' error any more. We had some stability problems with this mode some years ago, so I turned it off and only allowed it on in a debug menu on a per-session basis, but the situation seems to have cleared up, so it is now back on. If you need to turn it off, hit options->media.

Any time you have a normal single column list in the program, e.g. the list of URLs in 'manage urls', you can hit Ctrl+C or Ctrl+Insert and now it'll copy better strings (e.g. without '(1)' decorator cruft), and it'll copy every row you have selected.

I wrote a new emergency help document, 'help my db disappeared.txt', for the install_dir/db folder. If you ever boot up and get the 'this looks like the first time you have run the program' popup, there's now a guide to figure out what the hell happened.

next week

I didn't find the time to get to the 'share' menu rewrite, so I'll try again.

I had a mixed week. I did some boring stuff, some quality of life, and folded the 'future build' updates into the main release. Users who use the manual 'extract' releases will have special update instructions.

The release should otherwise be as normal tomorrow.

I had a good week. I worked on simple bug fixes, code cleanup, and quality of life.

The release should be as normal tomorrow.

other highlights

I also added a 'tag in reverse' checkbox to the new 'manage tags' 'incremental tagger' thing. It adds tags like 5, 4, 3, 2, 1 instead of 1, 2, 3, 4, 5.

And all new system:url predicates have new labels. They are all a bit simpler, and they should copy/paste into the system predicate parsing system. All existing system:url predicates still have their old labels, so if this is a big deal, you'll want to recreate them and re-save your session/search.

Thanks to a user, the new docx, xlsx, and pptx file formats get some nicer thumbnails and a little metadata. It should all recalculate soon after update.

The Client API is now more careful about which files you can undelete, and it also now lets you clear file deletion records.

next week

I want to put proper time into getting a 'future build' working. Last time I tried, I ran into some technical problems related to the newer libraries I wanted to bundle, so I'll see if I can fix it all and have a test release for people to try. Otherwise, I just want to clear out some small jobs that this URL work boshed.

Anyone knows if I can download files (through url downloader, non-booru site) whilst keeping their original filenames somehow (as tags or whatever)? And in a way that I can have the files renamed when exporting? I found plenty of places saying it can be done but I've tried messing with the tag import options among others and nothing seems to work.

>>1623
Servers typically give that filename in the 'http header' of a file download. Hydrus generally does not parse this in the same way that your browser would and does not push it into the general metadata pipeline of a file import, so I'm afraid the only way you can figure this out is to parse it from the html file or similar, as you would for other metadata like tags or related URLs.

In general, also, most downloader makers do not parse the filename. The imageboard watchers like the 4chan parser does parse for the filename, and gives it a 'filename:' namespace. By default, I have these set not to add in the 'tag import options' for watchers, since most users don't want them.

If you feel clever and brave, you can try to add a 'content parser' to a downloader you are interested in to try and grab filename from the boorus you like, but beyond that, I'm sorry to say I don't have a good answer for you. I've thought a few times about making a 'filename' tag service that remembers hard drive import filenames, and could potentially get server-set filenames too, but every time I return to the idea, I realise it'll probably just get overwhelmed by 'Image.jpg' garbage that isn't typically useful.

Although I can't give you a nice answer, you might like to read my FAQs on this question, just so you see better where I am coming from and why I don't want to put much time into this:

https://hydrusnetwork.github.io/hydrus/faq.html#filenames
https://hydrusnetwork.github.io/hydrus/faq.html#external_files (somewhat related)

And if you would like to dabble with making/editing your own downloaders, check out the help here:

https://hydrusnetwork.github.io/hydrus/downloader_intro.html

I'd seen those pages already, thank you. I understand about filenames but this time I really needed them. Over 90,000 work-related images that were numbered in a specific order, and that ordering had to be kept. I've found another solution but thank you for your work and your reply.

I had a good week. I fixed a bunch of small issues, and I figured out the problems with the 'future' build, so I'll also have a special version for more advanced users to test out.

The release should be as normal tomorrow.

>>1625
Great, I am glad you found a solution!

I had an ok week. I reworked some URL handling in ways that are mostly important behind the scenes and cleared some small jobs. It'll just be a simple release.

The release should be as normal tomorrow.

The v567 release is cancelled! We found a problem and I need to do more work on it. I will move it on to next Wednesday, sorry for the inconvenience!

I had a great week. I tightened up the URL storage/handling improvements that I was not confident in last week, so I am very happy to put out the release tomorrow. There are also advanced new tools for downloader makers for handling 'ephemeral token' parameters along with new quality of life UI in the manage URL Classes dialog. For normal users, there are also several bug fixes, file handling improvements, and a couple little things in system predicates, emoji tag presentation, and reversed tags in the new incremental tagger.

The release should be as normal tomorrow.

I had a good week. The 'incremental tagger' system is working, allowing you to tag page:3, page:4, page:5 ... page:17 and similar to a selection of files, and .docx files are now importable.

The release should be as normal tomorrow.

I had a simple week, mostly fixing a bunch of bugs.

The release should be as normal tomorrow.

Yeah, this is tricky. I still think your best answer is to wait for me to implement a proper xml/html parser for sidecars, or do the parsing yourself in an external script (e.g. with python) and then convert your html into nicer .txt files that hydrus can suck up easier. You could also filter your undesired tags better there.

If the 'description' string is always in the same location, and that location is always the first or last index of your list of strings, then the 'string selector' might help. It does list slicing like in programming, if you are familiar, like "my_list[4:6]". If you aren't familiar with that, or the description is in the middle of the list here, you are correct in trying a string match, which is basically a filter.

If the description line has a classname in the html, you might be able to exclude it with a string match before you Split/Convert all the html garbage away.

I am not totally sure what you mean by being unable to import by pngs, but if I click 'import->from pngs' (pic related is from the 'sidecars' tab of the 'add tags/urls with the import' dialog after you drop some files on the client), I get a file dialog that allows me to import a png like this. Do you get different?

Update:

(i am using a crucial mx500 500gb with luks and ext4, if that helps)

I was looking at launch options to fix the storage issue, do you think that --no_db_temp_files would help? I have 16gb ram, is that enough?

>>1610
>>1611
>>1612
Yeah, the write is most likely temp storage. I've heard that in some situations, the actual amount written to disk is less since some temp-file magic means some of the shortest-lived data is purged from the write cache before it can actually be committed, but I don't have good numbers on the topic.

The PTR does a lot of database work. The no_db_temp_files parameter will reduce it by a good amount. Not sure how much, but it would reduce it.

Increasing db_transaction_commit_period may also reduce it, I think, since it will reduce the commit frequency (at the cost of increasing per-transaction size, which will strain your temp folder and slow processing time).

A user also wrote this document with a lot of related info, if it helps: https://hydrusnetwork.github.io/hydrus/Fixing_Hydrus_Random_Crashes_Under_Linux.html

I'm not a Linux expert myself, so I can't talk super confidently here. Let me know how you get on!

I had a good week. A bunch of system predicates now support customisable +/- ranges, both by absolute and percentage values.

The release should be as normal tomorrow.

I had a good week. I fixed some bugs and made some shortcuts a bit nicer to deal with, and there are a bunch of macOS improvements.

The release should be as normal tomorrow.