[X] My Assistant

P1 = 44, P2 = 54, P3 = 101, P4 = 127, P5 = 2694

If P5 remains hugely superior to the rest, is it the sign it would be more efficient to decrease storage allocated to H@H?

It just means you have a lot of storage compared to your speed. You won't gain anything by decreasing the space, and your hath rate will probably decrease unless it is capped on speed.

It just means you have a lot of storage compared to your speed. You won't gain anything by decreasing the space, and your hath rate will probably decrease unless it is capped on speed.

hentaiathome page downed
(IMG:[s2.loli.net] https://s2.loli.net/2023/10/11/wBhXG53TpgEsme9.jpg)

Check again, they are visible to me now.

Yeah, it was briefly broken as it wasn't fully updated to match some backend library changes, should be fixed now.

(Incidentally, the entire state tracking backend stuff has been replaced, so uh, let me know if you notice any weirdness.)

[BUG] special unicode char replaced by a question mark "?" and cause an error.

CODE

2023-10-11T18:28:02Z [info] GalleryDownloader: Starting download of gallery: (C100) [Avalanche (ChimaQ)] Cosplay shita Yousei-san o Mederu Hon (Touhou Project) [Chinese] [白羽💯个人汉化]
Exception in thread "Thread-20" 2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)} java.nio.file.InvalidPathException: Malformed input or input contains unmappable characters: download/(C100) [Avalanche (ChimaQ)] Cosplay shita Yousei-san o Mederu Hon (Touhou Project) [Chinese] [白羽?... [2420379-1280x]/01_0_mh1672581090771.jpg
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at java.base/sun.nio.fs.UnixPath.encode(UnixPath.java:121)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at java.base/sun.nio.fs.UnixPath.<init>(UnixPath.java:68)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:279)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at java.base/java.io.File.toPath(File.java:2387)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at hath.base.GalleryDownloader$GalleryFile.download(GalleryDownloader.java:377)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at hath.base.GalleryDownloader.run(GalleryDownloader.java:91)
2023-10-11T18:28:02Z [ERR] {java.base/java.lang.Throwable$WrappedPrintStream.println(Throwable.java:763)}       at java.base/java.lang.Thread.run(Thread.java:

The unicode char is 💯

This failure also blocks the downloading queue. Any following or new-added task won't start.

The gallery is at https://e-hentai.org/g/2420379/887bbf2a54/

[BUG] special unicode char replaced by a question mark "?" and cause an error.

The unicode char is ������

This failure also blocks the downloading queue. Any following or new-added task won't start.

The gallery is at https://e-hentai.org/g/2420379/887bbf2a54/

I'm not able to reproduce a crash, but it does look like the truncating of the gallery title could mangle the last character if it's a UTF-16 supplementary character (roughly equivalent to a 3+ byte character in UTF-8). According to your logs, for some reason it doesn't fail on creating the directory, it only fails when it tries to download a file, so the "broken unicode" fallback doesn't trigger either.

I wrote a fix for this, but it turned out to be somewhat complicated so I'll need to put off testing it until tomorrow.

I'm not able to reproduce a crash ... so the "broken unicode" fallback doesn't trigger either.

It does sound complicated, and a little weird.

BTW, is it by design that any failure will block the following jobs in the queue? It's not able to download any gallery here. Will you consider changing it to skip the failure and continue the following jobs?

This post has been edited by ungrown: Oct 12 2023, 03:50

Try using this build: https://repo.e-hentai.org/hath/HentaiAtHome_b163.zip

This should fix the UTF-16 supplementary character splitting issue and also adds some hardening to the gallery downloader.

BTW, is it by design that any failure will block the following jobs in the queue? It's not able to download any gallery here. Will you consider changing it to skip the failure and continue the following jobs?

It's not by design, it's just failing on a system call that was not expected to fail like that. I've added some error catching around those parts to prevent it in general.

Try using this build: https://repo.e-hentai.org/hath/HentaiAtHome_b163.zip

This should fix the UTF-16 surrogate character splitting issue and also adds some hardening to the gallery downloader.
It's not by design, it's just failing on a system call that was not expected to fail like that. I've added some error catching around those parts to prevent it in general.

b163 looks good.
Now the queue is rolling. There are hundres of jobs to finish. If there's any issue I'll let you know.

---
update

It seems most of those old download jobs are just gone.

This post has been edited by ungrown: Oct 12 2023, 08:42

It seems most of those old download jobs are just gone.

If they were sitting around for more than a week, they were probably eaten by the queue pruner.

Offline 1 month Now I'll be back with New version client (IMG:[invalid] style_emoticons/default/rolleyes.gif)

As previous indicated, we've now switched to a new method to handle overloads.

The new one is a lot more lenient towards occasional incidents, and won't affect the speed of the client unless there is more than one overload in a relatively short period of time.

It will also no longer reduce the speed limit below the minimum threshold for whether a client is considered usable or not, and it will only ever look at overloads that happened in the last week.

Whenever a client is started up for the first time, the region it is located in will now be designated as the client's "home region". All existing clients have had their home region set to their current region.

If the client is moved to a different region at a later point, the country will light up on red on the H@H status page, indicating that the client is considered to be outside of its home region. This has a few effects:

- Clients outside the home region will no longer be assigned new ranges or be considered for range priority promotions

- If there are multiple clients with the same range/priority in the region, clients outside its home region will always be demoted first, and the client in the home region will be preferred for routing traffic

The main reason for this change is to prevent issues with range priority stemming from bad geolocation. It also prevents an "exploit" where people could run a client in a low-coverage low-traffic region to build up P1 ranges, then move it to a high-traffic region and bully out the legitimate clients in the region.

If you want to move a client to a different region, you will need to PM me. In most cases, this will require demoting all of the client's P1 ranges to P2 to allow the system to optimally rebalance range priority.

If your geolocation is simply wrong, this is not required and it will just be corrected like before, but note that I will check the history of the client for these cases.

If you want to move a client to a different region, you will need to PM me. In most cases, this will require demoting all of the client's P1 ranges to P2 to allow the system to optimally rebalance range priority.

i suggest not demote if move between both high traffic country(i.e:japan and hong kong,maybe also china mainland)

i suggest not demote if move between both high traffic country(i.e:japan and hong kong,maybe also china mainland)

Moves within the same region (Japan <-> Hong Kong) are not affected and can still be done at will. This is only relevant for moves between the seven regions.

There will generally already be clients with corresponding P1 ranges in the region the client is moved to, so for most moves it makes sense to demote to P2 and then let the system promote ranges that are missing from the region, if any.

Though I would consider allowing moving clients from any region to China without a priority demotion, simply because most clients there are at close to 100% utilization.

In a couple of days, the system will start to assign what I'm calling high-capacity ranges, which will be used for large and original quality files.

To qualify for these ranges, clients have to meet more stringent requirements. So if your client is close to meeting all the requirements and you want these ranges assigned, you may want to look into changing your configuration.

- Must be located in one of the three major regions with surplus capacity, in other words Europe, North America or Asia (excluding China)

- Must have at least 10 MB/s throttle and 8 MB/s tested speed

- Must not have a monthly bandwidth limit set

- Must have at least 100 GB of disk space assigned

Clients with a low ratio of P1 ranges will be preferred for HC ranges - reason being these clients will have surplus cache capacity. You'll want P1 ranges more than you want HC ranges, so trying to avoid P1 ranges is a bad idea.

Eventually HC ranges will also require 1.6.2 or newer. This will not be required for the initial distribution, but you should update any older clients that meet the other requirements if you want to keep these ranges in the future.

Note that HC ranges are set independently of priority, so a range can be both P1 and HC.

Hi

My H@H is unable to download any galleries again. Below are the warning messages:

2023-10-17T16:45:49Z Logging started
2023-10-17T16:48:23Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:48:23Z [WARN] Retrying.. (2 tries left)
2023-10-17T16:48:33Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:48:33Z [WARN] Retrying.. (1 tries left)
2023-10-17T16:48:54Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:48:54Z [WARN] Retrying.. (0 tries left)
2023-10-17T16:48:54Z [WARN] Exhaused retries or aborted getting [csqlgqr.jymhujcbggkj.hath.network] https://csqlgqr.jymhujcbggkj.hath.network:2...xres=org/hathdl
2023-10-17T16:49:27Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:49:27Z [WARN] Retrying.. (2 tries left)
2023-10-17T16:51:00Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:51:00Z [WARN] Retrying.. (1 tries left)
2023-10-17T16:52:37Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:52:37Z [WARN] Retrying.. (0 tries left)
2023-10-17T16:52:37Z [WARN] Exhaused retries or aborted getting [ofabdia.pksbecxrcjwe.hath.network] https://ofabdia.pksbecxrcjwe.hath.network:2...xres=org/hathdl
2023-10-17T16:53:08Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:53:08Z [WARN] Retrying.. (2 tries left)
2023-10-17T16:53:39Z [WARN] java.net.SocketTimeoutException: Read timed out
2023-10-17T16:53:39Z [WARN] Retrying.. (1 tries left)

May I ask for guidance on how to fix this?

This post has been edited by Celtae: Oct 17 2023, 18:54

- Must run on the default HTTPS port (443)

anyone willing to write up a detailed SNI L4 reverse proxy setup? (IMG:[invalid] style_emoticons/default/ph34r.gif)

In a couple of days, the system will start to assign what I'm calling high-capacity ranges, which will be used for large and original quality files.

To qualify for these ranges, clients have to meet more stringent requirements. So if your client is close to meeting all the requirements and you want these ranges assigned, you may want to look into changing your configuration.

- Must run on the default HTTPS port (443)

- Must be located in one of the three major regions with surplus capacity, in other words Europe, North America or Asia (excluding China)

- Must have at least 10 MB/s throttle and 8 MB/s tested speed

- Must have at least 200 GB of disk space assigned

Clients with a low ratio of P1 ranges will be preferred for HC ranges - reason being these clients will have surplus cache capacity. You'll want P1 ranges more than you want HC ranges, so trying to avoid P1 ranges is a bad idea.

Eventually HC ranges will also require 1.6.2 or newer. This will not be required for the initial distribution, but you should update any older clients that meet the other requirements if you want to keep these ranges in the future.

Note that HC ranges are set independently of priority, so a range can be both P1 and HC.

Megabytes not megabits right? This will be harder to achieve in some regions.

Megabytes not megabits right? This will be harder to achieve in some regions.

It would have to be megaBYTES not megabits because these are large files. Who would want to download a 50MB PNG at 10Mbps (1.25MB/s)?