Welcome Guest ( Log In | Register )

 
Closed TopicStart new topic
> Some H@H changes, Barely major enough to warrant a thread

 
post Oct 10 2011, 21:21
Post #1
Tenboro

Admin


清く正しく


Note that there has been a change to how the H@H clients calculate their allowable disk space usage. The changes are completely server-side, but will be noticeable on the clients that were limited in that they'll delete a bunch of files the next time you start it.

This might cause a temporary drop in utilization, but the limit for the disk space is calculated so that you should have no problems getting about the same speed as before. Furthermore, the clients should recover after a restart much faster than before since the system should no longer has to reset the file routing tables for it.

For the technically curious, the reason for this change is that I've been noticing a larger than expected rate of server-side file list resets, and after investigating with the fancy new database instrumentation I added the other day, the conclusion was that it was caused by clients with a large discrepancy between the local cache size and the cache size accepted by the server - which depends on the speed capabilities of the client. So essentially, the system had to reset the server-side list every time the client connected, which meant that a) it would be slow as ass to start up, b) it would take a long-ass time before you got decent traffic, and c) the system had to do an assload of unnecessary work to rebuild the client's file list.

Soo, yeah. About 800 clients had their existing disk limit reduced due to this. If you want, you can check if you're one of them from the H@H page, but otherwise, you don't have to do anything.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 11 2011, 02:41
Post #2
JukanX



Lurky McLurkFace
****
Group: Gold Star Club
Posts: 383
Joined: 8-July 09
Level 466 (Godslayer)


Does it affect the thumbnails servers too? On the gallery I've viewed today, a number of thumbnails failed to load...
Looking at the log of the past few minutes, the connection seems to hang with no response; most often with gt3 and sometimes with gt2; gt0 and gt1 are responsive.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 11 2011, 04:13
Post #3
pop9



Forever Alone
************
Group: Members
Posts: 26,039
Joined: 6-May 09
Level 500 (Dovahkiin)


QUOTE(JukanX @ Oct 11 2011, 02:41) *

Does it affect the thumbnails servers too? On the gallery I've viewed today, a number of thumbnails failed to load...

same here. tooooooo heavy
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 11 2011, 08:13
Post #4
Tenboro

Admin


清く正しく


Nah. Thumbnails were failing because a server died, and your browser might not properly support DNS failover.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 11 2011, 16:43
Post #5
Maximum_Joe



Legendary Poster
***********
Group: Gold Star Club
Posts: 24,074
Joined: 17-April 11
Level 500 (Dovahkiin)


QUOTE(Tenboro @ Oct 10 2011, 11:21) *

will be noticeable on the clients that were limited in that they'll delete a bunch of files the next time you start it.


Cache Utilization: 1216.21%
Free Cache Space: -61.40 GB

Woah boy, that's a lot of deleting... (IMG:[invalid] style_emoticons/default/blink.gif)

On the upside if my HDD doesn't light on fire from this defragging it will be much faster. (IMG:[invalid] style_emoticons/default/laugh.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 18 2011, 18:14
Post #6
Bunko



Hermit
*******
Group: Members
Posts: 1,262
Joined: 19-September 10
Level 338 (Godslayer)


Question to the boss.
My max upload is 120 kilobytes per second.
Is having 12 GB cache somehow better than having a 45 GB cache ?

I could keep my 45 GB cache by setting speed to 450 KB/s.
But does the tracker only keep a certain amount of stuff in memory ?

I can still have my line in good quality either by...
1) Using bandwidth controller, to limit the traffic (I used to do this)
2) Using ADSL modem's QOS (quality of service) to reduce max UL to 120K. (This is what I'm doing now)

When limiting with QOS-Premium setting, I can keep my gateping below 50 ms.
Without limiting, it whoops to 250+ ms. which is nasty.

To me it seems that limiting my cache size to 1/3 doesn't seem logical, well I don't have the info to make logic out of it.
It may help if tracker only registers certain amount of data.
If that tracked data is based on speed which I have set, instead of measured speed which I can do.
I'd still say, bigger cache wins ?
My Persocom runs 24 hours a day.

This post has been edited by Bunko: Oct 18 2011, 18:16
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 18 2011, 19:09
Post #7
Tenboro

Admin


清く正しく


The additional files are useless to the system and consume resources to track. Don't set your client to 450 KB/s unless you actually can push 450 KB/s.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 18 2011, 19:18
Post #8
Bunko



Hermit
*******
Group: Members
Posts: 1,262
Joined: 19-September 10
Level 338 (Godslayer)


Sir yes sir.
Speed set to 120K
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 18 2011, 19:23
Post #9
Lukabratzy



Newcomer
*
Group: Recruits
Posts: 14
Joined: 7-July 11
Level 36 (Journeyman)


Sooooooooo is that why i could not access the site for about a month
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Oct 18 2011, 19:55
Post #10
Bunko



Hermit
*******
Group: Members
Posts: 1,262
Joined: 19-September 10
Level 338 (Godslayer)


Nah.
No matter how high I set upload, the code is smart enough to grasp my real speed, never serving a picture to more than 1 client at a time.
Accessing the site has nothing to do with H@H.
Unless it took your h@h a month to shrink it's cache.
Sounds like a blocklist issue.
===EDIT===
I wonder. Would having a subforum in Helpdesk for H@H problems be useful ?
Only a slight problem with that. Someone would need to keep moving posts.
... and pretty much all h@h problems are problems with people using connection blocking software/antivirus/security and being behind firewalls, spiced with insufficient upload speed.
I haven't run to a h@h problem in the past year that was caused by e-hentai, they're always internet provider/pc/software problems. None of which belong to this forum, but we got people willing to help here.
Rest assured, if there is an actual problem with the site or a new version of h@h, loudmouths like me will quickly lash their tongue about it at the helpdesk.

Exception to that rule is that it uses way too few connections by default. About half of the available bandwidth over time. That's a "feature" of the program

This post has been edited by Bunko: Oct 19 2011, 01:13
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 2 2011, 05:40
Post #11
-Enforcer-



Casual Poster
***
Group: Gold Star Club
Posts: 231
Joined: 11-March 09
Level 361 (Godslayer)


I suggested having a sub-forum setup for H@H but Ten didn't feel it was worthy. I suppose that's true.

Any H@H questions can be forwarded to me if you choose. I don't maintain the code or have direct involvement of how its ran but I think I have the greatest understanding of it outside of a staff member.

And that's not to undermine Tenboro - just offering my help and lighten the question floods he's getting (IMG:[invalid] style_emoticons/default/smile.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 2 2011, 05:50
Post #12
-Enforcer-



Casual Poster
***
Group: Gold Star Club
Posts: 231
Joined: 11-March 09
Level 361 (Godslayer)


Additionally, H@H is greatly dependent on your ACTUAL - TESTED upload speed. I've ran enough tests myself to prove that even with a near unlimited upload capability, having a cache size over about 100GB is COMPLETELY USELESS and actually has an adverse effect on the servers.

I used to run a full 400GB cache but found it to thrash the drive so badly (it was handling between 200-350 conns per second) that I decided to cut it in half and see if it affected anything.

It didn't.

So I cut it back to 100GB and also noticed no major difference (it did drop a bit due to very old cache files, but has been since fixed with an H@H update from Tenboro's side) and I still average 200 connections per second - but its able to deliver them faster because the drive isn't having to search all over the disk to find a file. (it also helps to set aside a partition on a drive JUST for H@H if you're running over 50GB cache, because it limits the places where the OS has to store the files and therefore, limits the time it takes for the files to be written, read, deleted, etc. Look at it like putting all your bathroom stuff IN the bathroom instead of all over the house.

The main reason the cache size limiter was put in place was folks with slower upload speeds were amassing 100's of Gigabytes of cache (indirectly thinking they might get to send more files if they had a larger cache) but this created a mess on the server end because even if you had the file, you may not get selected because your (actual) upload speed kicked you down the priority list - YET - the server still had to process your H@H client ID and 90% of the time, it didn't need to because of the lower priority level.

Therefore, if you can't upload faster than 1MegaBYTE per second, its not helpful to you OR the H@H community to have a large cache size.

Hope that helps (IMG:[invalid] style_emoticons/default/smile.gif)
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

 
post Nov 2 2011, 22:01
Post #13
jA6sME4FZZ



Newcomer
*
Group: Members
Posts: 20
Joined: 26-October 09
Level 40 (Journeyman)


I noticed the change (meaning the client got a bit longer startup after this and deleted a bunch of files), but sure. If it's for the network. (IMG:[invalid] style_emoticons/default/smile.gif)

@Bunko - PM Tenboro. This happened to me too, and he helped me to set things up. The client works flawless since then.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post


Closed TopicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 


Lo-Fi Version Time is now: 28th March 2024 - 13:54