Hentai@Home 1.0.9, Dog's Work Options

1.0.9 Release Notes

This is mostly a bugfix release. If you don't have issues with 1.0.8, you don't need to update.

- Fixed a rare crash bug caused by the regex for fileids accepting certain strings that were actually invalid. (only with --use-more-memory enabled)

- Limited the maximum gallery and file name lengths to 100 characters each, to avoid a problem in certain operating systems (notably Windows) where the total maximum path length could be exceeded.

- The SQLite JDBC driver we were using was no longer being updated, so it was changed from Zentus to Xerial.

Download from the usual place.

For information on how to join Hentai@Home, check out The Hentai@Home Project FAQ.

first!

Lol, and I was wondering if this update might go without a single comment.

Update to silly minigame with little or no bearing on the site itself? 771 replies.

Update to fundamental system that allows the site to exist? 2 replies.

Thanks for the bug fixes Ten (IMG:[invalid] style_emoticons/default/smile.gif)

[tl;dr]No new critical issues found with this release, but an old harmless bug can be fixed and the database performances might have weakened.[/tl;dr]

I faced no serious issues with that release, though I noticed an unexpected exception in the error log:

CODE

2012-04-09T02:04:31Z [WARN] Invalid kvPair: keystamp%3D1333937067-a7c10578f3
2012-04-09T02:04:31Z [ERR] java.lang.NullPointerException
2012-04-09T02:04:31Z [ERR]      at org.hath.base.Out$OutPrintStream.println(Out.java:277)
2012-04-09T02:04:31Z [ERR]      at org.hath.base.Out.debug(Out.java:161)
2012-04-09T02:04:31Z [ERR]      at org.hath.base.HTTPSession.run(HTTPSession.java:246)
2012-04-09T02:04:31Z [ERR]      at java.lang.Thread.run(Unknown Source)

Don't pay attention to the lines numbers, as this is a custom build. The problem is in the catch block of the HTTSession.run() method: Exception.getMessage() can return null, but Out.debug() doesn't expect null (and when a NullPointerException¹ is thrown, it contains no message). IMHO, it would be better to replace Out.debug(e.getMessage()); by Out.debug(e); as the default implementation of Exception.toString() is to print the exception type and its message if there's one. That would avoid that problem, which isn't a critical issue anyway as the finally block will still be executed.

¹ the lack of a proper keystamp will cause the throwing of a NPE which will be caught in HTTSession.run().

BTW, if you wonder which browsers escape the '=' character (which is allowed by the specs IIRC):

CODE

$ grep -i '%3D' data/log_err* | egrep -o 'ua=.*' | sort -u
ua=AppEngine-Google; (+http://code.google.com/appengine; appid: s~senchaiosrc)
ua=Dalvik/1.2.0 (Linux; U; Android 2.2.1; A953 Build/MILA2_U6_3.18.4)
ua=Dalvik/1.4.0 (Linux; U; Android 2.3.5; HTC Desire Build/GRJ90)
ua=Dalvik/1.4.0 (Linux; U; Android 2.3.6; GT-I8150 Build/GINGERBREAD)
ua=Fuerza/2.2
ua=Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)
ua=Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727)
ua=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727)
ua=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; .NET CLR 2.0.50727)
ua=Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; .NET CLR 3.5.20706)
ua=Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; InfoPath.2)
ua=Mozilla/5.0 (compatible; Image2play/0.1; +http://image2play.com/)
ua=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
ua=Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)
ua=Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0.1) Gecko/20100101 Firefox/5.0.1
ua=Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.151 Safari/535.19
ua=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.142 Safari/535.19
ua=Nokia300/5.0 (07.03) Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0
ua=SAMSUNG-GT-S5230/S523MXEKE1 SHP/VPP/R5 Jasmine/1.0 Nextreaming SMM-MMS/1.2.0 profile/MIDP-2.1 configuration/CLDC-1.1
ua=UNTRUSTED/1.0 3gpp-gba

(My custom build logs these kind of information in case of error.)

- The SQLite JDBC driver we were using was no longer being updated, so it was changed from Zentus to Xerial.

I've modified the main CacheHandler.checkAndFreeDiskSpace() method so that queryCachedFileSortOnLasthit returns 100 rows instead of only 1 in order to speed up files deletion (the ORDER BY statement forces the query to return the whole result set before the LIMIT statement can reduce that set, so the query is pretty expensive, especially if it should be run for each files that needs to be deleted). On my test computer, with the old JDBC driver, it took around 3s to remove 1GB of files from the sqlite database (no real files were removed from the file system though, as the main purpose was to test the db performances), but with the new one, it takes around 24s to remove the same files (I used the same exact hath.db file for each test). Without my modification, it takes around 24s to remove the files with both drivers.

Not sure how relevant it is, nor which part of the driver is responsible for that, but I though it would be useful to share that experiment here, just in case other db performances reductions are observed with this release in the future ...

This post has been edited by Hairs Fan: Apr 9 2012, 20:26

BTW, if you wonder which browsers escape the '=' character (which is allowed by the specs IIRC):

Noted.

Not sure how relevant it is, nor which part of the driver is responsible for that, but I though it would be useful to share that experiment here, just in case other db performances reductions are observed with this release in the future ...

Hmm. Does it report running in pure or native mode on startup?

Hmm. Does it report running in pure or native mode on startup?

In native mode for both drivers (on Windows BTW).

The following code used to run quicker than the original code with the previous driver:

CODE

synchronized(sqlite) {
    queryCachedFileSortOnLasthit.setInt(1, 0);
    queryCachedFileSortOnLasthit.setInt(2, 100);

    while((filesToFree > 0 || bytesToFree > 0) && cacheCount > 0) {
        ResultSet rs = queryCachedFileSortOnLasthit.executeQuery();
        
        if(!rs.next()) {
            HentaiAtHomeClient.dieWithError("CacheHandler: Could not find more files to delete. Corrupt database?");
        }
        
        do {
            HVFile toRemove = HVFile.getHVFileFromFileid(rs.getString(1));
            
            if(toRemove != null) {
                bytesToFree -= toRemove.getSize();
                filesToFree -= 1;
                deletedFiles.add(toRemove);
                deleteFileFromCacheNosync(toRemove);
            }
        } while(rs.next() && (filesToFree > 0 || bytesToFree > 0) && cacheCount > 0);

        rs.close();
    }
}

The original code is:

CODE

synchronized(sqlite) {
    queryCachedFileSortOnLasthit.setInt(1, 0);
    queryCachedFileSortOnLasthit.setInt(2, 1);

    while((filesToFree > 0 || bytesToFree > 0) && cacheCount > 0) {
        ResultSet rs = queryCachedFileSortOnLasthit.executeQuery();
        HVFile toRemove = null;

        if(rs.next()) {
            toRemove = HVFile.getHVFileFromFileid(rs.getString(1));
        } else {
            HentaiAtHomeClient.dieWithError("CacheHandler: Could not find more files to delete. Corrupt database?");
        }

        rs.close();

        if(toRemove != null) {
            bytesToFree -= toRemove.getSize();
            filesToFree -= 1;
            deletedFiles.add(toRemove);
            deleteFileFromCacheNosync(toRemove);
        }
    }
}

... and contrary to what I thought, the reason why my modification used to run quicker is not only due to the fact that it retrieves 100 results in a row, as the following code gives the same performances as the original code (with both drivers):

CODE

synchronized(sqlite) {
    queryCachedFileSortOnLasthit.setInt(1, 0);
    queryCachedFileSortOnLasthit.setInt(2, 100);

    while((filesToFree > 0 || bytesToFree > 0) && cacheCount > 0) {
        ResultSet rs = queryCachedFileSortOnLasthit.executeQuery();
        
        if(!rs.next()) {
            HentaiAtHomeClient.dieWithError("CacheHandler: Could not find more files to delete. Corrupt database?");
        }
        
        // Put the results in a list so that rs isn't needed anymore
        LinkedList<String> fileIdList = new LinkedList<String>();
        do {
            fileIdList.addLast(rs.getString(1));
        } while(rs.next());
        
        rs.close();
        
        do {
            HVFile toRemove = HVFile.getHVFileFromFileid(fileIdList.pollFirst());
            
            if(toRemove != null) {
                bytesToFree -= toRemove.getSize();
                filesToFree -= 1;
                deletedFiles.add(toRemove);
                deleteFileFromCacheNosync(toRemove);
            }
        } while(!fileIdList.isEmpty() && (filesToFree > 0 || bytesToFree > 0) && cacheCount > 0);
    }
}

(Note: deleteFileFromCacheNosync() removes the row that is related to the result being processed.)

That was probably due to so kind of sqlite optimizations that were only useful in this particular case, and have been dropped before 3.7.2. Anyway, it doesn't seem to affect the official build so far, so I guess there's no official problem yet (IMG:[invalid] style_emoticons/default/smile.gif)

Update to fundamental system that allows the site to exist? 2 replies.

You want complaints on H@H updates now too!? (IMG:[invalid] style_emoticons/default/laugh.gif)

You want complaints on H@H updates now too!? (IMG:[invalid] style_emoticons/default/laugh.gif)

Tenboro how could you limit maximum gallery and file name lengths? (IMG:[s7.directupload.net] http://s7.directupload.net/images/120410/donzznbg.gif)

Another non working one, good thing there is 1.0.6

Hey Tenboro, can you add an uptime/bandwidth log? Something simple that just dumps the duration the client was connected and how much data passed through up and down to a text file on shutdown.

Nice. I'm gonna try out this version even tho I haven't had any problems with 1.0.8 (I like to stay up to date on things). :)

(IMG:[invalid] style_emoticons/default/tongue.gif) how to use this web....


Is it able?

I didn't even notice any bug. Must be cause I'm a Newfag.

Figure this as good a place as any to ask this, but what happened to the explanation of the rate of hath acquisition to the rate of speed your client runs at? Do we still get those multiples at the various rates or is everything one hath for one day?

Figure this as good a place as any to ask this, but what happened to the explanation of the rate of hath acquisition to the rate of speed your client runs at? Do we still get those multiples at the various rates or is everything one hath for one day?

http://ehwiki.org/wiki/Hath

http://ehwiki.org/wiki/Hath

Ah, thank you very much! (IMG:[invalid] style_emoticons/default/biggrin.gif)

Is it me or the rate of file received slowed considerably since this version. It would usually follow the Total Files Sent:Total Files Received by a ration of 4:1 when I would download from hathdl files. But now I'm seeing something closer to 32:1 with many Timeout detected messages.

I don't know why? But i can't enter in my Upload Gallery and Maintain Galleries section from a few days.