[X] My Assistant

New upload servers have been deployed and can be accessed here: [upld.e-hentai.org] https://upld.e-hentai.org/manage

Note that while this is officially a beta, the servers are live. If something were to break or end up in the wrong state with a gallery during the beta, don't try to "fix" it, leave it and let me know so I can figure out what went wrong.

The new uploaders are now used for new uploads and new versions of existing galleries. Existing galleries will be migrated soon.

The new uploaders are now used for all uploads, and all existing unpublished uploads have been migrated.

--

New uploader servers have now been deployed, which will eventually replace the current uploaders that were deployed six years ago. These new servers also come with a new major release of the uploader interface. While they have been in closed beta testing for the last week or so, I decided to run a public beta as well before switching to the new servers entirely.

The new servers run new major versions of pretty much everything, the internals of the uploader have been significantly reworked, and it also uses the new core library that was first added to the previous version of HV, so please be on the lookout for any issues.

Uploads may temporarily be somewhat slower than before, since files have to also sync over to the old uploaders for this initial release. When the old uploaders have been phased out, this will go back to normal.

Unpublished galleries that were added on the old uploaders will not show up on the new ones until they have been imported, which will happen some time after the new uploaders go fully live. The old uploaders will be available until then.

Similarly, galleries added or versioned (cloned) on the new uploaders will not show up on the old ones, though there was a check added to prevent a gallery from being versioned on both the new and old uploaders.


New Features

- Added time-based auto-sorting that allows you to easily resort a gallery to have newly added files first (or last) without having to manually reorder them, which is a common task for updated image galleries.

This uses the publish time for the gallery in the chain each file first appears in, and is only applicable for versioned (cloned) galleries. Note that the order the files were uploaded in for a given gallery in the chain does not matter for this sort as it goes by which gallery it was first added to, but you can keep the current order within each set of files.

- Added an "auto-sort" method called "Remove Exact Duplicates" which, as the name implies, will remove duplicated files from the gallery. The first file in the current page order will be preserved.

The available methods are now:
* Remove exact duplicate files from the gallery
* Sort based on natural or lexical ordering of the filenames, ignoring when they were added. (These are the old methods.)
* Sort based on the time a file was first added to a chain of galleries, using either a Recent First (descending) or Recent Last (ascending) sort. It can then either keep the current sort order within each set of files, or do an additional natural or lexical sort based on the filenames.


- Added the option to "disown" any published gallery. This will remove the gallery from the uploader interface, remove the uploader link/readout from the gallery, and prevent the gallery from being found with an uploader search.

You will not be able to make (or request) changes to a gallery after disowning it. You will however continue gaining toplist points and GP from disowned galleries.

You cannot disown a gallery before it has been fully published ("Visible" changes from "Soon"). Recently published disowned galleries will still show the uploader for a week after being published.

Note that expunged galleries will now also be "disowned" rather than "deleted". "Deleting" published galleries is never possible anymore. Unpublished galleries will still be erased as before.


- When creating a new gallery, you can now set the initial language for the gallery directly on the uploader screen. This will automatically create Autotagger tags for that language on the gallery when it is published.

For an already published or versioned (cloned) gallery, it will still show the currently active language based on the current tags, but you will not be able to change it from the uploader interface.

Before anyone tries to argue whether a language should be "common" or not, the selection list is dynamically generated and is purely based on the number of galleries using the tag for the language.

To prevent excessive clutter, languages with less than 5 tagged galleries are not included in the list. Any new language tags are automatically added to the list when they reach 5 tagged galleries.

If the category is set to "Image Set", the language is ignored if it is set to "Speechless" or "Text Cleaned". Similarly, if the category is set to "Game CG", the language is ignored if it is set to "Speechless". Note that it allows you to set it, it's just ignored when publishing.


- A new image previewer for unpublished galleries was added, primarily because galleries now only "exist" in the uploader until they are published. This will always use the 1280x resamples if available for the image, originals otherwise.

While previewing an image, you can use right arrow/numpad or D to go to the next page, and left arrow/numpad or A for the previous page. You can also close the image preview with spacebar.

Unlike the preview on the old uploaders, this preview will immediately mirror any file order changes or deletions, so you should be able to quickly check the new page order before saving the changes.


Changes and Fixes

- The way unpublished galleries are stored in the system was fundamentally changed; among other things, this fixes a problem where galleries that were created a long time before they were published would "bypass" the Watched page.

- After publishing, a gallery now have an explicit "processing" state while some final background tasks take place, where they are fully functional except that some thumbnail variants aren't generated yet. They can be accessed (and tagged) if you have the link, but aren't yet publicly listed.

- Thumbnails are now pushed to the standard thumbnail servers immediately rather than as a scheduled background job. The uploader thumbnail servers will be retired together with the old uploaders, and the "Asia delay" will be removed when this is done.

- Reworked some of the mechanisms to prevent excessive expunge posters and new gallery posters from spamming the system. Among other things, this check now happens at publish time rather than gallery creation time.

- You can now create a maximum of 250 upload folders. Note that if you are one of the 18 people who currently have more than this, you might want to consolidate folders manually, as any excess folders may be dropped in a future update, setting those galleries as "Unsorted".

- You can now have a maximum of 1000 unpublished galleries.

- Unpublished galleries will now be deleted 1 year after they were last touched. Older unpublished galleries will have a moratorium of 1 year from the time they are imported to the new system.

- Corrected an issue where restrictive/broken permissions for files in ZIP archives could prevent files from being included. Note that filenames in ZIP files still have to be unique across any subdirectories, otherwise only one of the files with the same filename will be included.

- Support for RAR archive files has been dropped.

- The same exact file can now only appear in a gallery once, mostly to prevent any lingering issues with accidentally starting multiple uploads at the same time. This is not retroactive to old published galleries, but if an old published gallery is cloned and when an old unpublished gallery is imported to the new system, any duplicate files will be removed.

- The same exact file with the same exact filename can now only appear in a gallery once, mostly to prevent any lingering issues with accidentally starting multiple uploads at the same time. This is not retroactive to old published galleries, but if an old published gallery is versioned/cloned, any duplicate files with the same exact filename will be removed; a warning will be printed in this case. Old unpublished galleries will also have any duplicated pages with identical file+name removed when imported to the new system

- When trying to access a replaced gallery in the uploader, it will now forward you to the current (published or unpublished) version of the gallery.

- The readout for "Child Gallery" on the manage screen was never actually used since older versions of a gallery could never be "managed", and was removed.

- "Standard Sort" was renamed "Lexical Sort" to be less ambiguous.

- Galleries that are being updated/replaced are now still shown in the uploader list until the replacement is published, and will have a link to the replacement gallery.

- The category selector was moved, and the "make public" checkbox was removed in favor of a separate option in the selector drop-down.

- Actions that are no longer available on a gallery (such as changing the category or name on non-recently published galleries) are now grayed out instead of just returning an error if you were to try to change them.

- Other minor UI tweaks and fixes.

I'm so happy to see upgrades (IMG:[invalid] style_emoticons/default/smile.gif)

Thankyou so much for your work tenboro

e-hentai is a so much important part of my life! (IMG:[invalid] style_emoticons/default/tongue.gif)

Hope there are some notice in the UI when a unpublished gallery will be deleted.

Sort by time is a blessing, now I will wait forever for more favorites categories :v

Cool. Kind of glad my old unpublished tweak of a published gallery that I have no intent of ever publishing will eventually leave my uploads screen now.

I'll still need to fix my "dark mode" CSS for this new page, but that's already 90% done by just adding upload.e-hentai.org and upld.e-hentai.org to the list. Not a huge fan of rounded squares, but it's not a life altering change. Might even be something I forgot from a prior update.

Also happy to hear we've dropped RAR support; there's little to no reason to use RAR anymore and its use should be actively discouraged in favour of more open formats.

…And script is updated. Was a pretty trivial couple of tweaks.

This post has been edited by dragontamer8740: Sep 27 2021, 16:02

I don't really get the disown (but then again I'm tired). So what will happen to the gallery? I get we won't find it via the uploader name.

Will it lose the uploader name? So the gallery would "anonymous"?

>Added time-based auto-sorting

Perfection.

>- Support for RAR archive files has been dropped.

Oh no I paid for WinRAR!

10/10 updates, thanks!

I don't really get the disown (but then again I'm tired). So what will happen to the gallery? I get we won't find it via the uploader name.

Will it lose the uploader name? So the gallery would "anonymous"?

Here's one I disowned if you want to see how it works https://e-hentai.org/g/1716636/1666f3899d/

Hope there are some notice in the UI when a unpublished gallery will be deleted.

Seconding that hope. It's just polite, added to being useful.

I made a small tweak to the gallery locking mechanism so whenever you try to publish, reorder or delete a gallery that was locked by a different process (usually an upload), it will bail out with an error message instead of waiting on the lock, since this made it seem like the page wasn't loading.

Hope there are some notice in the UI when a unpublished gallery will be deleted.

You mean automatically? Well, the pruning mechanism isn't actually implemented yet, but yeah, it's probably a good idea to put some red text at the top whenever something is closed to being pruned...

Here's one I disowned if you want to see how it works https://e-hentai.org/g/1716636/1666f3899d/

Perfect, thanks buddy.

The same exact file can now only appear in a gallery once, mostly to prevent any lingering issues with accidentally starting multiple uploads at the same time.

Just for my peace of mind, does "the same exact file" include images that are identical except for the filename? If so works that have multiple versions (e.g. with/without extreme content) in one gallery will have random parts missing.

Just for my peace of mind, does "the same exact file" include images that are identical except for the filename? If so works that have multiple versions (e.g. with/without extreme content) in one gallery will have random parts missing.

Files that are byte-for-byte identical. Filenames and visual similarity don't matter.

Files that are byte-for-byte identical.

That's concerning to me. (scat warning for the linked gallery, the linked images are safe) This, this and this image for instance are byte-for-byte identical and in the same gallery (but with different filenames), because the artist provided multiple versions of the work with different fetish content but not every image needed to be changed.
I have some ongoing translation projects of galleries like this, so if I clone an existing one of these galleries under the new system will these identical images from it disappear and someone trying to read a particular version will have to hunt for the missing ones?

Going forward I guess works like this can be uploaded as multiple galleries, one per version, but it's going to be an awkward transition.

That's concerning to me. (scat warning for the linked gallery, the linked images are safe) This, this and this image for instance are byte-for-byte identical and in the same gallery (but with different filenames), because the artist provided multiple versions of the work with different fetish content but not every image needed to be changed.
I have some ongoing translation projects of galleries like this, so if I clone an existing one of these galleries under the new system will these identical images from it disappear and someone trying to read a particular version will have to hunt for the missing ones?

They will not appear multiple times in the gallery, no.

Edit: Originally this was a design trade-off where I didn't think the ability to duplicate an image was really desirable, but looking at the stats from recent Artist/Game CG uploads, it seems to be a lot more common than I thought, so I probably will have to change it. (Stuff like this is why there's a beta.)

Note that I will be taking the beta down in an hour or so to apply some changes. I don't expect this to take very long, maybe an hour or two.


Update: The beta has been re-opened.

The following things have been changed:


- The ability to have identical files in the same gallery has been re-added, but it will only allow duplicates if the filenames are different. (Duplicate filenames are still allowed if the files are different.)

This should preserve the desired property where accidental duplicate uploads are prevented, while also allowing CG sets in particular to be uploaded as intended.

The "Time Added" autosort mechanism will take the filename into account, so if a file is added to multiple versions of a gallery with different filenames, it should preserve each copy at the intended order. Furthermore, if you upload an existing file with a new filename, it will be treated as a newly uploaded file for these sorts.


- Added a new "Auto-Sort" method called "Remove Exact Duplicates" which, as the name implies, will remove duplicated files from the gallery. The first file in the current page order will be preserved.


This is a somewhat larger backend change than I would usually make during a beta, so please try it out and let me know if you encounter any weirdness. As always, anything goes wrong with an updated gallery, you can always delete it and start over; deleting the updated version will also let you update it using the old uploaders instead.

I'll update the patch notes to take these changes into account in a little bit.

Seems to be a little funkiness with zip files I think

The zip I was using: [files.catbox.moe] https://files.catbox.moe/tx1vlo.zip

I first used this gallery https://e-hentai.org/g/2024540/f45d0a816e/ , where I was testing the new remove dupe feature (confirmed dupes with different filenames work, dupes with same filename fail properly, and remove dupe feature works)

I cloned and unlocked it (currently unpublished [upld.e-hentai.org] https://upld.e-hentai.org/managegallery?ulg...added=3#upload)

and uploaded that zip

In total there were 4 aces, which I expected only 2 to be duped, with the other 2 rejected for same file names and same file, but I only got 1

2 of spades didn't dupe which was expected
4 of spades was added which was expected
5 of spades didn't add, which was unexpected

It gave me the expected warning that some files weren't added


On a brand new gallery [upld.e-hentai.org] https://upld.e-hentai.org/managegallery?ulg...;added=3#upload I uploaded the zip alone

What I expected to happen, is for
2x Ace of spades
1x 2 of spades
1x 4 of spades
1x 5 of spades

What I got was 1x Ace, 1x 2 of spades, 1x 4 of spades, 5 didn't add at all
It didn't give the warning that some files weren't added this time

5 of spades had the same filename as the ace of spades, so I'm wondering if maybe zips are only going by the filenames alone, it's the only pattern I can see that would result in those files. Same filenames but different images work when uploaded without using a zip

Seems to be a little funkiness with zip files I think

I think the issue you're hitting is that files in ZIP archives need to have unique filenames across all subfolders, otherwise the unzip process itself will eat them. This is a known/documented limitation.

It should give you an error message for every file that was skipped, so if it doesn't print one for some files, that means it was eaten before the actual uploader code got to it.

Ah, I see
I'll add some notes about that to the uploading guide when we update it

Another minor change:

- When versioning/cloning an old gallery that had duplicate files with identical filenames, it will now print a warning that those pages were not included.

R.I.P RAR it was nice knowing you...we had a good 10 year run...great times

seriously RAR was 10x better than ZIP and now we can no longer use it? at least implement 7z for god's sake

Archives in general are kind of obsolete for uploading, it was really only useful back in the days when browsers only let you select one file at a time to upload.

And 7z can be unpredictably resource-intensive to extract, so no.