[X] My Assistant

I actually know where this image is from... The tags for this one seem super accurate. The "mature" tag is interesting; I think Takizawa is around 30 years old, so the "mature" tag is reasonable considering the average age of waifus on the boorus.

Best OL. I downloaded that to be my avatar originally, as I like the manga, though I (mostly) tend to keep to less explicit avatars now.
Incidentally, it came from sankaku, and I've seen mature tagged on loli over there, so it's already doing better than real humans.

I'm more curious about what it uses to pick tags based on images that dont belong on booru's, i.e the text one I uploaded first. "looking at viewer", 1girl", "smile" , must have seen a face in there somewhere.


This post has been edited by Ubershank: Jan 7 2020, 21:02

Ok Mikoyan Gurevich's method proved to be superior.

Everyone that has plenty of GP, join us to create a user backup this "library of Alexandria" to be save in the worst case scenario.

Mikoyan Gurevich already has 10 TB saved and once I started ammasing enough GP I will follow.

But the more people join the better. The Galleries have priority, since the take the most space, but the metadata will follow through SQL.

Or what would you say if e-hentai vanished from one day to another?

The Galleries have priority, since the take the most space, but the metadata will follow through SQL.

What's space got to do with priority? If you can't fit them both, it's pointless. Good luck finding that 10 year old gallery you vaguely remember what was in it, but not it's name or who drew it.

Or what would you say if e-hentai vanished from one day to another?

Go fap to it at low resolution on one of the many sites that rip e-h daily. Or torrent it from nyaa. The problem with e-hentai disappearing isn't just the potential loss of data, its the scattering of active uploaders/translators/editors, will be dark times for hentai when all those doujins are uploaded to random sites across the interwebs, sometimes behind paywalls, or patreon links, or whaver, and hosted by cheap companies that will take them down at the first dmca request.

Edit: That being said, backing up isn't a bad thing. If you're gonna focus on the galleries rather than the meta data (more fool you if you do), do at least prioritize the raws, and not just the English translations.

Edit2: How rude of me, I almost didn't mention a loss just as big as the porn if the site went down, the loss of the community.

This post has been edited by Ubershank: Jan 13 2020, 02:16

I'm joining the talk, a bit late maybe, but without going on the talk again what about the "reader"?

For now we have HappyPanda(x) and LANraragi, I don't use (yet) any of them but can they actually be a safe reader? Can they get the metadata easily? I'm using, and I guess other do too, the x/gallery-metadata script to get the .json. Which I save with the gallery (same folder basically).

That part is I think quite important too.


Anyway, if there is a place to talk and coordinate the backup I'm quite interested in joining the task.

I'm joining the talk, a bit late maybe, but without going on the talk again what about the "reader"?

For now we have HappyPanda(x) and LANraragi, I don't use (yet) any of them but can they actually be a safe reader? Can they get the metadata easily? I'm using, and I guess other do too, the x/gallery-metadata script to get the .json. Which I save with the gallery (same folder basically).

That part is I think quite important too.
Anyway, if there is a place to talk and coordinate the backup I'm quite interested in joining the task.

I use Lanraragi, and found it to be a lot better than HappyPandaX personally. They can both scrape meta-data from various sources, but all that does is add it to your database. What you actually want to do, is include json files for all the galleries inside the gallery archives/folders. I recommend taking a look at [github.com] https://github.com/dnsev-h/ehentai-archive-info/ which can scan through your library and download the corresponding meta-data. Just be sure to configure it properly first, you can filter out a lot of things to improve its accuracy, for instance languages.

Right now, my archive is sitting at 300gb with 3.8k (English translated) galleries, and I use dnsev's fork of E-H Downloader to grab most regular sized galleries, since it actually supports his json file meta-data format (eze is depreciated). [github.com] https://github.com/dnsev-h/E-Hentai-Downloader just turn off saving meta-data as a info.txt file in the settings, because this fork will include the json file even if it's turned off and you don't need both.

I wish I could use the archive downloader more, but with it not being able to grab these json files with meta-data I find myself not using it. The same is true for torrents, I value having archive names that match the gallery title and meta-data files too often to use them that much. I know it hurts the servers more, but there's not much choice. To put it simply, downloading a torrent, and then downloading the meta-data separately to include it in the archive, and then not being able to seed it, or having to keep a duplicate in order to seed... becomes a major pain. Which is why I tend to only torrent large galleries, especially tankoubons.

But when you do what I do and use scripts to download galleries original files you will end up using all your GP fast. My 1.6 million GP i've accumulated is completely gone. Trying to back things up is really hard, I do at least run H@H to try and make up for me abusing the gallery downloads.

This post has been edited by Black-lights: Jan 16 2020, 17:42

Thank you for your effort and information.

The reasons you mentioned make it crucial to work together on this matter.

Let's say the whole gallery is 50tb, having 100 people split the work would end with 500GB each.

Still a lot, but the annoying part is to get it all together at the beginning. Once we have everything it's basically just about updating it.

Metadata is useless without the actual gallery. Of course backing up the metadata itself is part of the project, but in the worst case scenario I rather have the galleries and simply search terms of interest on different sites like doujinshi.

This post has been edited by nobodyserio: Jan 22 2020, 19:23

Does anyone have the metadata for each gallery?

I have a script that can check if a gallery is unavailable or not. I'll need the URL (token and gid) for each gallery or else it won't work.

Metadata is useless without the actual gallery. Of course backing up the metadata itself is part of the project, but in the worst case scenario I rather have the galleries and simply search terms of interest on different sites like doujinshi.

No, the metadata is the most important part. It allows us to figure out if a gallery has been removed or not. It tells us how big the gallery is, and when it was uploaded, and by whom, and much more.

This post has been edited by saythe14wordsforme: May 20 2020, 21:15

No, the metadata is the most important part. It allows us to figure out if a gallery has been removed or not. It tells us how big the gallery is, and when it was uploaded, and by whom, and much more.

Metadata will not tell you whether a gallery has been removed. The expunged value in the metadata is not used to flag removal. You can only check that by going to the gallery.

Also, do not necropost.

Actually depending of the tool you use, you can have the "visibility" of the gallery.

Actually depending of the tool you use, you can have the "visibility" of the gallery.

Standard EH metadata are downloaded from EH API, but I am sure some tools or other websites can give additional metadata. I was going to add an option to check visibility in my LANraragi metadata plugin myself, but scrapped that because I thought some users probably cannot set stuff up properly.

Metadata will not tell you whether a gallery has been removed. The expunged value in the metadata is not used to flag removal. You can only check that by going to the gallery.

Also, do not necropost.

Did you just delete my post when all I did was correct you?

Metadata will tell you if a gallery has been removed or not.

Why are you making things up all the time?

Actually depending of the tool you use, you can have the "visibility" of the gallery.

No special tool is required. You only need the gid and token.

This post has been edited by saythe14wordsforme: May 21 2020, 21:34

Did you just delete my post when all I did was correct you?

If your post was deleted, it wasn't by Mayriad. He does not have the ability to do so.

It was probably due to the necro.

Did you just delete my post when all I did was correct you?

If your post was deleted, it wasn't by Mayriad. He does not have the ability to do so.

It was probably due to the necro.

I am not aware of any post by saythe14wordsforme that got deleted. His post history does seem to be missing one post, but his posts in this subforum are all up. I have no idea of what I am being accused, but like Mrsuperhappy said, I do not have the power to delete posts, so I cannot be me.

This post has been edited by mayriad: May 21 2020, 22:50

Did you just delete my post when all I did was correct you?

I got a report. Considered that a post saying "you're wrong because you're wrong and stupid" did not belong in this section and therefore executed the report.

May that serve as a warning btw. This is not the FSZ, if you want to actually correct someone then (1) make sure you are correcting someone instead of posting something that is untrue and (2) give evidence. A post "you're wrong, the sky is green. I needed to correct you" is not welcome here.

You have been warned, I will delete such posts without further warning. PS. this thread is not a discussion about your posts.

I got a report. Considered that a post saying "you're wrong because you're wrong and stupid" did not belong in this section and therefore executed the report.

May that serve as a warning btw. This is not the FSZ, if you want to actually correct someone then (1) make sure you are correcting someone instead of posting something that is untrue and (2) give evidence. A post "you're wrong, the sky is green. I needed to correct you" is not welcome here.

You have been warned, I will delete such posts without further warning. PS. this thread is not a discussion about your posts.

I said it was wrong and I explained why. I never called that person stupid. Why are you lying?

This post has been edited by saythe14wordsforme: May 22 2020, 01:02

I said it was wrong and I explained why.

No, you simply said it was wrong.

Anyway;

PS. this thread is not a discussion about your posts.