[X] My Assistant

PENGUIN EDIT: there is an official stance about this now, see below in this same thread (or read the quote if you are lazy)

As for the relevant rules have a look at the definition of superior versions on the wiki:

Images should be as released by the source. Conversions, "retouches", leveling, splitting double-pages into singles, or removing original source watermarks do NOT count.

https://ehwiki.org/wiki/Expunging#Superior_Versions

And the stance of the admin/mod team is that waifu2x is a valid form of retouching a scan. All other relevant rules for superior versions (on that same page) also remain intact.

In summary:

1. A gallery retouched by waifu2x is a valid gallery
2. A gallery retouched by waifu2x does NOT constitute a superior version to an existing non-retouched gallery
3. A non-retouched gallery does NOT constitute a superior version to an existing gallery retouched by waifu2x, unless it is a digital release in which case any form of retouching would not be consistent with "as released by the source".

waifu2x is a deep convolutional neural network. It does not perform a geometric (binomial or cubic) upscale of an image. Instead it bends the information contained in the image towards a more anime-like appearance. The fact that it is capable of producing bigger (in terms of resolution) is not relevant. The reason that a geometric upscale is bad is because it reduces the amount of information per square inch (or any other measure you may use) of an image. waifu2x does not reduce the amount of information per square inch, it fill in information (guesses) based on the networks knowledge about anime. This produces a slight loss of information of the original image whilst improving the overall consistency of the image by filling in information that seems more relevant.

By that evaluation waifu2x performs the same work as (good) grayscale levelling, loss of information which is weighted by image consistency.

This thread is now sticky, to ensure that everyone relevant sees it. (And to work as a precedence to similar software)

I'm sure by now quite a few of the people in here will have used Waifu2x, or at least heard of it.

It's a great thing in name. Allows you to clean up a artifacted, JPG image and put out a nice, clean, lossless PNG, which is a definite boon for any translators. I've actually used this functionality of it a time or two - it's great for that purpose.

But when people are using its ability to upscale images in order to game the system - I have a serious problem with that.

I hate to call out names, but I'm starting to see several uploads, from a particular (and well-known) uploader on the site, putting up CG sets that are way higher resolution than the "official" artist releases for sale on websites like DLSite. This means not only are they getting more GP with the upload (because it's a much bigger download), but in theory, this could be used to expunge "lower quality" versions of the same galleries, when in reality they're not lower quality - someone upscaled the originals via Waifu2x. That's not cool.

I think we should seriously consider the effects of this - especially since right now, very few of us have a monitor that could even properly display a 4000x3000 image.

It's important to note I'm perfectly fine with Waifu2x as a de-noiser and JPG -> PNG converter. I think that's a very worthwhile functionality, and that functionality can be entirely separated from the magnification one. But I also think that we shouldn't be letting people artificially blow up images and gain from it.

This post has been edited by blue penguin: May 12 2017, 00:26

We only accept images as they are from their source (aside from decensoring, colorizing, etc.) If you have any proof of modification or upscaling the gallery will be expunged.

Although it is against the rules, I think it is actually quite nice to have DCNN-magnified galleries, as long as the originals are protected, of course. Waifu2x is just flawless when it comes to upscaling of 2D graphics, but it can take a very long time to magnify even one picture and few people have the processing power/time/patience/courage to upscale entire CG sets. I am not trying to change the rules, but I would say if someone spent that time and processing power, then he should get some return for his efforts.

By the way, are you sure those CG sets you mentioned are artificially upscaled using Waifu2x? I find it hard to believe that someone has the generosity to run hundreds and thousands of pictures through Waifu2x, likely twice to achieve that "way higher resolution"... CG sets often include multiple subsets of very different dimensions and you could make a mistake if this information is not available to you.

Although it is against the rules, I think it is actually quite nice to have DCNN-magnified galleries, as long as the originals are protected, of course. Waifu2x is just flawless when it comes to upscaling of 2D graphics, but it can take a very long time to magnify even one picture and few people have the processing power/time/patience/courage to upscale entire CG sets. I am not trying to change the rules, but I would say if someone spent that time and processing power, then he should get some return for his efforts.

By the way, are you sure those CG sets you mentioned are artificially upscaled using Waifu2x? I find it hard to believe that someone has the generosity to run hundreds and thousands of pictures through Waifu2x, likely twice to achieve that "way higher resolution"... CG sets often include multiple subsets of very different dimensions and you could make a mistake if this information is not available to you.

With a good GPU, rescaling a single image in Waifu2x is literally a matter of a few seconds. I could do a 150-image set in perhaps a few minutes (but admittedly, I've got a GTX 1080, which is a $700 GPU). Admittedly, I've got a better GPU than most, but if you use Waifu2x-caffe and have an even halfway decent GPU, even a 2x resize only takes on the scale of ten to twenty seconds - at worst. It's definitely nowhere near on the scale of hours.

And yes, I am sure. I check the source release that any good uploader will put for where the set can be legitly gotten, if I feel there's no way the artist released at that rez. The DLSite page for it makes it clear the original images are 1600x1200, and a DL size of about 350 MB, but the gallery on e-H is 4000x3000 and a DL size of a whopping 1.21 GB. Usually if there's multiple resolutions, the remainders are smaller, or else only the biggest resolution images in the set are uploaded - even then, you can almost always tell what the resolution of the original images are from places like DLSite.

They've definitely been upscaled. If they weren't, I wouldn't be crying foul.

This post has been edited by Dark Pulse: Nov 28 2016, 05:22

Ugh, apologies, my assumptions were invalid! Waifu2x is magnitudes slower on my computer; I guess a GTX1080 really helps. So the whole thing is quite possible then.

the original images are 1600x1200, and a DL size of about 350 MB, but the gallery on e-H is 4000x3000 and a DL size of a whopping 1.21 GB.

I have found this gallery(s) and I think you are indeed correct. It looks obvious to me as well, especially given the older original gallery. This is a weird thing to do...

If you find a gallery that is definetly upscaled, or otherwise does not match the source, then it should be expunged in favour of one that matches the source material.

*edit* other than the exceptions Joe mentioned (decensors and the like)

This post has been edited by Mrsuperhappy: Nov 28 2016, 13:47

Out of pure curiosity I went and read what Waifu2x actually does. [arxiv.org] It is a convolutional neural net, therefore it is a stochastic algorithm. And, therefore again, running it two times on the same image should produce a slightly different output due to different neural net weight initialization.

So, it isn;t actually possible to prove with 100% accuracy that an image has been produced with Waifu2x upscalling.

Moreover, since it is a convolutional neural net, it is closer to a redraw than a computational rescale. That is because the neural net can be thought as a simulation of a human-like being evaluating several results and selecting the best one (that's hugely oversimplified but gives an image of how an error function works).

Yeah it is not conventional upscaling. It may not be technically possible to prove, but I think right now Waifu2x is the only popular algorithm capable of doing this. If I see a flawlessly upscaled 2D image, I can only guess Waifu2x. The actual problem is upscaling though, not the tool used.

Care to provide actual links to support what you're saying, or are we supposed to imagine which galleries you speak of?

Out of pure curiosity I went and read what Waifu2x actually does. [arxiv.org] It is a convolutional neural net, therefore it is a stochastic algorithm. And, therefore again, running it two times on the same image should produce a slightly different output due to different neural net weight initialization.

So, it isn;t actually possible to prove with 100% accuracy that an image has been produced with Waifu2x upscalling.

Moreover, since it is a convolutional neural net, it is closer to a redraw than a computational rescale. That is because the neural net can be thought as a simulation of a human-like being evaluating several results and selecting the best one (that's hugely oversimplified but gives an image of how an error function works).

It's a very nice thing in terms of both upscaling and denoising (a function I've used myself). The main problem is that while denoising isn't really going to fundamentally alter things in terms of file sizes (and thus rewards), magnification will.

As for your assertion it's not possible to prove with 100% accuracy, while technically true, that's kind of akin to saying that some other form of upscaling is okay, when really the upscaling is the problem period. While Waifu2x leaves no sort of "mark" on its images, we can still go by what the original DL size/resolution information is off a release site. If it doesn't match up, it's been magnified SOMEHOW. It may not be strictly with Waifu2x, but it's almost certainly the only filter capable of looking "good enough" since any rote bilinear or bicubic or lanczos or whatever filter is going to show some effects from it - Waifu2x really won't.

Care to provide actual links to support what you're saying, or are we supposed to imagine which galleries you speak of?

There's at least two uploads that I know of for sure.

Resized Gallery: https://e-hentai.org/g/989711/bf2801f08a/
Original Release Link: [www.dlsite.com] http://www.dlsite.com/maniax/work/=/product_id/RJ178491.html

Resized Gallery: https://e-hentai.org/g/985706/7d1536d295/
Original Release Link: [www.dlsite.com] http://www.dlsite.com/maniax/work/=/product_id/RJ185659.html

As both the original links show, the DLSite description indicates the source image rez is 1600x1200. These sets are anything but that, as indicated by the "Download original 4000x3000 source" link on the bottom. If the official releases are 1600x1200, why is this one's source resolution 4000x3000?

The answer is simple: These have been magnified 2.5x, and given the quality of it, presumably by Waifu2x. Even if not done with Waifu2x, either way it's a rather shady way of artificially boosting rewards gotten, no matter how it was resized.

This post has been edited by blue penguin: Dec 8 2016, 01:43

^ I needed to edit your post. The word filter exist for reasons.

Anyhow, the real point is not how you can prove that something is upscaled by Waifu2x but:

(...) it is closer to a redraw than a computational rescale. That is because the neural net can be thought as a simulation of a human-like being evaluating several results and selecting the best one (that's hugely oversimplified but gives an image of how an error function works).

Neural nets are neat things, and I do not see issues in using good neural nets to produce something that looks like it was done by a human being.

Am not the person deciding on this. But, since I wrote enough neural nets myself, I'll argue that NNs are pretty good simulators. The big issue with NNs is the huge computation power they require (on webcoin mining level). So, having someone pay for that energy and then sharing the result is a plus.

There's at least two uploads that I know of for sure.

Resized Gallery: https://e-hentai.org/g/989711/bf2801f08a/
Original Release Link: [www.dlsite.com] http://www.dlsite.com/maniax/work/=/product_id/RJ178491.html

Resized Gallery: https://e-hentai.org/g/985706/7d1536d295/
Original Release Link: [www.dlsite.com] http://www.dlsite.com/maniax/work/=/product_id/RJ185659.html

As both the original links show, the DLSite description indicates the source image rez is 1600x1200. These sets are anything but that, as indicated by the "Download original 4000x3000 source" link on the bottom. If the official releases are 1600x1200, why is this one's source resolution 4000x3000?

The answer is simple: These have been magnified 2.5x, and given the quality of it, presumably by Waifu2x. Even if not done with Waifu2x, either way it's a rather shady way of artificially boosting rewards gotten, no matter how it was resized.

It is a something of a moot point while they are the only examples of those galleries that are actually here, but if someone were to upload them in the correct scale then to my mind these inflated ones (no matter how well done) should be expunged, as you say, if we have the original sources to compare to.

^ I needed to edit your post. The word filter exist for reasons.

Fine, but... then those provided links don't work, so I hope people understand what it means and what to do. I tried to be obscure; guess that's not obscure enough.

Anyhow, the real point is not how you can prove that something is upscaled by Waifu2x but:Neural nets are neat things, and I do not see issues in using good neural nets to produce something that looks like it was done by a human being.

Am not the person deciding on this. But, since I wrote enough neural nets myself, I'll argue that NNs are pretty good simulators. The big issue with NNs is the huge computation power they require (on webcoin mining level). So, having someone pay for that energy and then sharing the result is a plus.

To the point where an upload that's larger than the source material - and the rewards thereof of increased sizes for DLs and so on - are perfectly kosher?

If so, the point of uploading the "highest resolution available" is 100% moot. You can upload whatever upscaled resolution you want; "highest resolution available" becomes meaningless except as a base and to minimize the amount of work that Waifu2x needs to guess. That doesn't somehow seem right.

Especially since with a decent videocard, this is literally not that much extra power at all. I can Waifu2x an image on my GPU at 2.5x magnification and it took me maybe about two minutes to do five images from 1600x1200 to 4000x3000. It's a lot longer if you're doing CPU-only magnification, but if you got a good GPU, it's seconds per image - in this case, perhaps about 20 seconds per image to denoise and magnify it 2.5x. Obviously it will be slower or quicker depending on the source resolution.

(To be fair, I do have a top of the line GPU, but even on older GPUs it wouldn't take too much more time.)

It is a something of a moot point while they are the only examples of those galleries that are actually here, but if someone were to upload them in the correct scale then to my mind these inflated ones (no matter how well done) should be expunged, as you say, if we have the original sources to compare to.

We do have at least one.

https://e-hentai.org/g/946029/9747bdc1ef/

(Granted, this one has its own problems, in the form of watermarks...)

This post has been edited by Dark Pulse: Dec 8 2016, 02:14

Fine, but... then those provided links don't work.


We do have at least one.

https://e-hentai.org/g/946029/9747bdc1ef/

(Granted, this one has its own problems, in the form of watermarks...)

Your links work fine, don't worry about it.

The quality of this one isn't as good as the larger sized one. I do not have the source to compare it to, but my opinion of expunging the inflated ones once we have one that matches the source relies on it being a good version (so not watermarked etc). Nobody is losing out at the moment even if the uploader is doing it for more e-currency as the 'viewer' is getting a gallery if they want to download it, the archive gives them the choice of smaller sizes.

Your links work fine, don't worry about it.

Huh. So they do. Okay, it must've just been something weird on my end then, as I was getting "Connection refused" even though going to the portal and picking some random gallery worked. It must've been something weird on my end, so apologies.

The quality of this one isn't as good as the larger sized one. I do not have the source to compare it to, but my opinion of expunging the inflated ones once we have one that matches the source relies on it being a good version (so not watermarked etc). Nobody is losing out at the moment even if the uploader is doing it for more e-currency as the 'viewer' is getting a gallery if they want to download it, the archive gives them the choice of smaller sizes.

Quality of the originals is never going to be as good as something pushed through Waifu2x, for three big reasons:

1. Waifu2x by default will convert whatever the image source is (usually JPG) to lossless PNG. Obviously this means it's not going to suffer quality loss on rescales, unlike the originals.
2. Waifu2x allows for denoising of those JPGs before it upscales. This produces higher quality than the JPG originals, and is the one functionality I'd be 100% fine with on this site, as it's not fundamentally altering the image so much as removing image compression artifacts (i.e; an "ideal" source image wouldn't have had those in the first place, but you only get that with stuff that was saved by the creators themselves as PNGs from the get-go).
3. The aforementioned upscaling.

I do know that the gallery stuff has been updated to download at user-selectable sizes, but then as I said, it pretty much defeats the point of "Higher resolution available" in the first place. Going by what you said, someone can just grab the originals, scale them up, denoise them, and that's a valid gallery even though the original creator never created the images in that resolution.

I'd like to know this myself, especially as I plan to be uploading translated content to this site relatively soon. If it's kosher by the rules, then at least that means I won't have to feel guilty about releasing upscaled work, but to my knowledge, this sort of thing flies in the spirit of the site's rules, which is that aside from decensoring/translation/cleaned images (i.e; denoised JPGs, balloon-cleaned mangas for translators, etc.), any other sort of image modification (which I'd consider upscaling to be) is forbidden, and I don't want somebody being able to get my uploads expunged because they took the originals, threw them through Waifu2x, and then my gallery is suddenly "lower quality."

It'd be a never-ending resolution arms race in no time.

This post has been edited by Dark Pulse: Dec 8 2016, 02:56

I do know that the gallery stuff has been updated to download at user-selectable sizes, but then as I said, it pretty much defeats the point of "Higher resolution available" in the first place. Going by what you said, someone can just grab the originals, scale them up, denoise them, and that's a valid gallery even though the original creator never created the images in that resolution.

My point here was that both those galleries you linked (the inflated one and the one in the original scale) are from the same uploader, so nobody has lost out, it isn't a situation where the uploader of the larger gallery has stolen any hits from the uploader of another.

I'd like to know this myself, especially as I plan to be uploading translated content to this site relatively soon. If it's kosher by the rules, then at least that means I won't have to feel guilty about releasing upscaled work, but to my knowledge, this sort of thing flies in the spirit of the site's rules, which is that aside from decensoring/translation/cleaned images (i.e; denoised JPGs, balloon-cleaned mangas for translators, etc.), any other sort of image modification (which I'd consider upscaling to be) is forbidden, and I don't want somebody being able to get my uploads expunged because they took the originals, threw them through Waifu2x, and then my gallery is suddenly "lower quality."

It'd be a never-ending resolution arms race in no time.

I agree with you, and I do not believe it would be within the rules, as far as I am concerned we just need a 'correct' gallery to be uploaded to replace the upscaled ones. I just don't think the watermarked version from the same uploader is worth expunging the bigger gallery for at the moment.

My point here was that both those galleries you linked (the inflated one and the one in the original scale) are from the same uploader, so nobody has lost out, it isn't a situation where the uploader of the larger gallery has stolen any hits from the uploader of another.

Right, and I get that. But obviously, sooner or later, this will happen, and a decision probably has to get made somewhere as to what's preferable - one that keeps the rez but perhaps denoises the original (which I'd be fine with), and resolution at all costs.

I know I'd be miffed if I released something, and someone else put up an upscale and mine was judged to be inferior, even if I literally uploaded the original files.

I agree with you, and I do not believe it would be within the rules, as far as I am concerned we just need a 'correct' gallery to be uploaded to replace the upscaled ones. I just don't think the watermarked version from the same uploader is worth expunging the bigger gallery for at the moment.

Which would then mean that basically your stance is "Upscales are allowed until a provable original-resolution, unaltered version is made available in the galleries; once it is, the upscales get expunged." That's a fair enough stance to take.

We just need a way of being able to confirm that somehow. That's going to be the tricky aspect, especially for stuff that's not on DLSite or DMM or whatever - not every maker nicely says what resolution the stuff is, and the only way to be 100% sure is for someone to buy a copy to verify.

"highest resolution available" becomes meaningless

I really hope so. i.e. I really do hope that we will get to almost perfect aspect translation given that the original is close to the limits of a human eye sensitivity. And I wouldn't give it more than 5-10 years to get there. We will need to think of several clever rules by then, but today we do not even know how to properly evaluate NN output.

Can you imagine?! NNs would be able to do descensors. That would be awesome.

And 20 seconds is actually a pretty long time. A 1000 page gallery would run for 55 hours non stop on a GPU. For easy estimates let's say that the GPU plus the controlling machine uses 500 Watts (that's pretty sensible: 400W GPU plus the computer itself), so you get 27.5 kWh. At the kWh price where I live (12 pence), that's still 3.30 GBP (~4 USD). That's comparable to the price of the H-game sometimes. And that's a fuckload of energy to be fair.

Basically, the argument is that: It does contribute something (It would not if that power was used for plain algorithmic upscale). Whilst searching for such upscales is a lot of effort. In simple words, not a problem worth going after.

I can agree with Mrsuperhappy though: If you can prove what the artist released something as A, then that should get precedence (just like translator precedence). But expunging something just because someone is making more GP out of it because he ran a big NN on it is silly. And searching for the version released by the author may be a freaking pain, e.g. in (decently secured) DRM games. For example:

and the only way to be 100% sure is for someone to buy a copy to verify.

Even that may not be true. DLsite is famous for adding more censorship to CGs than the same CGs on melonbooks (which means that DLsite runs the images through imagemagic or similar).

I know this isn't the point of the thread, but I played around with Waifu2x a little bit after learning about it from this thread and I decided to play around with it to figure out what exactly was being talked about. And yeah, while the noise reduction is the very definition of a joke when it comes to that whole uncensoring thing the ease at which it could upscale the converted pictures without a loss of quality is actually quite impressive. I don't think it should be used by the people uploading on the site as a quick way to get extra GP, but I'm also kind of impressed by some of the results. Keep in mind that is in general, as there are cases where the results can only be classified as bad, or they loose some of the finer qualities depending on which one you start with.

In the end, I think if the original artist uses it, then there really isn't an issue, or even just using it as a way to convert from image formats without upscaling isn't an issue, but the upscaling is the problem.

I know this isn't the point of the thread, but I played around with Waifu2x a little bit after learning about it from this thread and I decided to play around with it to figure out what exactly was being talked about. And yeah, while the noise reduction is the very definition of a joke when it comes to that whole uncensoring thing the ease at which it could upscale the converted pictures without a loss of quality is actually quite impressive. I don't think it should be used by the people uploading on the site as a quick way to get extra GP, but I'm also kind of impressed by some of the results. Keep in mind that is in general, as there are cases where the results can only be classified as bad, or they loose some of the finer qualities depending on which one you start with.

In the end, I think if the original artist uses it, then there really isn't an issue, or even just using it as a way to convert from image formats without upscaling isn't an issue, but the upscaling is the problem.

Well if the original artist used it and is releasing it as that, then obviously that's a moot problem. An artist whose source is 1080p but then uses Waifu2x to upscale it to 4k and then releases that 4k version, obviously the 4k one is the "highest one available" and so would be fine.

Where it's drawing a line for me is cases like this - the artist's final released highest-rez images are 1600x1200, but people can use Waifu2x to make them much bigger, put them up here in that artifically bigger rez, and reap the GP rewards, not to mention it consumes much more server space on the site (even if that's Tenboro's problem, it's still not cheap), and brings up the issue of potentially making the legitimately highest-quality versions of it being expunged for being "lower quality" when they really weren't unless you go by pure resolution numbers.

That said, yeah, Waifu2x isn't going to automate decensors. It's great at removing JPG or scanning artifacts, however.

This post has been edited by Dark Pulse: Dec 8 2016, 05:12