[X] My Assistant

I've been making torrents for a while for content I believe deserves archival and I thought I'd share some of my process.

I own a Linux machine of older hardware which I run a lot of things on, but the two which are relevant here are Hentai@Home and a couple [en.wikipedia.org] rtorrent instances. If I see a gallery I think deserves archival, I'll do an archive download at original resolution with the Hentai@Home downloader. Once it's finished, I transfer it over and compress it using [www.7-zip.org] 7-Zip.
[i.imgur.com] 7-Zip Settings On a 5800x3D, most galleries compress within 5-10 minutes at most. Using LZMA, and thus, being limited to two threads, certainly slows things down. I do it since it allegedly has slightly better compression ratios. Unfortunately, I'm limited to 32GB of RAM on this motherboard. If and when I get better hardware, I'll adjust it to use more RAM.

Once that's done, I'll upload the archive to my Linux machine and make a torrent using [github.com] dotTorrent-GUI.

[i.imgur.com] dotTorrent-GUI settings I always use 32MiB piece sizes to break compatibility with ancient closed-source clients people still use for whatever reason. After this, I pop it into the torrent client, enable [en.wikipedia.org] super seeding and away it goes.

My reasoning for doing this is to provide the best release possible, and part of that is the smallest release possible. I can often mog people by uploading smaller archives of the same content. Since some of this is specialized knowledge, I thought I may as well share my process. What can be uploaded as a torrent is very much non-standardized and irregular, as well as barely being incentivized by site mechanics. That might be worth considering changing in the future, modeled on certain other P2P and scene standards and communities.

(There is, of course, the downside that decompressing takes a lot more memory, but it's still insignificant on reasonably modern hardware.)

https://e-hentai.org/g/3257320/67ab4c6bf3/
Statistics on this gallery in particular:

Uncompressed size: 1.77 GiB 100%
Zip Archive Size: 1.68 GiB 94.9%
7-Zip Archive Size: 762.1 MiB 52.92%

Compression ratios obviously depend on content and format. Ancient formats like JPG and GIF tend to be inefficient and thus compress better.

Hi,

What does 'mog' mean please?

Thank you.

Hi,

What does 'mog' mean please?

Thank you.

[www.urbandictionary.com] https://www.urbandictionary.com/define.php?term=mog

mog
a term popularized by modern day aesthetic bodybuilders meaning out sizing or dwarfing somebody in muscle size, fullness, and definition
Watch me man I’m about to fucking mog these rockets over there!
Jesus Christ that guy is about to mog them!

What I meant by it was to sort of brag about how compressed my archives are. So, by showing that off, and showing how other people have larger archives (which is negative in this context), I sort of look better by comparison.

Thank you for explaining that!

Hello

Thank You for invitation to thread, sorry that late. Better late than never.

My EHentai seeding begun in ~2023 from raspberry pi. Through my path and torrents scale I had to
Upgrade both my compute, storage and network equipment.

I had to learn a lot of network engineering to accomplish what I wanted.

In 2024 I ran torrenting on FreeBSD host on minipc with 2.5" hdd. Now I use Proxmox for virtualization,
ZFS for torrent data and two rtorrent instances. I also switched to Linux.

For torrent creation I used mktorrent, but now I use mkbrr as it slightly faster when hashing large zip archives

My workflow is pretty manual - I review each gallery I download through hath participation, while 'gooning', I launch my script
to grap .zip from hath and create torrent file, which I upload to gallery torrents. This workflow is perfect balance between automatization
and manual labor.

Regarding torrenting, in past I used Transmission in daemon mode, but I noticed that it no longer can keep up with 1000 torrents,
so I made massive migration to rtorrent.
My first rtorrent instance has 1100 torrents and remote rtorrent instance has another 900.

Now I have issue with 2.5" drive not keeping up with iops at 250Mbps torrent seeding demand, with ~100 peers active.
So I plan to upgrade to enterprise server with HBA SAS card / workstation tower server.
My primary 3.5" SATA hard drive has no issue with iops though.

I use ZFS as it has powerful ARC cache, yes it demands RAM, but helps with growing demands.

rtorrent 0.15.1 + flood.js frontend

I'm limited to 32GB of RAM on this motherboard. If and when I get better hardware, I'll adjust it to use more RAM.

Why do you think more ram will help your compression ratio when 2x the source already fits in ram ? Does linux not have virtual memory?

Uncompressed size: 1.77 GiB 100%
Zip Archive Size: 1.68 GiB 94.9%
7-Zip Archive Size: 762.1 MiB 52.92%

The important detail omitted here is 7-zip archives require complete unpacking in order to be usable. So effective size required for consumer is 1.770 + 0.762 ~= 2.5 GiB, which is 48% bigger than 1.68 GiB of zip. 7-zip is a great format for cold storage, but not so much for sharing through torrents.

Why do you think more ram will help your compression ratio when 2x the source already fits in ram ? Does linux not have virtual memory?

To my understanding, more RAM means more space to search for patterns for compression. I'm not an expert on this, though. Linux of course has swapspace, but I'm compressing on Windows since my Windows machine has a better CPU.

The important detail omitted here is 7-zip archives require complete unpacking in order to be usable. So effective size required for consumer is 1.770 + 0.762 ~= 2.5 GiB, which is 48% bigger than 1.68 GiB of zip. 7-zip is a great format for cold storage, but not so much for sharing through torrents.

If someone's grabbing a torrent of an art gallery, my assumption is that they're going to want the whole thing and not just a couple images. Otherwise, they could just grab a couple images and be done with it. This doesn't have to do with .7z vs .zip, either, it's more a matter of block size. You can configure that with .7z (which I set to Solid for maximum compression), whereas it's not configurable at all with .zip. Plus, why do you count the size of the .7z archive but not the size of the .zip archive in your weird calculation? Does it have to do with decompression speed? If that's the case, and that's a concern, I recommend a better CPU before anything else. (And look, you're a catgirl, so is that really a concern for you?)

I'd also add that I have a separate Linux server acting as a seedbox, while I have whatever I want locally on my Windows machine. If someone's getting really into torrenting, I recommend a similar setup.

You are, of course, free to take the .7zip archives I make, decompress them, compress them however you wish and upload those as a torrent. I'm not stopping you.

This post has been edited by peacethroughpower: Jun 30 2025, 11:53

To my understanding, more RAM means more space to search for patterns for compression. I'm not an expert on this, though. Linux of course has swapspace, but I'm compressing on Windows since my Windows machine has a better CPU.

You're only going to run into RAM limits with huge compression libraries that are only effective when you're compressing massive datasets like several hundred gigabyte or more. Maybe try LZMA2 instead which has multithreaded support where LZMA is dual thread I think at max.

Even then, LZMA2 shares the dictionary where LZMA has a dictionary for each thread.

I hope this helps.

When you are dealing with images, as is the case here, instead of wasting your time trying to figure out the best settings to use with generic archive compressors, you guys should focus on specialized codecs, like jxl and webp. Both can do lossless, both will achieve better results than whatever generic app you are using and, when done, you can simply zip (or whatever) all the images together, using no archive compression at all.

PS: vote jxl

PS: pay me in pizza.

When you are dealing with images, as is the case here, instead of wasting your time trying to figure out the best settings to use with generic archive compressors, you guys should focus on specialized codecs, like jxl and webp. Both can do lossless, both will achieve better results than whatever generic app you are using and, when done, you can simply zip (or whatever) all the images together, using no archive compression at all.

PS: vote jxl

PS: pay me in pizza.

That would be something to talk to Tenboro about, not me. I've messed around with lossless image recompression but haven't had much luck creating anything I'd consider worth sharing. I don't claim to have enough knowledge for how to do that properly without potentially fucking something up for whomever torrents it.

I don't claim to have enough knowledge for how to do that properly without potentially fucking something up for whomever torrents it.

Don't worry about that. Think of JXL just as another image format. You know JPG exists, right? You know there is also something called GIF and that it is not the same as JPG. I'd be willing to bet you also know there is another format called PNG. And you know you can use a browser or image viewer app to see any of those images.

JXL and WEBP are just that, different image formats. Only newer. And better.

Don't worry about that. Think of JXL just as another image format. You know JPG exists, right? You know there is also something called GIF and that it is not the same as JPG. I'd be willing to bet you also know there is another format called PNG. And you know you can use a browser or image viewer app to see any of those images.

JXL and WEBP are just that, different image formats. Only newer. And better.

Yes, I'm aware of what different image formats are. That doesn't really address any reasons I brought up for not creating torrents using them. Again, if you want e-hentai to use newer, better image formats, that's something to talk to Tenboro about. Not me.

Respect, that’s impressive! But nowadays many people just use cloud drives to share files. The problem is you have to download them first before making a new torrent, which is really troublesome