[X] My Assistant

I'm not actually reading this thread anymore due to the heavy brigadeering. I get that it broke your favorite viewing tool, but all the reasons for why "bring back page number" isn't long-term technically feasible was already answered in the first few pages of the thread.

There are some bugs being fixed (missing galleries in favorites, some uploader search stuff and some navigation stuff), as well as a function to address the (ostensible) biggest gripe about not being able to jump to the middle of search results, but these things aren't written, tested and deployed overnight. I will however hold off on deploying the update to EH until it's ready.

The website is unusable now. We don't want to see first or last or scroll through every single page. Why fix something that is not broken and that no one asked for?
please listen to us!

I'm not actually reading this thread anymore due to the heavy brigadeering. I get that it broke your favorite viewing tool, but all the reasons for why "bring back page number" isn't long-term technically feasible was already answered in the first few pages of the thread.

There are some bugs being fixed (missing galleries in favorites, some uploader search stuff and some navigation stuff), as well as a function to address the (ostensible) biggest gripe about not being able to jump to the middle of search results, but these things aren't written, tested and deployed overnight. I will however hold off on deploying the update to EH until it's ready.

Now this is the type of developer ego self sabotage shit I've only seen from game developers up until now.

It seems like they are done managing this site and are trying to kill it so they dont have to anymore because outright ignoring peoples valid complaints is what will kill this site and they have that strategy down pat.

@Tenboro: I'm pretty sure no one is brigading this thread, and more like genuine complain about the removal of pages, If you go by pure numbers of submissions, Pixiv does come to mind and they don't remove the page numbers in search. Most people already read the few starting pages, but they complains because no archives or index site out there remove the pages counts. It simply doesn't work with the way we process information and searching, earlier page it said that it's impossible to keep it because of index size, but by pure numbers, pixiv doesn't lose out if you only count the sheer numbers of submissions. and Nyaa also kept it's page counts despite the crazy amount of upload and dups. Honest advice from me, rollbacks the update and kept a beta mode separate and ask people to test it rather than just dump it all on suddenly like this, most people are kinda force to go EH if you can't use ex with this update, right now, there is now way to keep track of where you are in search and you are required to hop on and off ID and playing guess games if you want to search for something, especially if your memory are hazy and you are trying to look for something long ago. There is also whole another with search engine and expunged gallery as well but others already putting it more eloquent than i am ever could about that.

I know it's being brigadeered, but some amount of the complaints are of course organic. However, since page 5 or so it's just people spamming about page numbers, and I can't keep replying to that.

I get that people feel strongly about it, but as a tradeoff between functionality and performance, this is pretty much the best I can do. The old search engine isn't actually working anymore despite people claiming otherwise, and the alternative was a severe reduction in functionality.

You mention Nyaa, which has page numbers, but you can only see 1000 results per search, is that really what people want?

And I'm not sure why people act like they can't use the dates of the posted galleries to "know where you are in a search".

(I'm actually busy with making some additions to the search engine, so I won't keep posting.)

I may sound like an idiot, but I don't understand the GID search, doesn't removing the paging effectively kill what's between the first and last pages?

It means that the list paging will be based on gid as a "cursor" rather than a paginated n results per page one. The benefit is that if you performed a search/filter, the list that you generate will be sure to not be cached in a way that certain entries are skipped or doubled in successive paging (i.e. one gallery appears at the end of page 1 results and at the start of page 2 results). This cursor pagination scheme is beneficial for database tables that get updated every so often, making sure that the current filter you queried has a complete set of results properly paged. That and the lower resource cost of querying through pages has the tradeoff of not being able to jump at a specific page of query list of the older "limit/offset" custom querying.

I'm personally for this scheme. There's not much sense to jump to around page 90 of a search query right off the bat. If you want to track where you are in your custom query, this gid cursor search does a better job of being sure you're on the right result. It will be kinda inconvenient to have to use the browser bookmark for tracking it so it will be nice for example to be able to save the "gid cursor" in tag filters for convenience's sake.

This post has been edited by sakuracircle: Nov 3 2022, 14:08

I'm not actually reading this thread anymore due to the heavy brigadeering. I get that it broke your favorite viewing tool, but all the reasons for why "bring back page number" isn't long-term technically feasible was already answered in the first few pages of the thread.

I've never used any viewing tools, can't understand why you are so persistent with that, since it seems like most people complaining just used pages normally. At the very least add the numbering back to favorites, since browsing them the way it is now it's an absolute chore.

I can think of several ways to do effective pagination.

As an example, have GID-SORTED (so, numerical sorted and chronological) databases (or sqlite) or whatever for every tag (possible due to tag whitelisting) with just the GID in it.

When a search is executed, start a poinrter on the newest gallery(GID) in each DB that the tag search was.

Ex. user searches "yuri kimono kissing" (minus quotes, I'm going to ignore wildcarding, as that's a solved issue)

(Assuming the most recent GIDs)
initialized to most recently added:
yuri DB at GID 2367160
kimono DB at GID 2367160
kissing DB at GID 2367081

pick the smallest DB with the least GIDs (kimono DB in this case).

pick the 2nd smallest DB and see if has a matching GID (recall all DBs are numerically GID sorted, so this is very very very fast).

If no, fail-fast, advance to the next GID in kimono DB, and repeat the search in kissing DB, but limit off the area you already searched and know to be before the first search.

If yes, proceed to the yuri DB and do the same thing. Repeat for every tag in a search. As you have more tags, the amount of galleries matching the smallest DB(least common tags) will fall off dramatically, so more tags doesn't necessarily mean more load on the search.

This would be VERY fast, as it only hits the smallest databases first, so you can give an accurate total number of search results, which allows GID based pagination/indexing at least client side.

For nasty searches like just "big breasts", just cache the results and either update it every few minutes, or have it update itself whenever that tag is added to a gallery.

TL;DR: paginate searches using the smallest DBs first, and if any search is used enough to cause performance issues, just add it (or the tag combination) to be cached. I'd suggest automating it, but I know that's just adding a ton more work on someone's plate.

Fair warning, it's really late and I'm somewhat drunk and wrote this up in 3 minutes.

EDIT:

Some back of the napkin math shows 3 bytes to an int to 15million, let's just say 4bytes for future proofing, which would be 4 billion or something.

If you had a DB with JUST a GID for EVERY gallery, it would be currently 700000*4bytes, so 2.8MB.

This is not going to happen so most of these hypothetical single tag GID DBs would be in the KB range.

If you're concerned about load, you can hand these to the clients and they can do the pagination processing.

Ex. Someone searches "yuri kimono kissing", local javascript pulls those 3 DBs (62,488 yuri results so ~300KB, 11,000 kimono results so ~50KB, 13,000 kissing results, so ~60KB, so ~410KB total), and gets to work generating local pagination for them for as long as they're in that specific search.

Double/Triple the sizes for overhead, and you're still in margin of error for bandwidth use for a single image.

This is amazing, do all drunkards end up reinventing the Venti(8) archival database filesystem I wonder? Assuming the tag->gid block headers are in-order append logs that's O(n*log(n)) to actually do the search, as OR would require sorting a concatenated list beforehand.

I'm not actually reading this thread anymore due to the heavy brigadeering. I get that it broke your favorite viewing tool, but all the reasons for why "bring back page number" isn't long-term technically feasible was already answered in the first few pages of the thread.

There are some bugs being fixed (missing galleries in favorites, some uploader search stuff and some navigation stuff), as well as a function to address the (ostensible) biggest gripe about not being able to jump to the middle of search results, but these things aren't written, tested and deployed overnight. I will however hold off on deploying the update to EH until it's ready.

it's not ostensible, it's how I and probably a lot of other people browse. an easy fix to this is to let us input the number of 'next' or 'prev' we want. (and maybe tell us how many 'next' it needs to get to the end)

So much effort for such a mediocre result... It's heartbreaking. A return to the old search engine is inevitable.

as a tradeoff between functionality and performance

Unless it's running at dialup speeds, the vast majority of users do not care about performance. When will developers understand this?

it's not ostensible, it's how I and probably a lot of other people browse. an easy fix to this is to let us input the number of 'next' or 'prev' we want. (and maybe tell us how many 'next' it needs to get to the end)

That actually kills the very purpose of the cursor pagination, because it can't do that in itself. It jumps based on the specific gid for previous/next buttons so it doesn't inherently know what gid is needed for the next/previous n pages.



@topic

I guess people just got too used to the old one that they refuse to adapt to this, or have custom apps/scripts that broke after the update which understandably can be frustrating. The main criticism I can think of is that the implementation came so quick that people weren't ready for the sudden change.

I'm actually more interested on getting back the 200 results per page option. This seems to be gone in the new schema.

This post has been edited by sakuracircle: Nov 3 2022, 14:33

I know it's being brigadeered, but some amount of the complaints are of course organic. However, since page 5 or so it's just people spamming about page numbers, and I can't keep replying to that.

I get that people feel strongly about it, but as a tradeoff between functionality and performance, this is pretty much the best I can do. The old search engine isn't actually working anymore despite people claiming otherwise, and the alternative was a severe reduction in functionality.

You mention Nyaa, which has page numbers, but you can only see the 1000 results per search, is that really what people want?

And I'm not sure why people act like they can't use the dates of the posted galleries to "know where you are in a search".

(I'm actually busy with making some additions to the search engine, so I won't keep posting.)

Like I said, made a beta site and ask people to test it before roll it up, even if it not make everyone happy, at least people can make feedback and prepare what need to do. Honestly most comments is about the pages only is because people aren't tech savvy enough to speak in detail, and that's by no mean make their comments invalid or brigading as they are end user, not mentioned many of them are lurker from like 10ish years ago.

I thinks you are making a slight misunderstanding, from user-end pov, most people don't really look at the date, and it's not their fault, Ex stuff are post individually from user, some stuff got post way later when the user submitted them and the time when they actually come out, and people memory is not a good indicator of anything, pages counts is good simply because user can visually see where they are, it's the very same reason why pretty much all index size like nyaa, or even everything torrents like TPB or rabg keep it's page number counts, simply because as an index, it's also a form of category so user can see where they are, a performances increase doesn't worth the trade of everything else.

Thinks of library, we can't see the shelves, we can't see the floor, the pages counts itself is essentially a guidepost for navigation, it's the very same reason why no index sites out there choose infinite scrolling to listing their index. It simply doesn't work with the way we process information.

The date is also super unreliable though, as i mentioned before, submitted date and release date are wildly different, this is even worse with pixiv and twitter contents, for those trying to browse something news and cannot check in every days, losing page counts is essentially turning it into a guessing game. People are actually doesn't mind nearly as much when it come to performance from what i have read, if anything, most people are actually prefer to get the paging counts back even with the cost of the performance.

This post has been edited by dragon1412: Nov 3 2022, 14:26

And I'm not sure why people act like they can't use the dates of the posted galleries to "know where you are in a search".

My favorite now is 920k+ (too many? yeah, almost 15 years of being here made all that), now tell me how can I search for that one specific gallery I know it's in my list, but only remember it's after that one and before those one and some vague tags that result in 10k+ galleries and the one I need is among them?

Click next 1000 times? 2000 times? 5000 times?

And please don't even talking about dates of posted or favorites here, even the tag is vaguely remembered, you expected me to remember 920k+ date and times?

So please at least bring back page number even it means simply just count the GID of the page and divided by the max result per page setting.

This post has been edited by epa: Nov 3 2022, 14:23

btw is it still in progress or already be done with the update ?? since everytime iam using advance search with expunge tag or low power tag the result somewhat is disordered

You mention Nyaa, which has page numbers, but you can only see 1000 results per search, is that really what people want?

It's at least a compromise which gives us page numbers. That and searches don't usually reach that much in the first place from what I've seen. Maybe 10,000 would be a good limit.

This post has been edited by jetpack003: Nov 3 2022, 14:20

Brigadeering? I've been a member of this site for fifteen years and came to this thread of my own volition to state my disdain for the usability of the gallery going down the drain overnight. I have never once used any external tools when browsing, and as a matter of fact the old search did work mere hours before the changes were implemented. Given the option between waiting several seconds to even a minute for pages to load and what you've implemented now, I'd take the prior in a heartbeat.

The fact that so many have come here with decade old accounts to express our dissatisfaction should be nothing to gloss over with such a blanket statement that complaints are all alike and not worth reading. If anything, that makes it seem that the biggest problem with this site, beyond any kind of functionality, is that those who run it are too egotistical to acknowledge any form criticism, no matter how unanimous or constructive.

Have some self-awareness and look at what's happening here for God's sake. Please try your best to both come up with a better solution and be better about how you're handling this.

>reducing all criticism and complaints to Brigadeering
I assume this means nothing is going to change, and no meaningful functionality/usability is going to be added. Am I correct in this assessment?

Well, we hope @OsenTen will come up with amazing script to bring back page then. Script he's doing now is super good. If he can make page number and push it as extension it'd be our lord savior.

No pressure dude, but unfortunately I'm more of back-end guy, can't do much about this front-end.

This post has been edited by epa: Nov 3 2022, 14:32

On the other hand the new system in fact open a door to third party search provider that can be seamlessly integrated with main site.

All it need is just leech all the tags & index and put them in a search system that a normal person would use.

And it only need to return 1 GID then you can use that to acquire the html from main site.

Then the things left is find a way to leech money from the site to make it self-sustaining.

This post has been edited by 117649: Nov 3 2022, 14:37

I know it's being brigadeered, but some amount of the complaints are of course organic. However, since page 5 or so it's just people spamming about page numbers, and I can't keep replying to that.

I get that people feel strongly about it, but as a tradeoff between functionality and performance, this is pretty much the best I can do. The old search engine isn't actually working anymore despite people claiming otherwise, and the alternative was a severe reduction in functionality.

You mention Nyaa, which has page numbers, but you can only see 1000 results per search, is that really what people want?

And I'm not sure why people act like they can't use the dates of the posted galleries to "know where you are in a search".

(I'm actually busy with making some additions to the search engine, so I won't keep posting.)

Can fix Favorites to show the gallery that has been remove.

It is very useful to do the expuge the Duplicate or Forbidden gallery