[kemonoparty] fix kemono api skipping latest posts #6931
Conversation
|
I ran some tests and discovered that while this does fix the missing posts at the beginning, it "moves" the problem to the next batch of posts. For example, when kemono skips the first 3 posts of a creator with
meaning instead of posts 1-3, it is now missing posts 51-53. I think the easiest solution would be to do 2 API requests for the first |
I have implemented a check for duplicate posts (or same revisions if they are available) This should also fix the problem where sometimes a same post without multiple revisions would get extracted twice (caused by the "moved" batch issue as well) I have reverted the previous changes.
|
I actually did some testing myself when I first made the commit and the behavior I've got was:
So I just assumed fetching for |
|
It seems like the API is just horribly broken. I used Here are the first few returned post IDs
|
|
Maybe we need to start using the |
From what I can see, it seems to have the same problems as the endpoint we currently use. |
Making requests with either the offset (
o) or query (q) parameters set to a blank string "?o=" for some reason fixes the problem where kemono API would skip posts that were just imported (see #6780).