*IMPORTANT*: APK Mirror hammering

[archon810]

Hey,

I'm the admin of APK Mirror, and your app is hammering our servers pretty badly, especially in the last day, probably because of press coverage https://www.ghacks.net/2017/01/02/apkupdater-keeps-your-sideloaded-android-apps-up-to-date/.

Can we talk about it ASAP (Hangouts? https://plus.google.com/+ArtemRussakovskii) so we don't have to start banning its users entirely? There are at the very least several optimizations that could have colossal effects on performance. Right now, what the app is doing is very resource intensive and highly inefficient.

Thanks,
Artem

[archon810]

Also your README links to http://androidannotations.org/ instead of http://www.apkmirror.com.

[archon810]

We've spent the last several hours optimizing the search queries to mitigate what is essentially a DDOS attack, but they still could be a LOT faster, which would be better for your users.

I've also checked out the app and was relieved that you're not just leeching our bandwidth but rather offer links to APKs for people to download on the site. That is acceptable behavior. It's actually quite nice.

Let's work together on this? Perhaps switch to search feeds that only query for pnames (something we don't currently support yet). This would result in 10000% speed improvements, plus the amount of data downloaded would be a LOT smaller.

[archon810]

Is anyone going to respond here?

[archon810]

Pretty sad and upsetting lack of responses. So nobody cares, especially after XDA coverage, if we'll start banning users of the app?

[koroshiya]

@archon810 What would be required for this? Just changing the query to something like "?pname=com.whatever.app"?
I'd be happy to implement this in my fork and send a pull request.

[archon810]

Thanks for responding, glad I have your attention now after the XDA PM.

My main dev is on vacation this week. Let me get back to you on the most optimal way. I think we'll be able to implement an RSS feed or a REST API endpoint for super fast pname querying, and possibly even one with batched pnames so you'd only have to issue one request (or maybe paginate to 100 pnames or something). Stay tuned!

[koroshiya]

"glad I have your attention now after the XDA PM"

Just to be clear, I'm not the author of this software. I saw the app on XDA yesterday and decided to create a fork after seeing that development had stalled.
So while I'd be happy to implement whatever fix alleviates the load on your servers, I can't guarantee that @rumboalla will accept any pull request I make after doing so.

[archon810]

Ah, I see, sorry about that @koroshiya. I did get a response out of @rumboalla today, so I'm hoping he, and you, will be able to fix the app up.

The only problem is, from what I can tell, there's no official update channel, so existing users would need to manually update. Perhaps the team can also work on update notifications?

[rumboalla]

Yes querying with pnames (or a list of them) with options to filter by Android version and architecture would be the best option.
If the app is added to apkmirror, the app itself will see the update and notify the user...probably the fastest way to get everyone updated.

[archon810]

@rumboalla That's actually a great point. Ha! A recursive solution of sorts.

We'll have to think about Android version and architecture filtering, so it definitely won't be done at release. We have plans to work on it at some point and roll it out as an advanced search option on the site. Once that's in place, it should be simple to port to RSS or REST API.

[archon810]

In the meantime, could you please set the user agent to something like APKUpdater-vN.N.N so that we can start recognizing which requests belong to APKUpdater, and to which version? Right now it just says Dalvik. This is really important.

Thank you.

[rumboalla]

User agent was added.
Closing this, let me know when the API changes.

[archon810]

Will do. We are working on the single pname API, it's almost ready. Then the bulk API.