landing page: proxying and handling Github stars fetch requests

[iAdramelk]

We have stars counter for the dvc repo on the title page. And it is not always works:

https://sentry.io/organizations/iterative/issues/1510361492/

Also the it not works we didn't handle rejections in GithubLine:

https://sentry.io/organizations/iterative/issues/1519509024/

Maybe we should start proxying and caching GR request on our and as we do now for community page. And also we should add catch to request.

[shcheklein]

@iAdramelk good catch, probably we should do the same as with the community page indeed, and cache results.

[jorgeorpinel]

I can't open the sentry.io URLs but OK I get the idea. In fact I see the GH API request duplicated?

Maybe we should start proxying and caching GR request on our and as we do now for community page. And also we should add catch to request.

I just want to repeat my Q from https://github.com/iterative/blog/pull/122#issuecomment-589873691 here:

if the API is basically a proxy to 3rd party APIs and only used for the community page and maybe blog posts, why do we need to do HTTP requests within the same API? Why not just import a module that does the fetching/processing of the 3rd party APIs?

Ivan's answer was:

mostly due to rate limits and response times, meaning we want to cache results aggressively. mostly means we need a server with some server-side caching mechanism

But what about a simple in-app memory cache instead?

Thanks

[shcheklein]

But what about a simple in-app memory cache instead?

that's exactly what we do now :)

[iAdramelk]

@jorgeorpinel

But what about a simple in-app memory cache instead?

Do you mean on the client side or in the browser?

[jorgeorpinel]

OK yes, but I mean without the need for an internal API that proxies them to the same web app: just import some module that fetches the data and caches it (server side).

[rogermparent]

I can fix this by grabbing data from GitHub at build time. The slight inaccuracy on old builds is practically a non-issue for something like stars, and it requires no runtime requests or logic.

I actually have a branch moving most everything to this setup, but I think I can backport the bit that fixes this issue specifically.

[shcheklein]

@rogermparent I would say, let's do this the same way as we do for other APIs. It's quite important for this counter to be up to date as much as possible.

Also, if GH down we'll have to rely on some default value or freeze the deployment.

[rogermparent]

I think I'm pretty much done, but we'll see if I missed anything.

My solution was to implement a Gatsby build-time solution primarily, but then update it via a proxy server API like the other ones. This way we get the advantages of a solid baked-in value as well as the ability to keep the count updated to-the-minute regardless of when the last rebuild was.