GPU/ CPU for public demo

[Rub21]

From: https://github.com/developmentseed/labs/issues/327#issuecomment-1629049057

Get the GPU work to a point where this can be made public. CPU always-on for small AOIs for demos. On-demand GPU for large AOI.

Sounds great to have a demo version for ds-anotale, but GPU could be expensive, and setting up the CPU for for encoding maybe slow.

@geohacker could you explain a bit more about the feature for ds-annotate. ?

cc. @rbavery

[rbavery]

@ingalls has set up auth now so that access to enable the GPU is controlled. It's planned to implement a simple user management interface so that an admin can grant accounts that enable someone to spin up the GPU service.

So I think there needs to be a way in the DS-Annotate UI to enable the user to specify a compute backend, either the CPU for encoding or the gpu for encoding, but always use CPU for decoding. I think right now it is hard coded to use the GPU for encoding.

An alternative is to just use cached embeddings on s3 for demos and stick with always using the GPU for encoding. Cahced embeddings is already enabled for a variety of areas Ruben has already set up in DS-Annotate. I think this could make sense since it takes over a minute to generate embeddings on the cpu and this is tedious for demos. thoughts @geohacker ?

[geohacker]

My apologies for not getting back here sooner @Rub21 @rbavery. I agree with @rbavery — let's implement using cached embeddings from S3 for demos and by default use GPU (which is only available to select users).

I do like the idea of using CPU by default but if that seems time intensive, let's not prioritise.

On PEARL, there's a little toggle on the top that allows users with GPU access to switch: