GenSpectrum Chat: Share your experiences and thoughts!

[chaoran-chen]

How are your experiences with GenSpectrum Chat? What worked well and what didn't? Any feedback is welcome!

[theosanderson]

Extremely nice!
At one point is started talking about the query it used:

I found all the non-human hosts that have been tested for SARS-CoV-2 in the dataset. The "distinct" keyword means that I only listed each host once, even if there were multiple samples from that host. The "where host != 'Human'" part means that I excluded all the samples that were collected from humans.

Even though it didn't explicitly show the query, which could be confusing.

It would be cool if it had the ability to calculate some proportions.

[chaoran-chen]

I told it not to use any technical SQL terms but it sometimes doesn't listen :( (GPT-4 is better but don't have API access, yet.)

Agree with the proportions, that would indeed be very nice!

[theosanderson]

I was wondering if it could generate SQL. I'm kind of surprised that it doesn't just try to do proportions on its own -- maybe you've been very explicit about what it can and can't do (which probably makes sense in a lot of ways). GPT-4 is awesome, but does get expensive quickly in my experience.

[chaoran-chen]

I asked ChatGPT to generate SQL but our database (LAPIS) is not an actual relational database and only supports a very limited set of SQL (which I wrote up in a day), so the limiting factor here is LAPIS.

[theosanderson]

Ah cool - that makes sense. One could imagine something where it can extract the pieces with LAPIS queries, and then do stuff to them with arbitrary SQL with in-memory sqlite or something. Anyway, super cool.

[chaoran-chen]

That could indeed also be a way to do it!

[AnonymousUserUse]

It does not work at all, except for randomly generated questions. It is much more difficult to let GenSpectrum Chat to understand my thought than to write the query directly in CoV-Spectrum directly by myself.

[chaoran-chen]

Thanks for the feedback! It currently does not support many features including calculating the proportions (except for proportions of mutations within a variant) and growth advantages and does not know about the collections. We have to either add the features or train the bot to inform the user about the unavailability of the features properly.

[AnonymousUserUse]

Ok. It seems that there are still a high number of features to be trained.
Why can bot not answer the question "When was the first XBB submitted?"? Is there specific reason behind that?

[chaoran-chen]

Good question! We'll investigate the error cases from the recorded messages next week.

[AnonymousUserUse]

Which languages does GenSpectrum Chat currently support? Or generally all languages that are supported by ChatGPT?

[chaoran-chen]

In principle, all languages of ChatGPT are supported. But the examples that we gave ChatGPT are in English so the support for other languages might not be very good. We have to do some proper tests!