GPT integration questions

[williamstein]

can you tell me which version of ChatGPT CoCalc is using?

Currently the model you get on cocalc.com is GPT-3.5-turbo.

Is it free?

It's not free for us -- we pay OpenAI for it, but we do not charge users for it and it is
available to all users of cocalc.com and in our on prem offerings: cocalc-docker and cocalc-cloud.

There's are some mild constraints to throttle usage to avoid the potential of a multi-thousand-dollar bill due to abuse or a bug.
You can read the source code that implements these limits right now here.

Likely very soon, there will be an option for users to use GPT-4, but they will have to enter a credit card, and will get billed once per month based on usage, since GPT-4 costs us 22x as much as GPT-3.5-turbo. That's not implemented yet, but we're working on it.
Our longterm plan is that GPT-3.5-turbo is always free, but more expensive models aren't. Note that in our data, GPT-3.5 is about 3x-4x faster than GPT-4.

CoCalc itself does have code to support GPT4 but it's only enabled right now for admins, e.g., right now if I type a chat and put @chatgpt4 in it, then GPT-4 will reply. This is useful for admins on cocalc-docker.

I also plan to implement support for Google Bard via their API, and other LLM providers, as things get more stable. It seem plausible maybe sometime in 2023 there will also be an open source self-hostable LLM that doesn't completely suck, and I'll make it easy to use in cocalc-docker as well, hopefully.

Will it be available in Docker-CoCalc as well?

The openai integration is already fully available in recent versions of cocalc-docker.
Note that I switched the database from postgres 10 to 14 with no automated upgrade path (YET),
so if you have an existing site you need to upgrade, you can't just pull the sagemathinc/cocalc-v2
image. Instead stick with sagemathinc/cocalc, which I also updated recently.

To use the integration in a recent cocalc-docker, go to site settings as an admin, and search
for openai (see attached image) and enter your api key (and also enable the UI). Then your
entire cocalc instance will fully support chatgpt integration: