Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

openai: use ollama in test #994

Open
codefromthecrypt opened this issue Aug 19, 2024 · 5 comments
Open

openai: use ollama in test #994

codefromthecrypt opened this issue Aug 19, 2024 · 5 comments

Comments

@codefromthecrypt
Copy link
Contributor

Ollama has some support for openai emulation and could be a nice way to support more tests without requiring an API key or a public endpoint. This might be a nice way to extend coverage of the openai, which could also run relevant versions of the tests also in the ollama llm package. What do you think?

e.g. Some test configuration could work when OPENAI_API_KEY is not set, but OLLAMA_TEST_MODEL is.

	llm, err := openai.New(
		openai.WithBaseURL("http://localhost:11434/v1"),
		openai.WithModel(ollamaModel),
		openai.WithToken("unused"),
		// openai.WithHTTPClient(httputil.DebugHTTPClient),
	)
@devalexandre
Copy link
Contributor

for test, I think that use testcontainer with ollama, but I need try it.

@codefromthecrypt
Copy link
Contributor Author

yep you can use testcontainers or use a system managed one. If you are using small models the perf hit of running via containers may not be a big deal.

@mdelapenya
Copy link
Contributor

@devalexandre there is an Ollama module for Go: https://golang.testcontainers.org/modules/ollama/ 😉

@codefromthecrypt
Copy link
Contributor Author

fyi if you are doing multiple tests or ones where the completions aren't quick, could be simpler to use system managed. here's an example of that in practice and it works https://github.com/block/goose/blob/49a30b4d22329e10262128d08aab03964f910d9f/.github/workflows/ci.yaml#L52-L100

@mdelapenya
Copy link
Contributor

I'm going to submit a PR for this, will link to this issue once ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants