Implement ClickHouse Support #3342
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| --- | ||
| sidebar_class_name: node-only | ||
| --- | ||
|
|
||
| import CodeBlock from "@theme/CodeBlock"; | ||
|
|
||
| # ClickHouse | ||
|
|
||
| :::tip Compatibility | ||
| Only available on Node.js. | ||
| ::: | ||
|
|
||
| [ClickHouse](https://clickhouse.com/) is a robust and open-source columnar database that is used for handling analytical queries and efficient storage, ClickHouse is designed to provide a powerful combination of vector search and analytics. | ||
|
|
||
| ## Setup | ||
|
|
||
| 1. Launch a ClickHouse cluster. Refer to the [ClickHouse Installation Guide](https://clickhouse.com/docs/en/getting-started/install/) for details. | ||
| 2. After launching a ClickHouse cluster, retrieve the `Connection Details` from the cluster's `Actions` menu. You will need the host, port, username, and password. | ||
| 3. Install the required Node.js peer dependency for ClickHouse in your workspace. | ||
|
|
||
| ```bash npm2yarn | ||
| npm install -S @clickhouse/client | ||
| ``` | ||
|
|
||
| ## Index and Query Docs | ||
|
|
||
| import InsertExample from "@examples/indexes/vector_stores/clickhouse_fromTexts.ts"; | ||
|
|
||
| <CodeBlock language="typescript">{InsertExample}</CodeBlock> | ||
|
|
||
| ## Query Docs From an Existing Collection | ||
|
|
||
| import SearchExample from "@examples/indexes/vector_stores/clickhouse_search.ts"; | ||
|
|
||
| <CodeBlock language="typescript">{SearchExample}</CodeBlock> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| import { ClickHouseStore } from "langchain/vectorstores/clickhouse"; | ||
| import { OpenAIEmbeddings } from "langchain/embeddings/openai"; | ||
|
|
||
| // Initialize ClickHouse store from texts | ||
| const vectorStore = await ClickHouseStore.fromTexts( | ||
| ["Hello world", "Bye bye", "hello nice world"], | ||
| [ | ||
| { id: 2, name: "2" }, | ||
| { id: 1, name: "1" }, | ||
| { id: 3, name: "3" }, | ||
| ], | ||
| new OpenAIEmbeddings(), | ||
| { | ||
| host: process.env.CLICKHOUSE_HOST || "localhost", | ||
| port: process.env.CLICKHOUSE_PORT || 8443, | ||
| username: process.env.CLICKHOUSE_USER || "username", | ||
| password: process.env.CLICKHOUSE_PASSWORD || "password", | ||
| database: process.env.CLICKHOUSE_DATABASE || "default", | ||
| table: process.env.CLICKHOUSE_TABLE || "vector_table", | ||
| } | ||
| ); | ||
|
|
||
| // Sleep 1 second to ensure that the search occurs after the successful insertion of data. | ||
| await new Promise((resolve) => setTimeout(resolve, 1000)); | ||
|
|
||
| // Perform similarity search without filtering | ||
| const results = await vectorStore.similaritySearch("hello world", 1); | ||
| console.log(results); | ||
|
|
||
| // Perform similarity search with filtering | ||
| const filteredResults = await vectorStore.similaritySearch("hello world", 1, { | ||
| whereStr: "metadata.name = '1'", | ||
| }); | ||
| console.log(filteredResults); | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| import { ClickHouseStore } from "langchain/vectorstores/clickhouse"; | ||
|
There was a problem hiding this comment. This comment is flagging the addition of code that accesses environment variables via |
||
| import { OpenAIEmbeddings } from "langchain/embeddings/openai"; | ||
|
|
||
| // Initialize ClickHouse store | ||
| const vectorStore = await ClickHouseStore.fromExistingIndex( | ||
| new OpenAIEmbeddings(), | ||
| { | ||
| host: process.env.CLICKHOUSE_HOST || "localhost", | ||
| port: process.env.CLICKHOUSE_PORT || 8443, | ||
| username: process.env.CLICKHOUSE_USER || "username", | ||
| password: process.env.CLICKHOUSE_PASSWORD || "password", | ||
| database: process.env.CLICKHOUSE_DATABASE || "default", | ||
| table: process.env.CLICKHOUSE_TABLE || "vector_table", | ||
| } | ||
| ); | ||
|
|
||
| // Sleep 1 second to ensure that the search occurs after the successful insertion of data. | ||
| await new Promise((resolve) => setTimeout(resolve, 1000)); | ||
|
|
||
| // Perform similarity search without filtering | ||
| const results = await vectorStore.similaritySearch("hello world", 1); | ||
| console.log(results); | ||
|
|
||
| // Perform similarity search with filtering | ||
| const filteredResults = await vectorStore.similaritySearch("hello world", 1, { | ||
| whereStr: "metadata.name = '1'", | ||
| }); | ||
| console.log(filteredResults); | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is flagging the addition of code that explicitly accesses environment variables via
process.envin the diff. Maintainers should review this change to ensure proper handling of environment variables.