Skip to content

Commit

Permalink
I messed up and am trying to fix it
Browse files Browse the repository at this point in the history
  • Loading branch information
univish committed Dec 21, 2023
1 parent 046a4a3 commit 27a61c1
Show file tree
Hide file tree
Showing 66 changed files with 1,005 additions and 619 deletions.
2 changes: 1 addition & 1 deletion .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<!--
Thank you for contributing to LangChainJS! Your PR will appear in our next release under the title you set above. Please make sure it highlights your valuable contribution.
Thank you for contributing to LangChain.js! Your PR will appear in our next release under the title you set above. Please make sure it highlights your valuable contribution.
To help streamline the review process, please make sure you read our contribution guidelines:
https://github.com/langchain-ai/langchainjs/blob/main/CONTRIBUTING.md
Expand Down
18 changes: 18 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,24 @@ even patch releases may contain [non-backwards-compatible changes](https://semve
If your contribution has made its way into a release, we will want to give you credit on Twitter (only if you want though)!
If you have a Twitter account you would like us to mention, please let us know in the PR or in another manner.

#### Integration releases

You can invoke the release flow by calling `yarn release` from the package root.

There are three parameters which can be passed to this script, one required and two optional.

- __Required__: `--workspace <workspace name>`. eg: `--workspace @langchain/core` (always appended as the first flag when running `yarn release`)
- __Optional__: `--version <version number>` eg: `--version 1.0.8`. Defaults to adding one to the patch version.
- __Optional__: `--bump-deps` eg `--bump-deps` Will find all packages in the repo which depend on this workspace and checkout a new branch, update the dep version, run yarn install, commit & push to new branch.
- __Optional__: `--tag <tag>` eg `--tag beta` Add a tag to the NPM release.
- __Optional__: `--inc <inc>` eg `--inc patch` The semver increment to apply to the version. Can be one of `major`, `minor`, `patch`, `premajor`, `preminor`, `prepatch`, or `prerelease`. Defaults to `patch`.

This script automatically bumps the package version, creates a new release branch with the changes, pushes the branch to GitHub, uses `release-it` to automatically release to NPM, and more depending on the flags passed.

Halfway through this script, you'll be prompted to enter an NPM OTP (typically from an authenticator app). This value is not stored anywhere and is only used to authenticate the NPM release.

Full example: `yarn release @langchain/core --version 2.0.0 --bump-deps --tag beta --inc major`.

### 🛠️ Tooling

This project uses the following tools, which are worth getting familiar
Expand Down
4 changes: 2 additions & 2 deletions docs/api_refs/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@
"scripts": {
"dev": "next dev -p 3001",
"typedoc": "npx typedoc --options typedoc.json",
"build:scripts": "yarn typedoc && node ./scripts/update-typedoc-css.js",
"build:scripts": "node ./scripts/create-entrypoints.js && yarn typedoc && node ./scripts/update-typedoc-css.js",
"build": "yarn clean && yarn run build:deps && yarn build:scripts && next build",
"build:vercel": "yarn clean && yarn run build:deps --force && yarn build:scripts && next build",
"build:deps": "yarn run turbo:command build --filter=@langchain/core --filter=@langchain/anthropic --filter=@langchain/openai --filter=@langchain/community --filter=langchain --concurrency=1",
"build:deps": "yarn run turbo:command build --filter=!api_refs --filter=!core_docs --filter=!examples --filter=!create-langchain-integration --concurrency=1",
"start": "yarn build && next start -p 3001",
"lint": "next lint",
"clean": "rm -rf .next .turbo public/ && mkdir public"
Expand Down
74 changes: 74 additions & 0 deletions docs/api_refs/scripts/create-entrypoints.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
const { Project, SyntaxKind } = require("ts-morph");
const fs = require("fs");

/**
*
* @param {string} relativePath
* @param {any} updateFunction
*/
const updateJsonFile = (relativePath, updateFunction) => {
const contents = fs.readFileSync(relativePath).toString();
const res = updateFunction(JSON.parse(contents));
fs.writeFileSync(relativePath, JSON.stringify(res, null, 2) + "\n");
};

function main() {
const project = new Project();
const entrypointFiles = [
"../../langchain/scripts/create-entrypoints.js",
"../../langchain-core/scripts/create-entrypoints.js",
"../../libs/langchain-community/scripts/create-entrypoints.js",
"../../libs/langchain-anthropic/scripts/create-entrypoints.js",
"../../libs/langchain-google-genai/scripts/create-entrypoints.js",
"../../libs/langchain-openai/scripts/create-entrypoints.js",
"../../libs/langchain-mistralai/scripts/create-entrypoints.js",
];

const entrypoints = new Set([]);
entrypointFiles.forEach((entrypointFile) => {
// load file contents from ts-morph
const file = project.addSourceFileAtPath(entrypointFile);
// extract the variable named entrypoints
const entrypointVar = file.getVariableDeclarationOrThrow("entrypoints");
// extract the `deprecatedNodeOnly` if it exists
const deprecatedNodeOnlyVar =
file.getVariableDeclaration("deprecatedNodeOnly");
/**
* @type {string[]}
*/
let deprecatedNodeOnly = [];
if (deprecatedNodeOnlyVar) {
const deprecatedNodeOnlyKeys = deprecatedNodeOnlyVar
.getInitializerIfKindOrThrow(SyntaxKind.ArrayLiteralExpression)
.getElements()
.map((element) => element.getText().replaceAll('"', ""));
deprecatedNodeOnly = deprecatedNodeOnlyKeys;
}
// get all keys from the entrypoints object
const entrypointKeys = entrypointVar
.getInitializerIfKindOrThrow(SyntaxKind.ObjectLiteralExpression)
.getProperties()
.map((property) => property.getText());
const entrypointKeysArray = entrypointKeys.map((kv) =>
kv.split(":").map((part) => part.trim().replace(/^"|"$/g, ""))
);

/**
* @type {Record<string, string>}
*/
const entrypointsObject = Object.fromEntries(entrypointKeysArray);
const entrypointDir = entrypointFile.split(
"/scripts/create-entrypoints.js"
)[0];

Object.values(entrypointsObject)
.filter((key) => !deprecatedNodeOnly.includes(key))
.map((key) => entrypoints.add(`${entrypointDir}/src/${key}.ts`));
});

updateJsonFile("./typedoc.json", (json) => ({
...json,
entryPoints: Array.from(entrypoints),
}));
}
main();
207 changes: 203 additions & 4 deletions docs/api_refs/typedoc.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"plugin": [
"./typedoc_plugins/hide_underscore_lc.js"
],
"tsconfig": "../../langchain/tsconfig.json",
"tsconfig": "../../tsconfig.json",
"readme": "none",
"excludePrivate": true,
"excludeInternal": true,
Expand All @@ -20,6 +20,7 @@
"includeVersion": true,
"sourceLinkTemplate": "https://github.com/langchain-ai/langchainjs/blob/{gitRevision}/{path}#L{line}",
"logLevel": "Error",
"name": "LangChain.js",
"entryPoints": [
"../../langchain/src/load/index.ts",
"../../langchain/src/load/serializable.ts",
Expand Down Expand Up @@ -79,7 +80,6 @@
"../../langchain/src/llms/openai.ts",
"../../langchain/src/llms/ai21.ts",
"../../langchain/src/llms/aleph_alpha.ts",
"./langchain/src/llms/azure_ml.ts",
"../../langchain/src/llms/cloudflare_workersai.ts",
"../../langchain/src/llms/cohere.ts",
"../../langchain/src/llms/hf.ts",
Expand Down Expand Up @@ -206,7 +206,6 @@
"../../langchain/src/chat_models/ollama.ts",
"../../langchain/src/chat_models/minimax.ts",
"../../langchain/src/chat_models/llama_cpp.ts",
"./langchain/src/chat_models/azure_ml.ts",
"../../langchain/src/chat_models/yandex.ts",
"../../langchain/src/chat_models/fake.ts",
"../../langchain/src/schema/index.ts",
Expand Down Expand Up @@ -303,6 +302,206 @@
"../../langchain/src/experimental/tools/pyinterpreter.ts",
"../../langchain/src/evaluation/index.ts",
"../../langchain/src/runnables/index.ts",
"../../langchain/src/runnables/remote.ts"
"../../langchain/src/runnables/remote.ts",
"../../langchain-core/src/agents.ts",
"../../langchain-core/src/caches.ts",
"../../langchain-core/src/callbacks/base.ts",
"../../langchain-core/src/callbacks/manager.ts",
"../../langchain-core/src/callbacks/promises.ts",
"../../langchain-core/src/chat_history.ts",
"../../langchain-core/src/documents/index.ts",
"../../langchain-core/src/embeddings.ts",
"../../langchain-core/src/example_selectors/index.ts",
"../../langchain-core/src/language_models/base.ts",
"../../langchain-core/src/language_models/chat_models.ts",
"../../langchain-core/src/language_models/llms.ts",
"../../langchain-core/src/load/index.ts",
"../../langchain-core/src/load/serializable.ts",
"../../langchain-core/src/memory.ts",
"../../langchain-core/src/messages/index.ts",
"../../langchain-core/src/output_parsers/index.ts",
"../../langchain-core/src/outputs.ts",
"../../langchain-core/src/prompts/index.ts",
"../../langchain-core/src/prompt_values.ts",
"../../langchain-core/src/runnables/index.ts",
"../../langchain-core/src/retrievers.ts",
"../../langchain-core/src/stores.ts",
"../../langchain-core/src/tools.ts",
"../../langchain-core/src/tracers/base.ts",
"../../langchain-core/src/tracers/console.ts",
"../../langchain-core/src/tracers/initialize.ts",
"../../langchain-core/src/tracers/log_stream.ts",
"../../langchain-core/src/tracers/run_collector.ts",
"../../langchain-core/src/tracers/tracer_langchain_v1.ts",
"../../langchain-core/src/tracers/tracer_langchain.ts",
"../../langchain-core/src/utils/async_caller.ts",
"../../langchain-core/src/utils/chunk_array.ts",
"../../langchain-core/src/utils/env.ts",
"../../langchain-core/src/utils/hash.ts",
"../../langchain-core/src/utils/json_patch.ts",
"../../langchain-core/src/utils/json_schema.ts",
"../../langchain-core/src/utils/math.ts",
"../../langchain-core/src/utils/stream.ts",
"../../langchain-core/src/utils/testing/index.ts",
"../../langchain-core/src/utils/tiktoken.ts",
"../../langchain-core/src/utils/types.ts",
"../../langchain-core/src/vectorstores.ts",
"../../libs/langchain-community/src/load/index.ts",
"../../libs/langchain-community/src/load/serializable.ts",
"../../libs/langchain-community/src/tools/aiplugin.ts",
"../../libs/langchain-community/src/tools/aws_lambda.ts",
"../../libs/langchain-community/src/tools/aws_sfn.ts",
"../../libs/langchain-community/src/tools/bingserpapi.ts",
"../../libs/langchain-community/src/tools/brave_search.ts",
"../../libs/langchain-community/src/tools/connery.ts",
"../../libs/langchain-community/src/tools/dadjokeapi.ts",
"../../libs/langchain-community/src/tools/discord.ts",
"../../libs/langchain-community/src/tools/dynamic.ts",
"../../libs/langchain-community/src/tools/dataforseo_api_search.ts",
"../../libs/langchain-community/src/tools/gmail/index.ts",
"../../libs/langchain-community/src/tools/google_custom_search.ts",
"../../libs/langchain-community/src/tools/google_places.ts",
"../../libs/langchain-community/src/tools/ifttt.ts",
"../../libs/langchain-community/src/tools/searchapi.ts",
"../../libs/langchain-community/src/tools/searxng_search.ts",
"../../libs/langchain-community/src/tools/serpapi.ts",
"../../libs/langchain-community/src/tools/serper.ts",
"../../libs/langchain-community/src/tools/wikipedia_query_run.ts",
"../../libs/langchain-community/src/tools/wolframalpha.ts",
"../../libs/langchain-community/src/agents/toolkits/aws_sfn.ts",
"../../libs/langchain-community/src/agents/toolkits/base.ts",
"../../libs/langchain-community/src/agents/toolkits/connery/index.ts",
"../../libs/langchain-community/src/embeddings/bedrock.ts",
"../../libs/langchain-community/src/embeddings/cloudflare_workersai.ts",
"../../libs/langchain-community/src/embeddings/cohere.ts",
"../../libs/langchain-community/src/embeddings/googlepalm.ts",
"../../libs/langchain-community/src/embeddings/googlevertexai.ts",
"../../libs/langchain-community/src/embeddings/gradient_ai.ts",
"../../libs/langchain-community/src/embeddings/hf.ts",
"../../libs/langchain-community/src/embeddings/hf_transformers.ts",
"../../libs/langchain-community/src/embeddings/llama_cpp.ts",
"../../libs/langchain-community/src/embeddings/minimax.ts",
"../../libs/langchain-community/src/embeddings/ollama.ts",
"../../libs/langchain-community/src/embeddings/tensorflow.ts",
"../../libs/langchain-community/src/embeddings/togetherai.ts",
"../../libs/langchain-community/src/embeddings/voyage.ts",
"../../libs/langchain-community/src/llms/ai21.ts",
"../../libs/langchain-community/src/llms/aleph_alpha.ts",
"../../libs/langchain-community/src/llms/bedrock/index.ts",
"../../libs/langchain-community/src/llms/bedrock/web.ts",
"../../libs/langchain-community/src/llms/cloudflare_workersai.ts",
"../../libs/langchain-community/src/llms/cohere.ts",
"../../libs/langchain-community/src/llms/fireworks.ts",
"../../libs/langchain-community/src/llms/googlepalm.ts",
"../../libs/langchain-community/src/llms/googlevertexai/index.ts",
"../../libs/langchain-community/src/llms/googlevertexai/web.ts",
"../../libs/langchain-community/src/llms/gradient_ai.ts",
"../../libs/langchain-community/src/llms/hf.ts",
"../../libs/langchain-community/src/llms/llama_cpp.ts",
"../../libs/langchain-community/src/llms/ollama.ts",
"../../libs/langchain-community/src/llms/portkey.ts",
"../../libs/langchain-community/src/llms/raycast.ts",
"../../libs/langchain-community/src/llms/replicate.ts",
"../../libs/langchain-community/src/llms/sagemaker_endpoint.ts",
"../../libs/langchain-community/src/llms/togetherai.ts",
"../../libs/langchain-community/src/llms/watsonx_ai.ts",
"../../libs/langchain-community/src/llms/writer.ts",
"../../libs/langchain-community/src/llms/yandex.ts",
"../../libs/langchain-community/src/vectorstores/analyticdb.ts",
"../../libs/langchain-community/src/vectorstores/cassandra.ts",
"../../libs/langchain-community/src/vectorstores/chroma.ts",
"../../libs/langchain-community/src/vectorstores/clickhouse.ts",
"../../libs/langchain-community/src/vectorstores/closevector/node.ts",
"../../libs/langchain-community/src/vectorstores/closevector/web.ts",
"../../libs/langchain-community/src/vectorstores/cloudflare_vectorize.ts",
"../../libs/langchain-community/src/vectorstores/convex.ts",
"../../libs/langchain-community/src/vectorstores/elasticsearch.ts",
"../../libs/langchain-community/src/vectorstores/faiss.ts",
"../../libs/langchain-community/src/vectorstores/googlevertexai.ts",
"../../libs/langchain-community/src/vectorstores/hnswlib.ts",
"../../libs/langchain-community/src/vectorstores/lancedb.ts",
"../../libs/langchain-community/src/vectorstores/milvus.ts",
"../../libs/langchain-community/src/vectorstores/momento_vector_index.ts",
"../../libs/langchain-community/src/vectorstores/mongodb_atlas.ts",
"../../libs/langchain-community/src/vectorstores/myscale.ts",
"../../libs/langchain-community/src/vectorstores/neo4j_vector.ts",
"../../libs/langchain-community/src/vectorstores/opensearch.ts",
"../../libs/langchain-community/src/vectorstores/pgvector.ts",
"../../libs/langchain-community/src/vectorstores/pinecone.ts",
"../../libs/langchain-community/src/vectorstores/prisma.ts",
"../../libs/langchain-community/src/vectorstores/qdrant.ts",
"../../libs/langchain-community/src/vectorstores/redis.ts",
"../../libs/langchain-community/src/vectorstores/rockset.ts",
"../../libs/langchain-community/src/vectorstores/singlestore.ts",
"../../libs/langchain-community/src/vectorstores/supabase.ts",
"../../libs/langchain-community/src/vectorstores/tigris.ts",
"../../libs/langchain-community/src/vectorstores/typeorm.ts",
"../../libs/langchain-community/src/vectorstores/typesense.ts",
"../../libs/langchain-community/src/vectorstores/usearch.ts",
"../../libs/langchain-community/src/vectorstores/vectara.ts",
"../../libs/langchain-community/src/vectorstores/vercel_postgres.ts",
"../../libs/langchain-community/src/vectorstores/voy.ts",
"../../libs/langchain-community/src/vectorstores/weaviate.ts",
"../../libs/langchain-community/src/vectorstores/xata.ts",
"../../libs/langchain-community/src/vectorstores/zep.ts",
"../../libs/langchain-community/src/chat_models/baiduwenxin.ts",
"../../libs/langchain-community/src/chat_models/bedrock/index.ts",
"../../libs/langchain-community/src/chat_models/bedrock/web.ts",
"../../libs/langchain-community/src/chat_models/cloudflare_workersai.ts",
"../../libs/langchain-community/src/chat_models/fireworks.ts",
"../../libs/langchain-community/src/chat_models/googlevertexai/index.ts",
"../../libs/langchain-community/src/chat_models/googlevertexai/web.ts",
"../../libs/langchain-community/src/chat_models/googlepalm.ts",
"../../libs/langchain-community/src/chat_models/iflytek_xinghuo/index.ts",
"../../libs/langchain-community/src/chat_models/iflytek_xinghuo/web.ts",
"../../libs/langchain-community/src/chat_models/llama_cpp.ts",
"../../libs/langchain-community/src/chat_models/minimax.ts",
"../../libs/langchain-community/src/chat_models/ollama.ts",
"../../libs/langchain-community/src/chat_models/portkey.ts",
"../../libs/langchain-community/src/chat_models/yandex.ts",
"../../libs/langchain-community/src/callbacks/handlers/llmonitor.ts",
"../../libs/langchain-community/src/retrievers/amazon_kendra.ts",
"../../libs/langchain-community/src/retrievers/chaindesk.ts",
"../../libs/langchain-community/src/retrievers/databerry.ts",
"../../libs/langchain-community/src/retrievers/metal.ts",
"../../libs/langchain-community/src/retrievers/supabase.ts",
"../../libs/langchain-community/src/retrievers/tavily_search_api.ts",
"../../libs/langchain-community/src/retrievers/vectara_summary.ts",
"../../libs/langchain-community/src/retrievers/zep.ts",
"../../libs/langchain-community/src/caches/cloudflare_kv.ts",
"../../libs/langchain-community/src/caches/ioredis.ts",
"../../libs/langchain-community/src/caches/momento.ts",
"../../libs/langchain-community/src/caches/upstash_redis.ts",
"../../libs/langchain-community/src/graphs/neo4j_graph.ts",
"../../libs/langchain-community/src/utils/event_source_parse.ts",
"../../libs/langchain-community/src/document_transformers/html_to_text.ts",
"../../libs/langchain-community/src/document_transformers/mozilla_readability.ts",
"../../libs/langchain-community/src/storage/convex.ts",
"../../libs/langchain-community/src/storage/ioredis.ts",
"../../libs/langchain-community/src/storage/upstash_redis.ts",
"../../libs/langchain-community/src/storage/vercel_kv.ts",
"../../libs/langchain-community/src/stores/doc/base.ts",
"../../libs/langchain-community/src/stores/doc/in_memory.ts",
"../../libs/langchain-community/src/stores/message/cassandra.ts",
"../../libs/langchain-community/src/stores/message/cloudflare_d1.ts",
"../../libs/langchain-community/src/stores/message/convex.ts",
"../../libs/langchain-community/src/stores/message/dynamodb.ts",
"../../libs/langchain-community/src/stores/message/firestore.ts",
"../../libs/langchain-community/src/stores/message/in_memory.ts",
"../../libs/langchain-community/src/stores/message/ioredis.ts",
"../../libs/langchain-community/src/stores/message/momento.ts",
"../../libs/langchain-community/src/stores/message/mongodb.ts",
"../../libs/langchain-community/src/stores/message/planetscale.ts",
"../../libs/langchain-community/src/stores/message/redis.ts",
"../../libs/langchain-community/src/stores/message/upstash_redis.ts",
"../../libs/langchain-community/src/stores/message/xata.ts",
"../../libs/langchain-community/src/memory/chat_memory.ts",
"../../libs/langchain-community/src/memory/motorhead_memory.ts",
"../../libs/langchain-community/src/memory/zep.ts",
"../../libs/langchain-community/src/utils/convex.ts",
"../../libs/langchain-anthropic/src/index.ts",
"../../libs/langchain-google-genai/src/index.ts",
"../../libs/langchain-openai/src/index.ts",
"../../libs/langchain-mistralai/src/index.ts"
]
}
Loading

0 comments on commit 27a61c1

Please sign in to comment.