Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add pinecone destination connector #1774

Merged
merged 118 commits into from
Nov 29, 2023
Merged

Conversation

ahmetmeleq
Copy link
Contributor

@ahmetmeleq ahmetmeleq commented Oct 17, 2023

Closes #1414
Closes #2039

This PR:

  • Uses Pinecone python cli to implement a destination connector for Pinecone and provides the ingest readme requirements (here) for the connector
  • Updates documentation for the s3 destination connector
  • Alphabetically sorts setup.py contents
  • Updates logs for the chunking node in ingest pipeline
  • Adds a baseline session handle implementation for destination connectors, to be able to parallelize their operations
  • For the bug related to persisting element data to ingest embedding nodes; this PR tests the solution with its ingest test
  • Solves a bug on ingest chunking params with bugfix on chunking params and implementing related test

@ahmetmeleq ahmetmeleq self-assigned this Oct 17, 2023
@ahmetmeleq ahmetmeleq marked this pull request as ready for review October 25, 2023 02:45
@ahmetmeleq
Copy link
Contributor Author

ahmetmeleq commented Oct 25, 2023

to do: update the code with clean index / create index functions in the branch, make the test runnable

@ahmetmeleq ahmetmeleq enabled auto-merge November 29, 2023 21:48
@ahmetmeleq ahmetmeleq added this pull request to the merge queue Nov 29, 2023
Merged via the queue into main with commit ed08773 Nov 29, 2023
46 checks passed
@ahmetmeleq ahmetmeleq deleted the ahmet/pinecone-connector branch November 29, 2023 23:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

bug/ingest-chunking-parameters-not-passed feat: create downstream vectordb (pinecone) connector
4 participants