[BUG][WIP]: Connection pool FD leak

[tazarov]

Description of changes

Summarize the changes made by this PR.

• Improvements & Bug fixes
  • Replacing thread locals with context vars for TxWrapper stacks
  • Replacing thread locals with a connection pool control via locks

Test plan

How are these changes tested?

• Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs repository?

[github-actions]

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

• Can you think of any use case in which the code does not behave as intended? Have they been tested?
• Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
• If appropriate, are there adequate property based tests?
• If appropriate, are there adequate unit tests?
• Should any logging, debugging, tracing information be added or removed?
• Are error messages user-friendly?
• Have all documentation changes needed been made?
• Have all non-obvious changes been commented?

System Compatibility

• Are there any potential impacts on other parts of the system or backward compatibility?
• Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

• Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

[tazarov]

Current thread 0x0000000175443000 (most recent call first):
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/segment/impl/metadata/sqlite.py", line 225 in _records
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/segment/impl/metadata/sqlite.py", line 216 in get_metadata
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/telemetry/opentelemetry/__init__.py", line 143 in wrapper
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/api/segment.py", line 724 in _query
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/rate_limiting/__init__.py", line 45 in wrapper
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/telemetry/opentelemetry/__init__.py", line 143 in wrapper
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/chromadb/server/fastapi/__init__.py", line 784 in process_query
  File "/Users/user/experiments/chroma/oss/taz-sprint-11/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 851 in run
  File "/Users/user/.pyenv/versions/3.11.7/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
  File "/Users/user/.pyenv/versions/3.11.7/lib/python3.11/threading.py", line 1002 in _bootstrap
[1]    4689 segmentation fault  anonymized_telemetry=False python -X faulthandler -m chromadb.cli.cli run 

how interesting 🧐

[tazarov]

The gift that keeps on failing:

[1]    23227 illegal hardware instruction  anonymized_telemetry=False python -X faulthandler -m chromadb.cli.cli run  

[tazarov]

Rabbit hole:

Connection:  5431431504 Thread:  15576228
Connection:  5431492880 Thread:  15576215 <-- what now
Connection:  5431431504 Thread:  15576228
Checking stack:  [5430868176, 5431431504, 5431492880]

[tazarov]

Closing this in favor of #2014, a simpler approach with weakrefs