Merge from llamaindex main 2 (#16)

* v0.11.5 (run-llama#15834)

* Box bug fix 202409 (run-llama#15836)

* fix document matadata for box file

* fix BoxSearchOptions class initialization

* bump versions to 0.2.1

* Add oreilly course cookbooks (run-llama#15845)

* correct course name (run-llama#15859)

* clone llama-deploy docs during docs builds (run-llama#15794)

* chore(deps-dev): bump cryptography from 43.0.0 to 43.0.1 in /llama-index-core (run-llama#15840)

* Refresh Opensearch index after delete operation (run-llama#15854)

* Update ImageReader file loading logic (run-llama#15848)

* Minor bug in SQQE (run-llama#15861)

* v0.11.6 (run-llama#15863)

* remove widgets from recent notebooks (run-llama#15864)

* Fix image document deserialization issue (run-llama#15857)

* Bugfix: Remove warnings during data insertion for Hybrid Search due to duplicated columns in the schema (run-llama#15846)

* Add TablestoreVectorStore. (run-llama#15657)

* Removed unused `llama-index-llms-anthropic` dependency from Bedrock Converse (run-llama#15869)

Removed unused `llama-index-llms-anthropic` dependency. Incremented to `0.3.0`.

* Update bedrock.py (run-llama#15879)

Fix minor issue in error message

* Add tool calling example in NVIDIA notebook (run-llama#15842)

* NVIDIA: Completion fixes (run-llama#15820)

* Fix PGVectorStore with latest pydantic, update pydantic imports (run-llama#15886)

* add type ignore for streaming agents (run-llama#15887)

* [wip] force openai structured output (run-llama#15706)

* feature: Make SentenceSplitter's secondary_chunking_regex optional (run-llama#15882)

* Bug fix for KuzuPropertyGraphStore: Allow upserting relations even when chunks are absent (run-llama#15889)

* Missing tablestore (run-llama#15890)

* v0.11.7 (run-llama#15891)

* nit: extend workflow tutorial (run-llama#15901)

* Modify generate_qa_embedding_pairs to use LLM from Settings (run-llama#15904)

* docs: document how to disable global timeout along with more docs (run-llama#15912)

document how to disable global timeout along with more docs

* fix: model_name property and pydantic v2 in Azure Inference package (run-llama#15907)

* fix: model_name property and pydantic v2

* fix: tests

* Add InternalServerError to retry decorator (run-llama#15921)

* Add InternalServerError to retry decorator

* Bump version

* docs: fix broken link (run-llama#15924)

* Adding vector store for Azure Cosmos DB NoSql (run-llama#14158)

* Initial changes

* initial changes fixed with example

* Initial commit with all the code and test changes

* fixing test cases

* adding jupyter notebook

* cleaning files

* resolving comments

* fix linting

* more linting

* add a basic readme

---------

Co-authored-by: Aayush Kataria <[email protected]>
Co-authored-by: Massimiliano Pippi <[email protected]>

* update docs for custom embeddings (run-llama#15929)

* Add doc.id to Langchain format conversions (run-llama#15928)

* Update RankLLM with new rerankers (run-llama#15892)

* Feature Request run-llama#15810 :: Add DynamoDBChatStore (run-llama#15917)

* Adding support for `MetadataFilters` to WordLift Vector Store (run-llama#15905)

* Repair and route colab links to official repo (run-llama#15900)

* feat: Add a retry policy to steps (run-llama#15757)

* Fix error handling in sharepoint reader (run-llama#15868)

* feat: add llama-index llms alibabacloud_aisearch integration (run-llama#15850)

* Opensearch Serverless filtered query support using knn_score script (run-llama#15899)

* Mistral AI LLM Integration fixes (run-llama#15906)

* Fix RagCLI (run-llama#15931)

* v0.11.8 (run-llama#15933)

* docs: document retry policy (run-llama#15938)

* docs: update AI21 docs and readme (run-llama#15937)

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* docs: update ai21 docs and readme with jamba 1.5 and tool calling

* Retry retriable errors in neo4j integrations (run-llama#15915)

* Add four alibabacloud-aisearch llama-index integrations: rerank, node_parser, readers, embeddings (run-llama#15934)

* Differentiate sync and async calls in OpenSearchVectorClient (run-llama#15945)

* feat(postgres): add support for engine parameters (run-llama#15951)

* feat(postgres): add support for engine parameters
- Introduced engine_params to support passing parameters to create_engine.
- Updated create_engine and create_async_engine calls to include engine_params.
- Initialized engine_params in the constructor.

* style(lint): reformat for readability

* refactor(postgres): rename engine_params to create_engine_kwargs

* refactor(postgres): rename engine_params to create_engine_kwargs

* chore: bump version to 0.2.3

* fix(postgres): rename engine_params to create_engine_kwargs

* update falkordb client (run-llama#15940)

* update falkordb client

* bump version

* update version

* fix

* fix: Error when parsing output if tool name contains non-English characters (run-llama#15956)

* fix attribute error in PGVectorStore (run-llama#15961)

* Catch nest_asyncio errors (run-llama#15975)

* NUDGE (run-llama#15954)

* Add support for o1 openai models (run-llama#15979)

* force temp to 1.0 for o1 (run-llama#15983)

* Update concepts.md - fix link for Structured Data Extraction page (run-llama#15982)

* Fix the import path example for SimpleMongoReader (run-llama#15988)

* Do not pass system prompt from fn calling runner to fn calling worker (run-llama#15986)

* Add callback manager to retriever query engine from args (run-llama#15990)

* v0.11.9 (run-llama#15992)

* Fix: get all documents from Elasticsearch KVStore (run-llama#16006)

* Fix Pydantic numeric validation in openai/base.py (run-llama#15993)

* feat: add quip reader (run-llama#16000)

* [docs/example] Human in loop workflow example (run-llama#16011)

* start choose own adventure hitl nb

* hitl example

* note on alternative implementation

* add module guides and run prepare_for_build

* fix: removed author info from quip reader (run-llama#16012)

fix: removed author info

* Fix Pydantic models definition (run-llama#16008)

* chore(deps): bump litellm from 1.43.7 to 1.44.8 in /llama-index-integrations/embeddings/llama-index-embeddings-litellm (run-llama#16013)

* Hotfix: Fix Citations Text (run-llama#16015)

* Jacques/opik integration (run-llama#16007)

* update llamacloud index with image nodes (run-llama#15996)

* chore: add o1 models pinned versions (run-llama#16025)

* add sparse embedding abstraction (run-llama#16018)

* Implement `get_nodes` on `PGVectorStore` (run-llama#16026)

* Implement `get_nodes` on `PGVectorStore`

* Bump version num in pyproject.toml

* Update docstring in base.py in MilvusVectorStore adding COSINE as available similarity metric (run-llama#16031)

Update docstring in base.py adding COSINE as similarity metric available

According to Milvus Documentation, **COSINE** as similarity metric is supported (Both Milvus and Milvus Lite) but in Llama-Index docs was missing.

[Link to Milvus official docs](https://milvus.io/docs/metric.md?tab=floating#Similarity-Metrics)

I've checked in the [code](https://github.com/run-llama/llama_index/blob/723c2533ed4b7b43b7d814c89af1838f0f1994c2/llama-index-integrations/vector_stores/llama-index-vector-stores-milvus/llama_index/vector_stores/milvus/base.py#L256) , so indeed COSINE is supported, so no more further changes are needed.

* Fix: unnecessary warning issue in HuggingFace LLM when tokenizer is provided as argument(run-llama#16035) (run-llama#16037)

* Attempt #3 of context/result refactor (run-llama#16036)

* temporarily limit lancedb version (run-llama#16045)

* fix: new Data Connector adaption for DashVector (run-llama#16028)

* v0.11.10 (run-llama#16046)

* Fix serde issue for huggingface inference API embedding (run-llama#16053)

* wip

* wip

* fix: fix regression in OctoAI llm provider after 0.11 (run-llama#16002)

* fix OctoAI llm provider

* adjust code to the latest client

* resolve conflicts

* fix message format conversion

* do not pass max_tokens if None

* add chat test

* bump version

* update LanceDB integration (run-llama#16057)

* Adding docstring inside preprocess base code. (run-llama#16060)

* Add default to optional args for BedrockEmbedding (run-llama#16067)

* wip

* wip

* Fix optional type for Cohere embedding (run-llama#16068)

wip

* fix vertex pydantic arguments (run-llama#16069)

* Fix result order (run-llama#16078)

* Fix result order

* Bump version

* fix elasticsearch embedding async function (run-llama#16083)

* docs: add decorators to the api reference (run-llama#16081)

* fix incorrect parameters in VertexAIIndex (run-llama#16080)

* feat: update JinaEmbedding for v3 release (run-llama#15971)

* Mistralai enable custom endpoint from env (run-llama#16084)

* Implement async for multi modal ollama (run-llama#16091)

* Remove circular package dep (run-llama#16070)

* [chore] fix incorrect `handler.context` for `handler.ctx` in docs (run-llama#16101)

fix

* fix bug missing import (run-llama#16096)

* Add more workflow references to docs (run-llama#16102)

* fix cohere tests (run-llama#16104)

* Use response synthesizer in context chat engines (run-llama#16017)

* Fix mongodb hybrid search, also pass hybrid_top_k in vector retriever (run-llama#16105)

* bump openai agent deps (run-llama#16112)

* Fixed up test_vectorstore.  (run-llama#16113)

* Issue 16071: wordpress requires username, password (run-llama#16072)

* Issue 16071: wordpress requires username, password

* Adding changes suggested in PR template

* Use Optional typing keyword

* feat: add configurable base_url field in rerank (run-llama#16050)

* add base_url

* version bump

* add default

---------

Co-authored-by: Massimiliano Pippi <[email protected]>
Co-authored-by: Logan <[email protected]>

* Enhance Pandas Query Engine Output Processor (run-llama#16052)

* Enhance output processor to temporarily adjust display options.

* Make format changes.

* Update pyproject.toml

---------

Co-authored-by: Logan <[email protected]>
Co-authored-by: Massimiliano Pippi <[email protected]>

* fix workflow docs (run-llama#16117)

cr

* Update OpenVINO LLM pyproject.toml (run-llama#16130)

* Wordpress: Allow control of whether Pages and/or Posts are retrieved (run-llama#16128)

* Async achat put operation (run-llama#16127)

* [fix] handler.stream_events() doesn't yield `StopEvent` (run-llama#16115)

* Improved TLM Rag cookbook (run-llama#16109)

* Update chat message class for multi-modal (run-llama#15969)

* Add support for Path for SimpleDirectoryReader (run-llama#16108)

* Add TopicNodeParser based on MedGraphRAG paper (run-llama#16131)

* Sql markdown response (run-llama#16103)

* v0.11.11 (run-llama#16134)

* Correct Pydantic warning(s) issed for llama-index-llms-ibm (run-llama#16141)

Fix Pydantic warnings in llam-index-llms-ibm

* feat: add drive link to google drive reader (run-llama#16156)

* Introducing new VoyageAI models (run-llama#16150)

* Add `required_exts` option to SharePoint reader (run-llama#16152)

* User-defined schema in MilvusVectorStore (run-llama#16151)

* safe format prompt variables in strings with JSON (run-llama#15734)

* account for tools in prompt helper (run-llama#16157)

* v0.11.12 (run-llama#16159)

---------

Co-authored-by: Logan <[email protected]>
Co-authored-by: Ravi Theja <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yannic Spreen-Ledebur <[email protected]>
Co-authored-by: Adnan Alkattan <[email protected]>
Co-authored-by: Laurie Voss <[email protected]>
Co-authored-by: saipjkai <[email protected]>
Co-authored-by: ScriptShi <[email protected]>
Co-authored-by: Bryce Freshcorn <[email protected]>
Co-authored-by: Graham Tibbitts <[email protected]>
Co-authored-by: Rashmi Pawar <[email protected]>
Co-authored-by: Jerry Liu <[email protected]>
Co-authored-by: Caroline Binley <[email protected]>
Co-authored-by: Prashanth Rao <[email protected]>
Co-authored-by: Asi Greenholts <[email protected]>
Co-authored-by: Massimiliano Pippi <[email protected]>
Co-authored-by: Facundo Santiago <[email protected]>
Co-authored-by: gsa9989 <[email protected]>
Co-authored-by: Aayush Kataria <[email protected]>
Co-authored-by: Ryan Nguyen <[email protected]>
Co-authored-by: David Riccitelli <[email protected]>
Co-authored-by: GICodeWarrior <[email protected]>
Co-authored-by: Javier Torres <[email protected]>
Co-authored-by: 才胜 <[email protected]>
Co-authored-by: George Dittmar <[email protected]>
Co-authored-by: Tarun Jain <[email protected]>
Co-authored-by: miri-bar <[email protected]>
Co-authored-by: Tomaz Bratanic <[email protected]>
Co-authored-by: kobiche <[email protected]>
Co-authored-by: Arthur Moura Carvalho <[email protected]>
Co-authored-by: Avi Avni <[email protected]>
Co-authored-by: Cheese <[email protected]>
Co-authored-by: Zac Wellmer <[email protected]>
Co-authored-by: Ragul Kachiappan <[email protected]>
Co-authored-by: Vinicius <[email protected]>
Co-authored-by: Bidimpata-Kerim Aramyan-Tshimanga <[email protected]>
Co-authored-by: Laura Ceconi <[email protected]>
Co-authored-by: N1eo <[email protected]>
Co-authored-by: Chirag Agrawal <[email protected]>
Co-authored-by: Andrei Fajardo <[email protected]>
Co-authored-by: David Oplatka <[email protected]>
Co-authored-by: Jacques Verré <[email protected]>
Co-authored-by: Harpinder <[email protected]>
Co-authored-by: Richard Liu <[email protected]>
Co-authored-by: Jorge Barrachina Gutiérrez <[email protected]>
Co-authored-by: Sourabh <[email protected]>
Co-authored-by: OceanPresent <[email protected]>
Co-authored-by: Simon Suo <[email protected]>
Co-authored-by: Ayush Chaurasia <[email protected]>
Co-authored-by: preprocess-co <[email protected]>
Co-authored-by: guodong <[email protected]>
Co-authored-by: polarbear567 <[email protected]>
Co-authored-by: Aaron Ji <[email protected]>
Co-authored-by: enrico-stauss <[email protected]>
Co-authored-by: Selim Çavaş <[email protected]>
Co-authored-by: Casey Clements <[email protected]>
Co-authored-by: Jonathan Springer <[email protected]>
Co-authored-by: Anirudh31415926535 <[email protected]>
Co-authored-by: Matin Khajavi <[email protected]>
Co-authored-by: Ethan Yang <[email protected]>
Co-authored-by: Matthew Turk <[email protected]>
Co-authored-by: Andrew Kim <[email protected]>
Co-authored-by: José Henrique Luckmann <[email protected]>
Co-authored-by: Emanuel Ferreira <[email protected]>
Co-authored-by: fzowl <[email protected]>
Co-authored-by: João Martins <[email protected]>
Co-authored-by: Petros Mitseas <[email protected]>