Initial commit for RAG in py-ml

[hmumtazz]

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dhrubo-os]

DCO is missing

[brianf-aws]

really cool stuff here! 📖 🤖 💬 Lets try and refactor some code so its reusable across classes also lets apply SRP so that a class isnt burdened by doing a lot at once.

Make sure to add documentation to files and methods.
Also Im seeing there exists a query python file within py-ml https://github.com/opensearch-project/opensearch-py-ml/blame/main/opensearch_py_ml/query.py if possible maybe we can aggregate to existing code?

Lastly lets come up with a great description of the feature. You put a lot of effort so lets make it visually appealing in the PR description (diagrams, how to use, gifs, concise summary; emojis) You can use this as influence opensearch-project/neural-search#933 talk to @dhrubo-os if maybe we want to open up a Issue and then link this PR not sure if thats too much.

Great work!

[brianf-aws]

Is there input validation involved? maybe we can catch this earlier so it doesn't have to be a headache later?

[dhrubo-os]

DCO is missing.

[brianf-aws]

Lets put a new line on the following files thats probably why its not able to pick up the license header

No license header found in:

• opensearch_py_ml/ml_commons/rag_pipeline/init.py
• opensearch_py_ml/ml_commons/rag_pipeline/rag/init.py
• opensearch_py_ml/ml_commons/rag_pipeline/rag/rag.py
• opensearch_py_ml/ml_commons/rag_pipeline/rag/ml_models/init.py

[dhrubo-os]

@hmumtazz lint is still failing:

Run nox -s lint
  nox -s lint
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
nox > Running session lint
nox > Creating virtual environment (virtualenv) using python3 in .nox/lint
nox > python -m pip install black flake8 mypy isort numpy
nox > python utils/lint/license-headers.py check setup.py noxfile.py opensearch_py_ml/ utils/ tests/
All files had license header
nox > black --check --target-version=py38 setup.py noxfile.py opensearch_py_ml/ utils/ tests/
All done! ✨ 🍰 ✨
180 files would be left unchanged.
nox > isort --check --profile=black setup.py noxfile.py opensearch_py_ml/ utils/ tests/
nox > flake8 --ignore=E501,W503,E402,E712,E203 setup.py noxfile.py opensearch_py_ml/ utils/ tests/
opensearch_py_ml/ml_commons/rag_pipeline/rag/serverless.py:189:96: E226 missing whitespace around arithmetic operator
nox > Command flake8 --ignore=E501,W503,E402,E712,E203 setup.py noxfile.py opensearch_py_ml/ utils/ tests/ failed with exit code 1
nox > Session lint failed.

[dhrubo-os]

Integ tests are failing too!! Please fix these two