Skip to content

Commit

Permalink
Merge branch 'main' into release
Browse files Browse the repository at this point in the history
  • Loading branch information
@SebastianNiehusAA
SebastianNiehusAA authored May 31, 2024
2 parents 18946cb + 90388d3 commit fccd8bd
Show file tree
Hide file tree
Showing 17 changed files with 362 additions and 218 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,16 @@
- This is not supported for the `OpenTelemetryTracer` because of technical incompatibilities.
- All exported spans now contain the status of the span.
- We now support python 3.12
- Add `description` parameter to `Evaluator.evaluate_runs` and `Runner.run_dataset` to allow individual descriptions without the need to create a new `Evaluator` or `Runner`.

### Fixes
- The document index client now correctly URL-encodes document names in its queries.
- The `ArgillaEvaluator` not properly supports `dataset_name`
- Update outdated `how_to_human_evaluation_via_argilla.ipynb`
- Fix bug in `FileSystemBasedRepository` causing spurious mkdir failure if the file actually exists
- Update broken README links to Read The Docs
- The `evaluation` tutorial contained a broken multi-label classify example. This was fixed.


### Deprecations
...
Expand Down
30 changes: 15 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ The key features of the Intelligence Layer are:
- [How to use the Intelligence Layer in Docker](#how-to-use-the-intelligence-layer-in-docker)
- [Via the GitHub repository](#via-the-github-repository)
- [Getting started](#getting-started)
- [Setup LLM access](#setup-llm-access)
- [Setup LLM access](#setup-llm-access)
- [Tutorial Notebooks](#tutorial-notebooks)
- [How-Tos](#how-tos)
- [Models](#models)
Expand Down Expand Up @@ -176,20 +176,20 @@ To give you a starting point for using the Intelligence Layer, we provide some p

| Type | Task | Description |
|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Classify | [EmbeddingBasedClassify](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.EmbeddingBasedClassify) | Classify a short text by computing its similarity with example texts for each class. |
| Classify | [PromptBasedClassify](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.PromptBasedClassify) | Classify a short text by assessing each class' probability using zero-shot prompting. |
| Classify | [PromptBasedClassifyWithDefinitions](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.PromptBasedClassifyWithDefinitions) | Classify a short text by assessing each class' probability using zero-shot prompting. Each class is defined by a natural language description. |
| Classify | [KeywordExtract](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.KeywordExtract) | Generate matching labels for a short text. |
| QA | [MultipleChunkRetrieverQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.MultipleChunkRetrieverQa) | Answer a question based on an entire knowledge base. Recommended for most RAG-QA use-cases. |
| QA | [LongContextQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.LongContextQa) | Answer a question based on one document of any length. |
| QA | [MultipleChunkQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.MultipleChunkQa) | Answer a question based on a list of short texts. |
| QA | [SingleChunkQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.SingleChunkQa) | Answer a question based on a short text. |
| QA | [RetrieverBasedQa (deprecated)](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.RetrieverBasedQa) | Answer a question based on a document base using a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Search | [Search](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.Search) | Search for texts in a document base using a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Search | [ExpandChunks](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.ExpandChunks) | Expand chunks retrieved with a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Summarize | [SteerableLongContextSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.SteerableLongContextSummarize) | Condense a long text into a summary with a natural language instruction. |
| Summarize | [SteerableSingleChunkSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.SteerableSingleChunkSummarize) | Condense a short text into a summary with a natural language instruction. |
| Summarize | [RecursiveSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.use_cases.html#intelligence_layer.use_cases.RecursiveSummarize) | Recursively condense a text into a summary. |
| Classify | [EmbeddingBasedClassify](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.EmbeddingBasedClassify) | Classify a short text by computing its similarity with example texts for each class. |
| Classify | [PromptBasedClassify](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.PromptBasedClassify) | Classify a short text by assessing each class' probability using zero-shot prompting. |
| Classify | [PromptBasedClassifyWithDefinitions](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.PromptBasedClassifyWithDefinitions) | Classify a short text by assessing each class' probability using zero-shot prompting. Each class is defined by a natural language description. |
| Classify | [KeywordExtract](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.KeywordExtract) | Generate matching labels for a short text. |
| QA | [MultipleChunkRetrieverQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.MultipleChunkRetrieverQa) | Answer a question based on an entire knowledge base. Recommended for most RAG-QA use-cases. |
| QA | [LongContextQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.LongContextQa) | Answer a question based on one document of any length. |
| QA | [MultipleChunkQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.MultipleChunkQa) | Answer a question based on a list of short texts. |
| QA | [SingleChunkQa](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.SingleChunkQa) | Answer a question based on a short text. |
| QA | [RetrieverBasedQa (deprecated)](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.RetrieverBasedQa) | Answer a question based on a document base using a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Search | [Search](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.Search) | Search for texts in a document base using a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Search | [ExpandChunks](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.ExpandChunks) | Expand chunks retrieved with a [BaseRetriever](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.connectors.html#intelligence_layer.connectors.BaseRetriever) implementation. |
| Summarize | [SteerableLongContextSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.SteerableLongContextSummarize) | Condense a long text into a summary with a natural language instruction. |
| Summarize | [SteerableSingleChunkSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.SteerableSingleChunkSummarize) | Condense a short text into a summary with a natural language instruction. |
| Summarize | [RecursiveSummarize](https://aleph-alpha-intelligence-layer.readthedocs-hosted.com/en/latest/intelligence_layer.examples.html#intelligence_layer.examples.RecursiveSummarize) | Recursively condense a text into a summary. |

Note that we do not expect the above use cases to solve all of your issues.
Instead, we encourage you to think of our pre-configured use cases as a foundation to fast-track your development process.
Expand Down
11 changes: 10 additions & 1 deletion docs/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,20 @@
Welcome to Intelligence Layer's documentation!
==============================================

This documentation provides API-level code documentation for the Intelligence Layer, covering modules such as connectors, core, evaluation, and examples.

A comprehensive overview of the Intelligence Layer can be found on GitHub, including `tutorials <https://github.com/Aleph-Alpha/intelligence-layer-sdk/tree/main?tab=readme-ov-file#tutorial-notebooks>`_ and `how-tos <https://github.com/Aleph-Alpha/intelligence-layer-sdk/tree/main?tab=readme-ov-file#how-tos>`_.
This code documentation is intended to serve as a reference companion to the main documentation, providing detailed information on the Intelligence Layer's API and implementation.

.. toctree::
:maxdepth: 1
:caption: Contents:

intelligence_layer
intelligence_layer.connectors
intelligence_layer.core
intelligence_layer.evaluation
intelligence_layer.examples


Indices and tables
==================
Expand Down
2 changes: 1 addition & 1 deletion docs/intelligence_layer.connectors.rst
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
intelligence\_layer.connectors package
intelligence\_layer.connectors
======================================

Module contents
Expand Down
2 changes: 1 addition & 1 deletion docs/intelligence_layer.core.rst
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
intelligence\_layer.core package
intelligence\_layer.core
================================

Module contents
Expand Down
2 changes: 1 addition & 1 deletion docs/intelligence_layer.evaluation.rst
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
intelligence\_layer.evaluation package
intelligence\_layer.evaluation
======================================


Expand Down
2 changes: 1 addition & 1 deletion docs/intelligence_layer.use_cases.rst → docs/intelligence_layer.examples.rst
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
intelligence\_layer.use\_cases package
intelligence\_layer.examples
======================================


Expand Down
18 changes: 0 additions & 18 deletions docs/intelligence_layer.rst
View file

This file was deleted.

Loading

0 comments on commit fccd8bd

Please sign in to comment.