Skip to content

Commit

Permalink
pre commit
Browse files Browse the repository at this point in the history
  • Loading branch information
Rutger Cappendijk committed Oct 31, 2023
1 parent 12352d9 commit 9447eb1
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 9 deletions.
12 changes: 6 additions & 6 deletions src/intelligence_layer/core/evaluator.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ def evaluate(
expected_output: ExpectedOutput,
) -> Evaluation:
"""Executes the evaluation for this use-case.
Arguments:
input: Interface to be passed to the task that shall be evaluated.
logger: Debug logger used for tracing of tasks.
Expand All @@ -71,7 +71,7 @@ def evaluate(
Evaluation: interface of the metrics that come from the evaluated task.
The implementation of this method is responsible for running a task (usually supplied by the __init__ method)
and making any comparisons relevant to the evaluation.
and making any comparisons relevant to the evaluation.
Based on the results, it should create an `Evaluation` class with all the metrics and return it.
"""
pass
Expand All @@ -80,7 +80,7 @@ def evaluate_dataset(
self, dataset: Dataset[Input, ExpectedOutput], logger: DebugLogger
) -> AggregatedEvaluation:
"""Evaluates an entire datasets in a threaded manner and aggregates the results into an `AggregatedEvaluation`.
Arguments:
dataset: Dataset that will be used to evaluate a task.
logger: Logger used for tracing.
Expand Down Expand Up @@ -110,13 +110,13 @@ def evaluate_dataset(
@abstractmethod
def aggregate(self, evaluations: Sequence[Evaluation]) -> AggregatedEvaluation:
"""`Evaluator`-specific method for aggregating individual `Evaluations` into report-like `Aggregated Evaluation`.
Arguments:
evalautions: The results from running `evaluate_dataset` with a task.
evalautions: The results from running `evaluate_dataset` with a task.
Returns:
AggregatedEvaluation: The aggregated results of an evaluation run with a dataset.
This method is responsible for taking the results of an evaluation run and aggregating all the results.
It should create an `AggregatedEvaluation` class and return it at the end.
It should create an `AggregatedEvaluation` class and return it at the end.
"""
pass
6 changes: 3 additions & 3 deletions src/intelligence_layer/core/task.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,11 @@ def run(self, input: Input, logger: DebugLogger) -> Output:
"""Executes the implementation of run for this use case.
Args:
input: Generic input defined by the task implementation
input: Generic input defined by the task implementation
Returns:
output: Generic output defined by the task implementation
output: Generic output defined by the task implementation
This takes an input and runs the implementation to generate an output.
This takes an input and runs the implementation to generate an output.
It takes a `DebugLogger` for tracing of the process.
The Input and Output are logged by default.
"""
Expand Down

0 comments on commit 9447eb1

Please sign in to comment.