refactor eval code, create eval cli

[Ben-Epstein]

there was massive overlap between the eval rag and eval retriever scripts.

I broke those out into functions in eval/utils.py so they could be reused.

Cleaned up the functions a bit, and then built the eval CLI

Tested with the following (note I had to comment out this line to work with gpt2

# install the dalm repo
pip install -e .

# train rag e2e
dalm train-rag-e2e \
"./dalm/datasets/toy_data_train.csv" \
"BAAI/bge-small-en" \
"gpt2" \
--output-dir "rag_e2e_checkpoints" \
--per-device-train-batch-size 32

# eval retriever
dalm eval-retriever "./dalm/datasets/toy_data_train.csv" \
 --retriever-name-or-path "BAAI/bge-small-en" \
 --retriever-peft-model-path "rag_e2e_checkpoints/retriever" \
 --embed-dim 384

###
Construct passage index
Evaluation start
Retriever results:
Recall: 0.10000000000000003
Precision: 1.0
Hit Rate: 1.0
*************
###

# eval rag
dalm eval-rag "./dalm/datasets/toy_data_train.csv"  \
 --retriever-name-or-path "BAAI/bge-small-en" \
 --generator-name-or-path "gpt2" \
 --retriever-peft-model-path rag_e2e_checkpoints/retriever \
 --generator-peft-model-path rag_e2e_checkpoints/generator \
 --query-batch-size 5 \
 --embed-dim 384

###
A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set `padding_side='left'` when initializing the tokenizer.
Retriever results:
Recall: 0.10000000000000003
Precision: 1.0
Hit Rate: 1.0
*************
Generator evaluation:
Exact match: 0.0
###

[shamanez]

Is this okay to pass a function? I am ok with it. Is there a better way?

[Ben-Epstein]

it's pretty normal to pass callables typing.Callable around :)

[shamanez]

There are major structural changes. I am ok with the logic. But please run the eval with the given dataset and confirm we get the same results.