init_prompts.json

{
    "coherence": "Evaluate and compare the coherence of the two summary candidates for the given source text.                         Consider coherence aspects such as clarity and logical flow.                         A summary is coherent if it accurately captures the key information from the article, and presents them in a clear manner.                         Which summary candidate has better coherence?                         If the candidate A is better, please return 'A'.                         If the candidate B is better, please return 'B'.                         You must return the choice only.",
    "fluency": "Evaluate and compare the fluency of the two summary candidates for the given source text.                         Which summary candidate has better fluency?                         If the candidate A is better, please return 'A'.                         If the candidate B is better, please return 'B'.                         You must return the choice only.",
    "consistency": "Evaluate and compare the consistency of the two summary candidates for the given source text.                         A summary is consistent with the article if it faithfully reflects the main points, facts, and tone of the article.                         A summary is inconsistent if it introduces any errors, contradictions, or distortions of the original article.                         Which summary candidate has better consistency?                         If the candidate A is better, please return 'A'.                         If the candidate B is better, please return 'B'.                         You must return the choice only.",
    "relevance": "Evaluate and compare the relevance of the two summary candidates for the given source text.                         A summary is relevant if it captures the main points from the article, without leaving out any crucial details or adding any unnecessary or inaccurate ones.                         A summary is more relevant if it uses the same or similar terms and expressions as the article.                         A summary is less relevant if it omits some of the key facts from the article, or if it introduces irrelevant information that is not supported by the article.                         Which summary candidate has better relevance?                         If the candidate A is better, please return 'A'.                         If the candidate B is better, please return 'B'.                         You must return the choice only.",
    "informativeness": "Evaluate and compare the informativeness of the two summary candidates for the given source text.                         Evaluate how each summary converts their input text to natural language text, without omitting, adding, or distorting any facts.                         Which summary candidate has better informativeness?                         If the candidate A is better, please return 'A'.                         If the candidate B is better, please return 'B'.                         You must return the choice only.",
    "engaging": "Evaluate and compare the engagement of the two response candidates for the given dialog history.                     Which response candidate is more engaging?                     If the candidate A is better, please return 'A'.                     If the candidate B is better, please return 'B'.                     You must return the choice only.",
    "natural": "Evaluate and compare the naturalness of the two response candidates for the given dialog history.                     Which response candidate is more natural?                     If the candidate A is better, please return 'A'.                     If the candidate B is better, please return 'B'.                     You must return the choice only.",
    "overall": "Evaluate and compare the overall response of the two candidates for the given dialog history.                     Which response candidate has better quality?                     If the candidate A is better, please return 'A'.                     If the candidate B is better, please return 'B'.                     You must return the choice only."
}