From c7109dc9791c7202318160c94692dd8e170a51c7 Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Wed, 31 May 2023 15:47:18 -0400 Subject: [PATCH 1/7] USAGE.md: add section mapping documentation in AI revision part --- USAGE.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/USAGE.md b/USAGE.md index c867139f..2b377339 100644 --- a/USAGE.md +++ b/USAGE.md @@ -332,15 +332,21 @@ Spellchecking is currently only supported for English language manuscripts. ## AI-assisted authoring The workflow [`ai-revision`](.github/workflows/ai-revision.yaml) is available to assist authors in writing their manuscripts. -It uses large language models to revise the manuscript text, fixing spelling and grammar errors, and improving the sentence structure and the writing style with section-specific prompts. +It uses large language models to revise the manuscript text, fix spelling and grammar errors, and improve the sentence structure and the writing style with section-specific prompts. It is manually triggered by the user (it never runs automatically), and it generates a pull request with suggested revisions. Then the user can review these changes and merge the pull request if they are acceptable. More information about this tool is available in [this manuscript](https://greenelab.github.io/manubot-gpt-manuscript/). You need to change your repository settings to 1) provide a secret with name `OPENAI_API_KEY` containing your OpenAI API token, and 2) allow workflows to create pull requests. +Optionally, you need to 3) provide the manuscript section each of your Markdown files belongs to (Introduction, Methods, etc.). For 1), go to the settings page and, within "Secrets and variables," select "Actions." Next, create a repository secret with the name `OPENAI_API_KEY` and the value of the API token (you can also do this using "Organization secrets" if available). For 2), go to "Actions", "General", "Workflow permissions", and activate the checkbox "Allow GitHub Actions to create and approve pull requests." +For 3), since the tool uses section-specific prompts for revision, it needs to know which section your Markdown files belong to. +For this, it will try to infer them from the file names automatically. +If this fails, the tool might not revise all of your files. +In this case, you need to modify the "Revise manuscript" step in the workflow file (`.github/workflows/ai-revision.yaml`) to indicate the section of each file using the `AI_EDITOR_FILENAME_SECTION_MAPPING` environment variable that is described [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py). +An example would be `AI_EDITOR_FILENAME_SECTION_MAPPING: '{"05.main-text.md": "methods"}'`. By default, the tool uses the model `text-davinci-003`. Make sure to check the [pricing](https://openai.com/api/pricing/) of the OpenAI API. From 4022a63d8639610d9c2f8c57bcdb15033dfac35e Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Thu, 1 Jun 2023 16:03:56 -0400 Subject: [PATCH 2/7] add custom prompt field --- .github/workflows/ai-revision.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/ai-revision.yaml b/.github/workflows/ai-revision.yaml index 58eb0d35..db2bac5c 100644 --- a/.github/workflows/ai-revision.yaml +++ b/.github/workflows/ai-revision.yaml @@ -17,6 +17,11 @@ on: required: true type: string default: 'text-davinci-003' + custom_prompt: + description: 'Custom prompt' + required: false + type: string + default: '' branch_name: description: 'Output branch' required: true @@ -54,6 +59,7 @@ jobs: OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} AI_EDITOR_LANGUAGE_MODEL: ${{ inputs.model }} AI_EDITOR_FILENAMES_TO_REVISE: ${{ inputs.file_names }} + AI_EDITOR_CUSTOM_PROMPT: ${{ inputs.custom_prompt }} # More variables can be specified to control the behavior of the model: # https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py run: manubot ai-revision --content-directory content/ From 2824043fbfd020dc86f114e7ce86250f45d4df7c Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Fri, 2 Jun 2023 09:59:52 -0400 Subject: [PATCH 3/7] USAGE.md: add paragraph about types of prompts --- USAGE.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/USAGE.md b/USAGE.md index 2b377339..b61fc739 100644 --- a/USAGE.md +++ b/USAGE.md @@ -338,17 +338,20 @@ Then the user can review these changes and merge the pull request if they are ac More information about this tool is available in [this manuscript](https://greenelab.github.io/manubot-gpt-manuscript/). You need to change your repository settings to 1) provide a secret with name `OPENAI_API_KEY` containing your OpenAI API token, and 2) allow workflows to create pull requests. -Optionally, you need to 3) provide the manuscript section each of your Markdown files belongs to (Introduction, Methods, etc.). For 1), go to the settings page and, within "Secrets and variables," select "Actions." Next, create a repository secret with the name `OPENAI_API_KEY` and the value of the API token (you can also do this using "Organization secrets" if available). For 2), go to "Actions", "General", "Workflow permissions", and activate the checkbox "Allow GitHub Actions to create and approve pull requests." -For 3), since the tool uses section-specific prompts for revision, it needs to know which section your Markdown files belong to. -For this, it will try to infer them from the file names automatically. -If this fails, the tool might not revise all of your files. -In this case, you need to modify the "Revise manuscript" step in the workflow file (`.github/workflows/ai-revision.yaml`) to indicate the section of each file using the `AI_EDITOR_FILENAME_SECTION_MAPPING` environment variable that is described [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py). -An example would be `AI_EDITOR_FILENAME_SECTION_MAPPING: '{"05.main-text.md": "methods"}'`. -By default, the tool uses the model `text-davinci-003`. +Additionally, you also need to decide which type of prompts you want to use. +"Prompts" are the instructions given to the language model to revise your manuscript. +Basically, you can select 1) the default set of section-specific prompts already provided by the tool, or 2) a custom prompt that you provide. +For 1), you can check [this manuscript](https://greenelab.github.io/manubot-gpt-manuscript/) for a more detailed description of the section-specific prompts. +These prompts are already provided, but they need to know the section of each of your Markdown files (for instance, if it is the abstract, or the introduction, etc.). +For this, the tool will try to infer them from the file names automatically, and if this fails, the tool might not revise all of your files. +In this case, you need to indicate the section of each file using the "section mapping" environment variable that is described [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) (read the header of the file for more instructions). + + +By default, the tool uses the model `text-davinci-003`, but you are encouraged to check the [OpenAI documentation](https://platform.openai.com/docs/models) to see which models are available, which one is the most suitable for your manuscript, and [whether our tools supports it](https://github.com/manubot/manubot-ai-editor). Make sure to check the [pricing](https://openai.com/api/pricing/) of the OpenAI API. With $0.02 per 1000 tokens using the most powerful AI models, the cost for a revision of a standard manuscript (around 35 paragraphs) should be around $0.50. The workflow allows specifying the branch and file names (in the `content/` directory) to revise, the language model to use, and the output branch name. From 644e53183b9d5a991feb453135b85affa5ba2a04 Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Fri, 2 Jun 2023 10:00:35 -0400 Subject: [PATCH 4/7] USAGE.md: add paragraph about types of prompts --- USAGE.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/USAGE.md b/USAGE.md index b61fc739..87a702c5 100644 --- a/USAGE.md +++ b/USAGE.md @@ -349,7 +349,10 @@ For 1), you can check [this manuscript](https://greenelab.github.io/manubot-gpt- These prompts are already provided, but they need to know the section of each of your Markdown files (for instance, if it is the abstract, or the introduction, etc.). For this, the tool will try to infer them from the file names automatically, and if this fails, the tool might not revise all of your files. In this case, you need to indicate the section of each file using the "section mapping" environment variable that is described [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) (read the header of the file for more instructions). - +For 2), you can provide your own custom prompt, which will be used for all the files regardless of their section. +For example, instead of the more complex section-specific prompts in 1), you might just want to perform simpler revision tasks. +An example of custom prompt is "proofread the following paragraph". +Read the documentation [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) to set the "custom prompt" environment variable. By default, the tool uses the model `text-davinci-003`, but you are encouraged to check the [OpenAI documentation](https://platform.openai.com/docs/models) to see which models are available, which one is the most suitable for your manuscript, and [whether our tools supports it](https://github.com/manubot/manubot-ai-editor). Make sure to check the [pricing](https://openai.com/api/pricing/) of the OpenAI API. From 5d3be367c9d336f2d91342dbc7dfc8e9d14c3c3d Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Fri, 2 Jun 2023 10:06:53 -0400 Subject: [PATCH 5/7] USAGE.md: add paragraph about types of prompts --- USAGE.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/USAGE.md b/USAGE.md index 87a702c5..8b97edab 100644 --- a/USAGE.md +++ b/USAGE.md @@ -351,8 +351,10 @@ For this, the tool will try to infer them from the file names automatically, and In this case, you need to indicate the section of each file using the "section mapping" environment variable that is described [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) (read the header of the file for more instructions). For 2), you can provide your own custom prompt, which will be used for all the files regardless of their section. For example, instead of the more complex section-specific prompts in 1), you might just want to perform simpler revision tasks. -An example of custom prompt is "proofread the following paragraph". -Read the documentation [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) to set the "custom prompt" environment variable. +An example of a custom prompt is "proofread the following paragraph". +You can provide your custom prompt when you manually trigger the workflow by using the "custom prompt" field. +This could be more appropriate if you are testing different prompts. +To set a fixed prompt for all runs, read the documentation [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) to set the "custom prompt" environment variable. By default, the tool uses the model `text-davinci-003`, but you are encouraged to check the [OpenAI documentation](https://platform.openai.com/docs/models) to see which models are available, which one is the most suitable for your manuscript, and [whether our tools supports it](https://github.com/manubot/manubot-ai-editor). Make sure to check the [pricing](https://openai.com/api/pricing/) of the OpenAI API. From 20df6353f9d23e1f18302a1c21426227094d3d13 Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Fri, 2 Jun 2023 10:08:12 -0400 Subject: [PATCH 6/7] USAGE.md: add paragraph about types of prompts --- USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/USAGE.md b/USAGE.md index 8b97edab..74ace722 100644 --- a/USAGE.md +++ b/USAGE.md @@ -352,7 +352,7 @@ In this case, you need to indicate the section of each file using the "section m For 2), you can provide your own custom prompt, which will be used for all the files regardless of their section. For example, instead of the more complex section-specific prompts in 1), you might just want to perform simpler revision tasks. An example of a custom prompt is "proofread the following paragraph". -You can provide your custom prompt when you manually trigger the workflow by using the "custom prompt" field. +You can provide your custom prompt when you manually trigger the workflow by using the "Custom prompt" field. This could be more appropriate if you are testing different prompts. To set a fixed prompt for all runs, read the documentation [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) to set the "custom prompt" environment variable. From c6f2b40d8f4175eb98e89cad5b278e4c07295579 Mon Sep 17 00:00:00 2001 From: Milton Pividori Date: Fri, 2 Jun 2023 10:09:04 -0400 Subject: [PATCH 7/7] USAGE.md: add paragraph about types of prompts --- USAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/USAGE.md b/USAGE.md index 74ace722..2ace93b7 100644 --- a/USAGE.md +++ b/USAGE.md @@ -356,7 +356,7 @@ You can provide your custom prompt when you manually trigger the workflow by usi This could be more appropriate if you are testing different prompts. To set a fixed prompt for all runs, read the documentation [here](https://github.com/manubot/manubot-ai-editor/blob/main/libs/manubot_ai_editor/env_vars.py) to set the "custom prompt" environment variable. -By default, the tool uses the model `text-davinci-003`, but you are encouraged to check the [OpenAI documentation](https://platform.openai.com/docs/models) to see which models are available, which one is the most suitable for your manuscript, and [whether our tools supports it](https://github.com/manubot/manubot-ai-editor). +By default, the tool uses the model `text-davinci-003`, but you are encouraged to check the [OpenAI documentation](https://platform.openai.com/docs/models) to see which models are available, which one is the most suitable for your manuscript, and [whether our tool supports it](https://github.com/manubot/manubot-ai-editor). Make sure to check the [pricing](https://openai.com/api/pricing/) of the OpenAI API. With $0.02 per 1000 tokens using the most powerful AI models, the cost for a revision of a standard manuscript (around 35 paragraphs) should be around $0.50. The workflow allows specifying the branch and file names (in the `content/` directory) to revise, the language model to use, and the output branch name.