update_file method to change occurences of text in a file

[bfalans]

Background

Auto-gpt is not able to edit files, the addtion of an update_file method would tackle this issue
old PR: #3464

Changes

addition of an update_file method that can change one or all occurences of specified text

Documentation

Inline documentation provided for the function

Test Plan

2 tests written in unittest Test_file_operations.py and verified

PR Quality Checklist

• My pull request is atomic and focuses on a single change.
• I have thoroughly tested my changes with multiple different prompts.
• I have considered potential risks and mitigations for my changes.
• I have documented my changes clearly and comprehensively.
• I have not snuck in any "extra" small tweaks changes

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
docs	⬜️ Ignored (Inspect)	Visit Preview		Jun 3, 2023 10:34pm

[Boostrix]

I am all in when it comes to formalizing these things, but I have seen it use CLI editors like sed/awk and perl to edit existing files in a rather clever way, at least on *nix. So this may boil down to the OS in use. Note that one recurring "issue" reported frequently on the issue tracker is that Auto-GPT starts interactive editors like vim/nano or joe to edit stuff - so it's definitely trying to do the right thing. Merely failing at NOT using interactive apps, and failing at knowing more about the OS in question, and the tooling/version numbers available.

On a Linux system, if augment the shell_execute description to add a corresponding constraint, you will definitely see much better results.

Then again, like I said, using a dedicated update_file/edit_file command sounds definitely like a good idea, and it would also be the right place to encode preference for CLI tools like sed/awk there.

addition of an update_file method that can change all occurences of specified text

not having looked at the code/patch in question, I find that rather limited if that's indeed what it is doing.
Like I said, for people on Linux, Auto-GPT can definitely be told to update/edit files using standard CLI toolks like sed/vi or even perl (non-interactively). Thus, if you need just a search/replace command, my suggestion would be to use a regex wrapper for that - that way it's broader in scope, and the backend could still be using sed on these systems.

[bfalans]

@Boostrix, I found that on a windows system, autogpt is not able to use any programs the right way afaik. Yes it is a basic function, but we need to start somewhere and keep updates atomized.

[Boostrix]

For cross-platform file editing functionality similar to sed and awk, you can use Python's built-in string manipulation and regular expression modules: re, along with its file I/O functions.

[bfalans]

@Boostrix would you care to help me adjust it? i am not that known to all the python packages yet and am learning a lot from help provided.

[Boostrix]

assuming that you have access to GPT, you can probably just copy your changes and ask it to adapt those to use the Python "re" module instead - it's a fairly self-contained change since your code is already using the re module.

[bfalans]

It is already using re, so I will try to adjust it. I have access to gpt but only 3.5

[bfalans]

@Boostrix I tried chat gpt, but it comes up with the same code constantly only changing re.finditer to re.sub. I don't think this is what you meant 😅.

[bfalans]

function changed to use re for cross_platform compatibilty
Function now changes one or all occurences of specified text

[Boostrix]

function changed to use re for cross_platform compatibilty Function now changes one or all occurences of specified text

my 2c: I still believe having a lower-level function to run a regex on an input/file analogous to sed/awk would be more powerful and more general, higher level functions like "replace_text" could be built on top of this, simply by invoking the regex command. The "workhorse" portion of this code should not be replacing a text in a file, but it should be running a regex - so that other, higher level, command would simply specify the regex/file name to be used when invoking this.

Don't get me wrong, this is great and useful "as is" - but it could be MUCH more useful by using the regex/file approach - also to edit stuff in place.

[bfalans]

@Boostrix I will take a look, but could use some help as my programming skills are not up-to-date yet

[Boostrix]

@Boostrix I will take a look, but could use stem help as my programming skiles are not up-to-date yet

I would not expend any time/effort on this, unless others agree with my opinion.
But the idea itself is rather simple: rather than run a search/replace, just run the regex.

The search/replace would then be built on top of your regex command, which would then be much simpler/shorter

[bfalans]

@Boostrix great new coding experience for me to look into that, but for now I hope someone can review this so it can be implemented or so I know what to change. I can change it when I have new code ready

[Pwuts]

The code mostly looks fine. But the question remains: does this work in practice? Do you have a demo/log/example showing that the bot can use this command effectively, and that it adds significant value over the existing available file operations?

[bfalans]

Executing file '/app/autogpt/auto_gpt_workspace/retrieve_ticker_data.py'
SYSTEM: Command execute_python_file returned: Error: File "/app/autogpt/auto_gpt_workspace/retrieve_ticker_data.py", line 13 print(btc_usd_ticker)import collections.abc ^^^^^^ SyntaxError: invalid syntax
THOUGHTS: There is another syntax error in the Python script. I will use the 'update_file' command to fix the error by
adding a newline character between the 'print' statement and the 'import' statement.
REASONING: I will use the 'update_file' command to fix the error by adding a newline character between the 'print' statement and the 'import' statement.
PLAN:

• Use the 'update_file' command to fix the error by adding a newline character between the 'print' statement and the 'import' statement.
  CRITICISM: I need to ensure that the error is fixed and that the Python script executes successfully.
  NEXT ACTION: COMMAND = update_file ARGUMENTS = {'filename': '/app/autogpt/auto_gpt_workspace/retrieve_ticker_data.py', 'old_text': 'print(btc_usd_ticker)import collections.abc', 'new_text': 'print(btc_usd_ticker)\nimport collections.abc', 'occurrence_index': 0}
  Enter 'y' to authorise command, 'y -N' to run N continuous commands, 's' to run self-feedback commands'n' to exit program, or enter feedback for ...
  Asking user via keyboard...
  Input:do not use cbpro but an alternative
  SYSTEM: Human feedback: do not use cbpro but an alternative
  THOUGHTS: I have received feedback to use an alternative to the 'cbpro' library. I will use the 'update_file' command to replace the 'cbpro' library with the 'ccxt' library, which is a popular alternative for cryptocurrency trading.
  REASONING: I will use the 'update_file' command to replace the 'cbpro' library with the 'ccxt' library, which is a popular alternative for cryptocurrency trading.
  PLAN:
• Use the 'update_file' command to replace the 'cbpro' library with the 'ccxt' library, which is a popular alternative for cryptocurrency trading.
  CRITICISM: I need to ensure that the 'ccxt' library is installed and that the Python script executes successfully.
  NEXT ACTION: COMMAND = update_file ARGUMENTS = {'filename': '/app/autogpt/auto_gpt_workspace/retrieve_ticker_data.py', 'old_text': 'import cbpro', 'new_text': "import ccxt\n\n# Connect to the Coinbase API\npublic_client = ccxt.coinbasepro()\n\n# Retrieve the latest ticker data for the cryptocurrency pair(s) you wish to trade\nbtc_usd_ticker = public_client.fetch_ticker('BTC/USD')\n\n# Print the latest ticker data\nprint(btc_usd_ticker)", 'occurrence_index': 0}
  Enter 'y' to authorise command, 'y -N' to run N continuous commands, 's' to run self-feedback commands'n' to exit program, or enter feedback for ...
  Asking user via keyboard...
  Input:y
  -=-=-=-=-=-=-= COMMAND AUTHORISED BY USER -=-=-=-=-=-=-=
  SYSTEM: Command update_file returned: File /app/autogpt/auto_gpt_workspace/retrieve_ticker_data.py updated successfully

as an example that is running as we speak

[github-actions]

Conflicts have been resolved! 🎉 A maintainer will review the pull request shortly.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[github-actions]

This pull request has conflicts with the base branch, please resolve those so we can evaluate the pull request.

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel]

Deployment failed with the following error:

Resource is limited - try again in 3 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel]

Deployment failed with the following error:

Resource is limited - try again in 2 minutes (more than 100, code: "api-deployments-free-per-day").

[github-actions]

This PR exceeds the recommended size of 200 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size

[vercel]

Deployment failed with the following error:

Resource is limited - try again in 58 seconds (more than 100, code: "api-deployments-free-per-day").