Replace browsergym with browsergym-core to reduce dependencies

[neubig]

This PR replaces the dependency on browsergym with browsergym-core to reduce unneeded dependencies. The full browsergym package includes many additional dependencies that are not needed for our use case.

Changes:

• Changed dependency from browsergym to browsergym-core in pyproject.toml
• Updated poetry.lock to remove unneeded dependencies

All tests pass successfully, confirming that the core browsergym functionality works as expected.

---

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:355c208-nikolaik   --name openhands-app-355c208   docker.all-hands.dev/all-hands-ai/openhands:355c208

[neubig]

Tests are now failing due to a missing dependency on "joblib". I expect that this was because joblib was included by browsergym, but it not explicitly included in our pyproject.toml file. We should also include a relevant version of joblib in pyproject.toml to adjust for this.

[github-actions]

OpenHands started fixing the pr! You can monitor the progress here.

[openhands-agent]

New OpenHands update

[github-actions]

The workflow to fix this issue encountered an error. Please check the workflow logs for more information.

[enyst]

@neubig Sorry, we can continue to fix it here, but just for discussion, have you seen this?

• Switch dependency to browsergym-core #5242
• it passes CI as long as we completely disable the miniwob test (the one that's currently failing here too)

---

That PR is in the stage "thinking about" 😅 because:

• the envs can be part of the 'evaluation' poetry group, which is not normally run in CI
  • • I moved miniwob etc to 'evaluation' group on that branch
  • CI is or should be run with poetry install --without evaluation,llama-index
• but there's a browsing test with miniwob that will fail as long as they're not installed
• which I had some trouble disabling in a way that remains very visible (= not prone to be forgotten)
  • because, for example, Xingyao recently needed it, and used it, in the latest browser action PR
  • feat(agent): add BrowseURLAction to CodeAct (produce markdown from URL) #5285
  • I mean, totally removing or commenting it out seems the wrong thing to do
• this isn't a new problem, we have it already with llama-index, same thing, disabled in normal CI, should not be forgotten, needs a place to run and us keeping an eye on
  • with the test_memory.py, currently disabled in CI

So my current theory about how to fix this, is to move both the llama-index and evaluation groups tests to some other pipeline like we now have the integration tests one:

• Nightly Integration Tests Runs #5318 (comment)
• or the miniwob test belongs with an eval nightly / weekly?
• or a new category like it?

[neubig]

Hey @enyst , I indeed had not seen that. Apologies! I'll look through carefully when I have a moment.

[neubig]

Closing in favor of #5242