Support more browsers #340

jakethekoenig · 2023-12-01T11:35:00Z

The system default browser is detected and opened if it is of type firefox, safari or edge. Otherwise we fallback on chrome.

mentat/vision/vision_manager.py

granawkins · 2023-12-04T23:26:47Z

I got an error testing on Dec 5:

Unhandled Exception: Traceback (most recent call last):
  File "/Users/granthawkins/mentat/mentat/session.py", line 154, in run_main
    await self._main()
  File "/Users/granthawkins/mentat/mentat/session.py", line 115, in _main
    message = await collect_user_input()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/granthawkins/mentat/mentat/session_input.py", line 41, in collect_user_input
    await command.apply(*arguments[1:])
  File "/Users/granthawkins/mentat/mentat/commands.py", line 433, in apply
    image = vision_manager.screenshot(*args)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/granthawkins/mentat/mentat/vision/vision_manager.py", line 70, in screenshot
    self.open(path)
  File "/Users/granthawkins/mentat/mentat/vision/vision_manager.py", line 51, in open
    self._open_browser()
  File "/Users/granthawkins/mentat/mentat/vision/vision_manager.py", line 43, in _open_browser
    service = Service(ChromeDriverManager().install())
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/webdriver_manager/chrome.py", line 40, in install
    driver_path = self._get_driver_binary_path(self.driver)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/webdriver_manager/core/manager.py", line 40, in _get_driver_binary_path
    file = self._download_manager.download_file(driver.get_driver_download_url(os_type))
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/webdriver_manager/drivers/chrome.py", line 32, in get_driver_download_url
    driver_version_to_download = self.get_driver_version_to_download()
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/webdriver_manager/core/driver.py", line 48, in get_driver_version_to_download
    return self.get_latest_release_version()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/webdriver_manager/drivers/chrome.py", line 64, in get_latest_release_version
    determined_browser_version = ".".join(determined_browser_version.split(".")[:3])
                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'split'

jakethekoenig · 2023-12-05T14:45:15Z

It turns out there's no easy cross platform way to detect the users default browser. You actually can't uninstall safari on macs so I just use that on MacOs. On Windows I attempt to use edge then chrome then firefox and on Linux Chrome then firefox. Reporting an error if all fail.

PCSwingle · 2023-12-05T18:46:08Z

Hmm, that is annoying; why don't we just fall through all of the browsers regardless of os? So test Safari first, then if that errors try and open firefox, then chrome, etc. and not worry about the os at all.

mentat/vision/vision_manager.py

jakethekoenig · 2023-12-05T21:28:11Z

Hmm, that is annoying; why don't we just fall through all of the browsers regardless of os? So test Safari first, then if that errors try and open firefox, then chrome, etc. and not worry about the os at all.

Good idea, that is simpler.

granawkins · 2023-12-06T00:16:28Z

mentat/vision/vision_manager.py

-            self.driver = webdriver.Chrome(service=service)
+            try:
+                self.driver = webdriver.Safari()
+            except Exception:


Suggested change

except Exception:

except Exception as e:

if "remote automation" in str(e).lower():

stream.send(f"To use Safari, enable remote automation.", color="red")

The first time I tried /screenshot, it failed (didn't recognize Safari, no backups installed). I found I needed to:

In the Safari menu bar, click 'Safari' and then 'Preferences'.

Go to the 'Advanced' tab and check 'Show Develop menu in menu bar'.

Open the 'Develop' menu in the menu bar and choose 'Allow Remote Automation'.

..and then it would work.

Ah thanks. I think given that I'll try to open with chrome before telling users to enable remote automation. But if they have safari installed I'll print a different message informing them of that option.

PCSwingle · 2023-12-06T20:08:33Z

mentat/command/commands/screenshot.py

            stream.send(
-                'No browser open. Run "/screenshot path" with a url or local file',
-                color="red",
+                "Error taking screenshot. Please run with a valid url or local path.",


I would prefer if this except was inside vision_manager.screenshot; that way we don't have to rewrite this except if we decide to use .screenshot again somewhere.

PCSwingle · 2023-12-06T20:10:04Z

mentat/vision/vision_manager.py

+                            if safari_installed:
+                                ctx.stream.send(
+                                    "No suitable browser found. To use Safari, enable"
+                                    " remote automation. Alternatively install Chrome.",


Suggested change

" remote automation. Alternatively install Chrome.",

" remote automation.",

PCSwingle · 2023-12-06T20:10:45Z

mentat/vision/vision_manager.py

+                                    "No suitable browser found. Install Chrome or"
+                                    " Firefox.",


Suggested change

"No suitable browser found. Install Chrome or"

" Firefox.",

"No suitable browser found.",

PCSwingle

Left a couple of really small comments, but it looks good to me! I can't test this but it looks like @granawkins already tested this for Safari so I'm ok approving it.

Support more browsers

4d6194a

The system default browser is detected and opened if it is of type firefox, safari or edge. Otherwise we fallback on chrome.

jakethekoenig force-pushed the more-browsers branch from 3c02a98 to 4d6194a Compare December 1, 2023 11:40

test fix

e29ab87

PCSwingle reviewed Dec 1, 2023

View reviewed changes

mentat/vision/vision_manager.py Outdated Show resolved Hide resolved

Browser determined from OS

cb31ab3

PCSwingle reviewed Dec 5, 2023

View reviewed changes

mentat/vision/vision_manager.py Outdated Show resolved Hide resolved

jakethekoenig added 3 commits December 5, 2023 13:18

driver type change

8219f15

try/catch in plaform independent way

b5aa068

Merge branch 'main' into more-browsers

cce8f4b

jakethekoenig added 2 commits December 5, 2023 13:33

try/catch fix

47874e0

lint fix

e689ff3

granawkins reviewed Dec 6, 2023

View reviewed changes

Tell user to enable remote automation

7acae16

PCSwingle reviewed Dec 6, 2023

View reviewed changes

PCSwingle approved these changes Dec 6, 2023

View reviewed changes

Review feedback

ca501b8

jakethekoenig merged commit cf46c59 into main Dec 7, 2023
16 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support more browsers #340

Support more browsers #340

jakethekoenig commented Dec 1, 2023

granawkins commented Dec 4, 2023

jakethekoenig commented Dec 5, 2023

PCSwingle commented Dec 5, 2023

jakethekoenig commented Dec 5, 2023

granawkins Dec 6, 2023

jakethekoenig Dec 6, 2023

PCSwingle Dec 6, 2023

PCSwingle Dec 6, 2023

PCSwingle Dec 6, 2023

PCSwingle left a comment

-            except Exception:
+            except Exception as e:
+                if "remote automation" in str(e).lower():
+                    stream.send(f"To use Safari, enable remote automation.", color="red")

	" remote automation. Alternatively install Chrome.",
	" remote automation.",

	"No suitable browser found. Install Chrome or"
	" Firefox.",
	"No suitable browser found.",

Support more browsers #340

Support more browsers #340

Conversation

jakethekoenig commented Dec 1, 2023

granawkins commented Dec 4, 2023

jakethekoenig commented Dec 5, 2023

PCSwingle commented Dec 5, 2023

jakethekoenig commented Dec 5, 2023

granawkins Dec 6, 2023

Choose a reason for hiding this comment

jakethekoenig Dec 6, 2023

Choose a reason for hiding this comment

PCSwingle Dec 6, 2023

Choose a reason for hiding this comment

PCSwingle Dec 6, 2023

Choose a reason for hiding this comment

PCSwingle Dec 6, 2023

Choose a reason for hiding this comment

PCSwingle left a comment

Choose a reason for hiding this comment