Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quit timeout #599

Closed
dgr opened this issue Jul 9, 2024 · 10 comments
Closed

Quit timeout #599

dgr opened this issue Jul 9, 2024 · 10 comments

Comments

@dgr
Copy link
Contributor

dgr commented Jul 9, 2024

Version
1.0.40

Platform
Operating System: MacOS 14.5 (Sonoma)
Clojure version: 1.11.1
Babashka version: 1.3.191
JDK vendor and version: Azul Zulu 17.0.11,17.50.19

Browser vendor: firefox
Browser version: 125.0.1
WebDriver version: Geckodriver 0.34.0, installed via brew

Symptom
Invoking etaoin/quit or etaoin/delete-session results in an exception being thrown indicating a timeout. Note that the browser window is destroyed prior to the timeout but the firefox process remains running.

Reproduction
This is the simplest reproduction.

> lein try etaoin 1.0.40
Retrieving babashka/fs/0.1.6/fs-0.1.6.pom from clojars
nREPL server started on port 53242 on host 127.0.0.1 - nrepl://127.0.0.1:53242
REPL-y 0.5.1, nREPL 1.0.0
Clojure 1.11.1
OpenJDK 64-Bit Server VM 17.0.11+9-LTS
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/firefox))
#'user/driver
user=> (e/quit driver)
Execution error (SocketTimeoutException) at sun.nio.ch.NioSocketImpl/timedRead (NioSocketImpl.java:288).
Read timed out

Actual behavior
Timeout occurs.

Expected behavior
quit/delete-session should quit cleanly without a timeout.

Diagnosis
No.

Action
Unsure. I'll try to gather more info.

@lread
Copy link
Collaborator

lread commented Jul 9, 2024

Thanks @dgr, this might be related/same/similar to #517?

@lread
Copy link
Collaborator

lread commented Jul 9, 2024

I vaguely remember that I might have some work-in-progress that could address this... I'll take a peek sometime soon.

@lread
Copy link
Collaborator

lread commented Jul 9, 2024

Oh hold on, sorry, I think I misread your issue entirely.
I am not familiar with lein try.

Here's me doing what you did but with clojure cli (from a new empty dir)

$ clojure -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.40"}}}'                                                       
Clojure 1.11.3
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/firefox))
#'user/driver
user=> (e/quit driver)
{:args ("geckodriver" "--port" 38433), :capabilities {:loggingPrefs {:browser "ALL"}}, :process {:proc #object[java.lang.ProcessImpl 0x13cc3984 "Process[pid=160086, exitValue=143]"], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x55465bfc "java.lang.ProcessImpl$ProcessPipeOutputStream@55465bfc"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x5625ba2 "java.lang.ProcessBuilder$NullInputStream@5625ba2"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x5625ba2 "java.lang.ProcessBuilder$NullInputStream@5625ba2"], :prev nil, :cmd ["geckodriver" "--port" "38433"]}, :locator "xpath", :type :firefox, :port 38433, :host "127.0.0.1", :url "http://127.0.0.1:38433"}
user=> 

I'm not seeing the timeout issue here, can you double-check versions of things you are running?
If you clone etaoin you can type bb tools-versions from its project root dir, then you could paste the result in a comment.

@lread
Copy link
Collaborator

lread commented Jul 9, 2024

Just for comparison, here's what I am running:

$ bb tools-versions

[ TASK tools-versions  ]---------------------------------------------------------

Linux
  version: 6.9.3-76060903-generic
  arch: amd64
Java 21.0.3 - /home/lee/.sdkman/candidates/java/current/bin/java
  openjdk version "21.0.3" 2024-04-16 LTS
  OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS)
  OpenJDK 64-Bit Server VM Temurin-21.0.3+9 (build 21.0.3+9-LTS, mixed mode, sharing)
Babashka 1.3.191 - /home/linuxbrew/.linuxbrew/bin/bb
  babashka v1.3.191
Image Magick 6.9.11 - /usr/bin/identify
  Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org
Chrome 126.0.6478.126 - /usr/bin/google-chrome
  Google Chrome 126.0.6478.126
Chrome Webdriver 126.0.6478.62 - /home/lee/bin/chromedriver
  ChromeDriver 126.0.6478.62 (b2781ca4c383af675f7a25c7b03b4a54e25e559a-refs/branch-heads/6478_56@{#4})
Firefox 127.0 - /usr/bin/firefox
  Mozilla Firefox 127.0
Firefox Webdriver 0.34.0 - /home/lee/bin/geckodriver
  geckodriver 0.34.0 (c44f0d09630a 2024-01-02 15:36 +0000)

Warnings
- Version mismatch: Chrome 126.0.6478.126 != Chrome Webdriver 126.0.6478.62

TASK tools-versions done.

@dgr
Copy link
Contributor Author

dgr commented Jul 9, 2024

Thanks for the quick replies. Been stuck without Internet most of today, right after I filed the issue. Yea, lein try is a simple way to get a particular library loaded quickly and then dump into a REPL. I'm more used to it than using Clojure CLI, but yes, you can do the same thing. I originally found this in another program I was writing. I used lein try because I wanted to make sure that I wasn't picking up a strange library interaction in my program. To be clear, I've used Etaoin a few times in the past and never seen this issue. I have seen hiccups in the past, but typically it was a versioning dependency between gecko driver and Etaoin and upgrading one or the other to the latest version cleared things up. I'll try with Clojure CLI tooling and see if I can reproduce it that way, too. Finally, you asked about whether this is related to #517? I looked at that issue before I filed this one and it doesn't sound the same symptom-wise, but I could be wrong. Let me try with Clojure CLI.

@lread
Copy link
Collaborator

lread commented Jul 9, 2024

Yeah, I was out to lunch thinking this was related to #517. 🙂

Looking forward to learning more about your timeout issue.

@dgr
Copy link
Contributor Author

dgr commented Jul 10, 2024

OK, some more data.

Firstly, using Clojure CLI had no effect on the outcome. Same result.

$ clojure -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.40"}}}' 
Clojure 1.11.3
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/firefox))
#'user/driver
user=> (e/quit driver)
Execution error (SocketTimeoutException) at sun.nio.ch.NioSocketImpl/timedRead (NioSocketImpl.java:288).
Read timed out
user=> 

And bb tools-versions reports:

$ bb tools-versions

[ TASK tools-versions  ]---------------------------------------------------------

Mac OS X
  version: 14.5
  arch: aarch64
Java 17.0.11 - /usr/bin/java
  openjdk version "17.0.11" 2024-04-16 LTS
  OpenJDK Runtime Environment Zulu17.50+19-CA (build 17.0.11+9-LTS)
  OpenJDK 64-Bit Server VM Zulu17.50+19-CA (build 17.0.11+9-LTS, mixed mode, sharing)
Babashka 1.3.191 - /opt/homebrew/bin/bb
  babashka v1.3.191
Image Magick - Error
  bin not found: magick
Chrome - Error
  mac app not found: Google Chrome
Chrome Webdriver - Error
  bin not found: chromedriver
Firefox 127.0.2 - /Applications/Firefox.app
  127.0.2
Firefox Webdriver 0.34.0 - /opt/homebrew/bin/geckodriver
  geckodriver 0.34.0
Edge - Error
  mac app not found: Microsoft Edge
Edge Webdriver - Error
  bin not found: msedgedriver
Safari 17.5 - /Applications/Safari.app
  17.5
Safari Webdriver 17.5 - /System/Cryptexes/App/usr/bin/safaridriver
  Included with Safari 17.5 (19618.2.12.11.6)

TASK tools-versions done.

Finally, I also tried it from Clojure CLI with Safari. This worked. So, there's something at issue with Firefox/Geckodriver. Perhaps it's not an Etaoin bug at all, but a bug there?

user=> (def driver (e/safari))
#'user/driver
user=> (e/quit driver)
{:args ("safaridriver" "--port" 50800), :capabilities {:loggingPrefs {:browser "ALL"}}, :process {:proc #object[java.lang.ProcessImpl 0x6a1b4854 "Process[pid=6186, exitValue=0]"], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x6a175162 "java.lang.ProcessImpl$ProcessPipeOutputStream@6a175162"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x25814d3c "java.lang.ProcessBuilder$NullInputStream@25814d3c"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x25814d3c "java.lang.ProcessBuilder$NullInputStream@25814d3c"], :prev nil, :cmd ["safaridriver" "--port" "50800"]}, :locator "xpath", :type :safari, :port 50800, :host "127.0.0.1", :url "http://127.0.0.1:50800"}
user=> 

I'm not sure what to do next. Let me know if you need more info and I'll try to get it for you.

@lread
Copy link
Collaborator

lread commented Jul 10, 2024

Hmm... here's a try from an M1 MacBook:

First tools-versions:

~ bb tools-versions

[ TASK tools-versions  ]---------------------------------------------------------

Mac OS X
  version: 14.5
  arch: aarch64
Java 21.0.2 - /Users/lee/.sdkman/candidates/java/current/bin/java
  openjdk version "21.0.2" 2024-01-16 LTS
  OpenJDK Runtime Environment Temurin-21.0.2+13 (build 21.0.2+13-LTS)
  OpenJDK 64-Bit Server VM Temurin-21.0.2+13 (build 21.0.2+13-LTS, mixed mode)
Babashka 1.3.191 - /opt/homebrew/bin/bb
  babashka v1.3.191
Image Magick 7.1.1 - /opt/homebrew/bin/magick
  Version: ImageMagick 7.1.1-34 Q16-HDRI aarch64 22301 https://imagemagick.org
Chrome 126.0.6478.127 - /Applications/Google Chrome.app
  126.0.6478.127
Chrome Webdriver 126.0.6478.126 - /Users/lee/bin/chromedriver
  ChromeDriver 126.0.6478.126 (d36ace6122e0a59570e258d82441395206d60e1c-refs/branch-heads/6478@{#1591})
Firefox 128.0 - /Applications/Firefox.app
  128.0
Firefox Webdriver 0.34.0 - /opt/homebrew/bin/geckodriver
  geckodriver 0.34.0
Edge 126.0.2592.87 - /Applications/Microsoft Edge.app
  126.0.2592.87
Edge Webdriver 126.0.2592.87 - /Users/lee/bin/msedgedriver
  Microsoft Edge WebDriver 126.0.2592.87 (883b9421f9ba9ce195934ad8d16cb5215b659245)
Safari 17.5 - /Applications/Safari.app
  17.5
Safari Webdriver 17.5 - /System/Cryptexes/App/usr/bin/safaridriver
  Included with Safari 17.5 (19618.2.12.11.6)

Warnings
- Version mismatch: Chrome 126.0.6478.127 != Chrome Webdriver 126.0.6478.126

TASK tools-versions done.

And now let's try sending output to the console via {:log-stdout :inherit :log-stderr :inherit} opts to see what we see.

~ clj -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.40"}}}'
Clojure 1.11.3
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/firefox {:log-stdout :inherit :log-stderr :inherit}))
1720631896039	geckodriver	INFO	Listening on 127.0.0.1:59023
1720631896369	webdriver::command	WARN	You are using deprecated legacy session negotiation patterns (desiredCapabilities/requiredCapabilities), see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities#Legacy
1720631896375	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" MOZ_NO_REMOTE="1" "/App ... "--marionette" "-foreground" "-no-remote" "-profile" "/var/folders/9h/kqq1qz3d2s76css3qn9yyxw00000gp/T/rust_mozprofileUGOWc2"
console.warn: services.settings: Ignoring preference override of remote settings server
console.warn: services.settings: Allow by setting MOZ_REMOTE_SETTINGS_DEVTOOLS=1 in the environment
1720631896608	Marionette	INFO	Marionette enabled
1720631896872	Marionette	INFO	Listening on port 59029
UNSUPPORTED (log once): POSSIBLE ISSUE: unit 1 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable
#'user/driver
user=> (e/quit driver)
1720631909618	Marionette	INFO	Stopped listening on port 59029
{:args ("geckodriver" "--port" 59023), :capabilities {:loggingPrefs {:browser "ALL"}}, :process {:proc #object[java.lang.ProcessImpl 0x6b7ebac1 "Process[pid=8426, exitValue=143]"], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x39dee5fd "java.lang.ProcessImpl$ProcessPipeOutputStream@39dee5fd"], :out #object[java.lang.ProcessBuilder$NullInputStream 0x3e8eea59 "java.lang.ProcessBuilder$NullInputStream@3e8eea59"], :err #object[java.lang.ProcessBuilder$NullInputStream 0x3e8eea59 "java.lang.ProcessBuilder$NullInputStream@3e8eea59"], :prev nil, :cmd ["geckodriver" "--port" "59023"]}, :locator "xpath", :type :firefox, :port 59023, :host "127.0.0.1", :url "http://127.0.0.1:59023"}

If you do the above, do you see anything very different?

Also you might check your geckodriver architecture via:

~ (file $(which geckodriver))
/opt/homebrew/bin/geckodriver: Mach-O 64-bit executable arm64

Maybe there is something wrong with your geckodriver, I suppose you could try to reinstall it:

~  brew reinstall geckodriver

@dgr
Copy link
Contributor Author

dgr commented Jul 10, 2024

Hm... things are getting odd. I tried doing exactly what you did above, and this time it worked.

clj -Sdeps '{:deps {etaoin/etaoin {:mvn/version "1.0.40"}}}'
Clojure 1.11.3
user=> (require '[etaoin.api :as e])
nil
user=> (def driver (e/firefox {:log-stdout :inherit :log-stderr :inherit}))
12:56:18.730 [main] DEBUG etaoin.api -- Created driver: firefox 127.0.0.1:51746
12:56:18.733 [main] DEBUG etaoin.api -- Starting process: geckodriver --port 51746
12:56:18.743 [main] DEBUG etaoin.api -- Waiting until 127.0.0.1:51746 is running
1720634178851	geckodriver	INFO	Listening on 127.0.0.1:51746
12:56:19.079 [main] DEBUG etaoin.api -- Waiting until 127.0.0.1:51746 is running
12:56:19.083 [main] DEBUG etaoin.impl.client -- firefox 127.0.0.1:51746   POST session {:desiredCapabilities {:loggingPrefs {:browser "ALL"}}}
1720634179207	webdriver::command	WARN	You are using deprecated legacy session negotiation patterns (desiredCapabilities/requiredCapabilities), see https://developer.mozilla.org/en-US/docs/Web/WebDriver/Capabilities#Legacy
1720634179211	mozrunner::runner	INFO	Running command: MOZ_CRASHREPORTER="1" MOZ_CRASHREPORTER_NO_REPORT="1" MOZ_CRASHREPORTER_SHUTDOWN="1" MOZ_NO_REMOTE="1" "/App ... "--marionette" "-foreground" "-no-remote" "-profile" "/var/folders/wb/ffzkcshd39l9zmbf5092fg9w0000gp/T/rust_mozprofile8YEihD"
console.warn: services.settings: Ignoring preference override of remote settings server
console.warn: services.settings: Allow by setting MOZ_REMOTE_SETTINGS_DEVTOOLS=1 in the environment
1720634183066	Marionette	INFO	Marionette enabled
1720634183369	Marionette	INFO	Listening on port 51752
UNSUPPORTED (log once): POSSIBLE ISSUE: unit 1 GLD_TEXTURE_INDEX_2D is unloadable and bound to sampler type (Float) - using zero texture because texture unloadable
#'user/driver
user=> 1720634214014	addons.xpi	ERROR	System addon update list error SyntaxError: XMLHttpRequest.open: 'http://%(server)s/dummy-system-addons.xml' is not a valid URL.
(e/quit driver)
12:57:11.984 [main] DEBUG etaoin.impl.client -- firefox 127.0.0.1:51746 DELETE session/704e285d-2b1b-4fda-aeee-cb8e45e79189 
1720634231988	Marionette	INFO	Stopped listening on port 51752
{:args ("geckodriver" "--port" 51746), :capabilities {:loggingPrefs {:browser "ALL"}}, :process {:proc #object[java.lang.ProcessImpl 0x1d12eff1 "Process[pid=1729, exitValue=143]"], :exit nil, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x206a465f "java.lang.ProcessImpl$ProcessPipeOutputStream@206a465f"], :out #object[java.lang.ProcessBuilder$NullInputStream 0xc509742 "java.lang.ProcessBuilder$NullInputStream@c509742"], :err #object[java.lang.ProcessBuilder$NullInputStream 0xc509742 "java.lang.ProcessBuilder$NullInputStream@c509742"], :prev nil, :cmd ["geckodriver" "--port" "51746"]}, :locator "xpath", :type :firefox, :port 51746, :host "127.0.0.1", :url "http://127.0.0.1:51746"}
user=> 

No hang or timeout exception. Weird.

For completeness:

 $ (file $(which geckodriver))
/opt/homebrew/bin/geckodriver: Mach-O 64-bit executable arm64

So, I guess it fixed itself, maybe? I did have to reboot my machine this morning. The only thing I can think of is that something in MacOS bork'd Firefox (?!). Maybe the reboot unbork'd it? I ran bb tools-versions again and all the output is the same as before, so if something updated, it was on the MacOS side of things, not Firefox or Geckodriver.

Given that it works, let's close this. I'll reopen it if it happens again and try to gather more data. Sorry for the trouble.

@lread
Copy link
Collaborator

lread commented Jul 10, 2024

Sorry for the trouble.

No trouble at all! Thanks for reaching out, your issue just might help some other person who happens to hit the same symptom. Best guess at resolution: reboot your mac!

@lread lread closed this as not planned Won't fix, can't repro, duplicate, stale Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants