bug: xcode 15.0.1 - WDA is not reused instead it is rebuilded all the time

[RSM-SBA]

Do I have the most recent component updates?

• I use the most recent available driver/plugin and server versions

Is the component officially supported by the Appium team?

• I have verified the component repository is present under the Appium organization in GitHub

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

While starting the tests, the WDA is always rebuilded regadless there is a WDA on the simulator which was running before.

Launching WebDriverAgent on the device
2023-11-21 09:41:03:072 - [XCUITestDriver@c62f (f7563732)] Cleaning up the WebDriverAgent project after the module upgrade has happened (5.0.0 < 5.14.0)
2023-11-21 09:41:03:073 - [XCUITestDriver@c62f (f7563732)] Cleaning the project scheme 'WebDriverAgentLib' to make sure there are no leftovers from previous

Expected Behavior

The behavior while using xcode 14.2 was that once the simulator had a WDA installed and running, i could use it all day long without rebuilding it.

Minimal Reproducible Example

starting with useNewWDA=false

Environment

• Operating system: Sonoma 14.1.1
• Appium server version (output of appium --version): 2.2.2
• Appium driver(s) and their version(s): XCUITESTdriver 5.9.1
• Appium plugin(s) and their version(s): images 2.1.7
• Node.js version (output of node --version): 16.19.1
• npm version (output of npm --version): 8.19.3
• Last component(s) version which did not exhibit the problem:
• Platform and version under test: iOS 16.4 (with xcode 15 Simulator)
• Real device or emulator/simulator: Simulator

Link to Appium Logs

https://gist.github.com/RSM-SBA/b7ad356f2ad4dc232164191877fd8b19

Further Information

No response

[mykola-mokhnach]

2023-11-21 09:50:47:278 - [XCUITestDriver@e036 (9e4d3ea1)] Upgrade timestamp of the currently bundled WDA: 1700555974015
2023-11-21 09:50:47:278 - [XCUITestDriver@e036 (9e4d3ea1)] Upgrade timestamp of the WDA on the device: 1700555974015
2023-11-21 09:50:47:279 - [XCUITestDriver@e036 (9e4d3ea1)] Will reuse previously cached WDA instance at 'http://127.0.0.1:8100/' with 'com.facebook.WebDriverAgentRunner'. Set the wdaLocalPort capability to a value different from 8100 if this is an undesired behavior.

Are you sure the attached log is the right one?
There I can observe the running WDA instance was reused, but it has failed to create a session (possibly crashed)

[mykola-mokhnach]

2023-11-21 09:41:03:072 - [XCUITestDriver@c62f (f7563732)] Cleaning up the WebDriverAgent project after the module upgrade has happened (5.0.0 < 5.14.0)
2023-11-21 09:41:03:073 - [XCUITestDriver@c62f (f7563732)] Cleaning the project scheme 'WebDriverAgentLib' to make sure there are no leftovers from previous

Also the above flow should only be executed after the driver upgrade. The driver persists the actual WDA version every time a new version of it is installed, so after upgrade it compares the actual version with the persisted one and decides whether project cleanup is necessary

[RSM-SBA]

yes its the correct one
the building of the WDA is working and afterwards the tests are running, but when restarting a new session which should reuse the old WDA the above log occurs and rebuilding starts.

I had completly uninstalled appium and the appium_home components.
Also the simulator was fresh.
Any ideas where the persists of the upgrade can fail ?

[mykola-mokhnach]

but when restarting a new session which should reuse the old WDA the above log occurs and rebuilding starts

By default the driver always rebuilds WDA after upgrade. The only way to prevent that is to provide the appium:webDriverAgentUrl capability, which I wouldn't recommend as it may lead to unexpected issues, because the driver always expects the correct linked WDA version to run on the device.

[RSM-SBA]

But i thought the upgrade does only occurs once and not every testrun.
I still don't understand why this upgrade occurs each start

[RSM-SBA]

if i call the /status after a test is shutdown, the WDA is still running

{
"value" : {
"build" : {
"upgradedAt" : "1700555974015",
"time" : "Nov 21 2023 10:46:56",
"productBundleIdentifier" : "com.facebook.WebDriverAgentRunner"
},
"os" : {
"testmanagerdVersion" : 65535,
"name" : "iOS",
"sdkVersion" : "17.0",
"version" : "16.4"
},
"device" : "iphone",
"ios" : {
"simulatorVersion" : "16.4",
"ip" : "192.168.178.22"
},
"message" : "WebDriverAgent is ready to accept commands",
"state" : "success",
"ready" : true
},
"sessionId" : null
}

[mykola-mokhnach]

Unfortunately I cannot see it occurs every start from the above log

[mykola-mokhnach]

What I can see is the cached build is used then session creation fails because of some reason (maybe visible in device logs) and then it tries to reinit xcodebuild

[RSM-SBA]

ok tried it with the iOS deviceLogs
When i see it correctly from line 5718 the WDA was connected and starting the app (the app is starting on the sim) but afterwards it looses the connection to WDA

https://gist.github.com/RSM-SBA/064e12a4f3688857f222d6093326813b

[mykola-mokhnach]

afterwards it looses the connection to WDA

Exactly. Unfortunately I cannot find any traces of a possible reason in the logs. To me it looks like the app under test never returns the control to WDA. Are you able to start the session without providing the app capability, e.g. just start from Home screen?

[RSM-SBA]

Seems that the app under test is not the problem.
have removed the appId and BundleId from capabilities and installed the app afterwards with mobile:installApp

When the WDA is re-builded the test runs.
If the following test wants to re-use the WDA (which is still available via /status) the initialising of the driver works and than the connection is closed anyhow directly afterwards. In the following xcodeLog there was no starting of the testapp, but still same behavior...

https://gist.github.com/RSM-SBA/64bc6388e5d15d76b5ef4e3a02ff2afe

[mykola-mokhnach]

Does it change anything if you also remove these both capabilities? :

2023-11-21 15:26:57:527 [XCUITestDriver@07da]     "appium:language": "de",
2023-11-21 15:26:57:527 [XCUITestDriver@07da]     "appium:locale": "de_DE",

[RSM-SBA]

will try it tomorrow and respond here, thank you so far for your input :)

[RSM-SBA]

@mykola-mokhnach ok removing the locale "fixed" the problem...
Strange that it is only a problem when reusing the WDA and not all the time.

[mykola-mokhnach]

@mykola-mokhnach ok removing the locale "fixed" the problem... Strange that it is only a problem when reusing the WDA and not all the time.

Then I probably know what the issue might be: appium/appium-ios-simulator#384

Restarting Springboard also kills WDA as a side effect. cc @mwakizaka

[mwakizaka]

Restarting Springboard also kills WDA as a side effect. cc @mwakizaka

I'm sorry for the issue. I'm afraid but no have good ideas. Should not we restart Springboard when a WDA session already exists?
Although we may try rebooting WDA session in such situation, I guess it will break the motivation (reuse a WDA session as far as possible in order to start a test run quickly)

[RSM-SBA]

I'm sorry for the issue. I'm afraid but no have good ideas. Should not we restart Springboard when a WDA session already exists? Although we may try rebooting WDA session in such situation, I guess it will break the motivation (reuse a WDA session as far as possible in order to start a test run quickly)

But would'nt rebooting WDA much faster than completly rebuilding it (as it is now) ?
At least a workaround if restarting Springboard is necessary

[mykola-mokhnach]

@RSM-SBA I have created the above PR to avoid services restart if the Simulator language is already set to the desired value. This should prevent WDA restart for the following sessions.

[mykola-mokhnach]

Restarting Springboard also kills WDA as a side effect. cc @mwakizaka

I'm sorry for the issue. I'm afraid but no have good ideas. Should not we restart Springboard when a WDA session already exists? Although we may try rebooting WDA session in such situation, I guess it will break the motivation (reuse a WDA session as far as possible in order to start a test run quickly)

We may make this behaviour configurable (e.g. add an additional argument to either allow or deny services restart after defaults are changed). So people have a choice. Also, calling this method before the session start is not as bad, because it retries, although calling it mid-session, using https://github.com/appium/appium-xcuitest-driver/blob/master/docs/execute-methods.md#mobile-configurelocalization endpoint, would probably have sad consequences with the current implementation.

[mwakizaka]

Restarting Springboard also kills WDA as a side effect. cc @mwakizaka

I'm sorry for the issue. I'm afraid but no have good ideas. Should not we restart Springboard when a WDA session already exists? Although we may try rebooting WDA session in such situation, I guess it will break the motivation (reuse a WDA session as far as possible in order to start a test run quickly)

We may make this behaviour configurable (e.g. add an additional argument to either allow or deny services restart after defaults are changed). So people have a choice. Also, calling this method before the session start is not as bad, because it retries, although calling it mid-session, using https://github.com/appium/appium-xcuitest-driver/blob/master/docs/execute-methods.md#mobile-configurelocalization endpoint, would probably have sad consequences with the current implementation.

It sounds good. Let me work in on next week if no problem.

[RSM-SBA]

@mwakizaka Thank you very much to fix it so quickly!
How does this switch now work?
Can I set it always to true and than the SpringBoard is not restarted when the locale has not changed ?

[mykola-mokhnach]

With the current fix WDA should only restart once between locale changes. The process must not be restarted If the locale remains unchanged between sessions.

[RSM-SBA]

the fix is working, thank you both for your quick response and fixing it