feat: uninstall the test app once and install that again when MismatchedApplicationIdentifierEntitlement installation error occurs

docs/capabilities.md

@@ -12,7 +12,7 @@ Capability           &nbs
 `appium:platformVersion` | The platform version of an emulator or a real device. This capability is used for device autodetection if `udid` is not provided
 `appium:udid` | UDID of the device to be tested. Could be retrieved from Xcode->Window->Devices and Simulators window. Always set this capability if you run parallel tests or use a real device to run your tests.
 `appium:noReset` | Prevents the device to be reset before the session startup if set to `true`. This means that the application under test is not going to be terminated neither its data cleaned. `false` by default
-`appium:fullReset` | Being set to `true` always enforces the application under test to be fully uninstalled before starting a new session. `false` by default
+`appium:fullReset` | Being set to `true` always enforces the application under test to be fully uninstalled before starting a new session. The application data might be cached on real devices under particular circumstances. Please check [troubleshooting](troubleshooting.md#clear-the-application-local-data-explicitly-for-real-devices) for more details regarding obsolete application data cleanup on real devices. `false` by default
 `appium:printPageSourceOnFindFailure` | Enforces the server to dump the actual XML page source into the log if any error happens. `false` by default.
 `browserName` | The name of the browser to run the test on. If this capability is provided then the driver will try to start the test in Web context mode (Native mode is applied by default). Read [Automating hybrid apps](https://appium.io/docs/en/writing-running-appium/web/hybrid/) for more details. Usually equals to `safari`.
 `appium:includeDeviceCapsToSessionInfo` | Whether to include screen information as the result of [Get Session Capabilities](http://appium.io/docs/en/commands/session/get/). It includes `pixelRatio`, `statBarHeight` and `viewportRect`, but it causes an extra API call to WDA which may increase the response time like [this issue](https://github.com/appium/appium/issues/15101). Defaults to `true`. **This capability has no effect since driver version 5**
@@ -25,7 +25,7 @@ Capability | Description
 --- | ---
 `appium:bundleId` | Bundle identifier of the app under test, for example `com.mycompany.myapp`. The capability value is calculated automatically if `app` is provided. If neither `app` or `bundleId` capability is provided then XCUITest driver starts from the Home screen.
 `appium:app` | Full path to the application to be tested (the app must be located on the same machine where the server is running). `.ipa` and `.app` application extensions are supported. Zipped `.app` bundles are supported as well. Could also be an URL to a remote location. If neither of the `app` or `bundleId` capabilities are provided then the driver starts from the Home screen and expects the test to know what to do next. Do not provide both `app` and `browserName` capabilities at once.
-`appium:enforceAppInstall` | If set to `false` it will make xcuitest driver to verify whether the app version currently installed on the device under test is older than the one, which is provided as `appium:app` value. No app reinstall is going to happen if the candidate app has the same or older version number than the already installed copy of it. The version number used for comparison must be provided as [CFBundleVersion](https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleversion) [Semantic Versioning](https://semver.org/)-compatible value in the application's Info.plist. No validation is performed by default, e.g. the provided app is always (re)installed, which could potentially slow down your test suites. Available since XCUITest driver 4.19.0. | false
+`appium:enforceAppInstall` | If set to `false` it will make xcuitest driver to verify whether the app version currently installed on the device under test is older than the one, which is provided as `appium:app` value. No app reinstall is going to happen if the candidate app has the same or older version number than the already installed copy of it. The version number used for comparison must be provided as [CFBundleVersion](https://developer.apple.com/documentation/bundleresources/information_property_list/cfbundleversion) [Semantic Versioning](https://semver.org/)-compatible value in the application's Info.plist. No validation is performed by default, e.g. the provided app is always (re)installed, which could potentially slow down your test suites. The application data might be cached on real devices under particular circumstances when `appium:enforceAppInstall` is `true` if the application under test remained on the device under a certain situation. Please check [troubleshooting](troubleshooting.md#clear-the-application-local-data-explicitly-for-real-devices) for more details regarding obsolete application data cleanup on real devices. Available since XCUITest driver 4.19.0. | false
 `appium:localizableStringsDir` | Where to look for localizable strings in the application bundle. Defaults to `en.lproj`
 `appium:otherApps` | App or list of apps (as a JSON array) to install prior to running tests. For example: `["http://appium.github.io/appium/assets/TestApp9.4.app.zip", "/path/to/app-b.app"]`
 `appium:language` | Language to set for iOS app, for example `fr`. Please read [Language IDs](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPInternational/LanguageandLocaleIDs/LanguageandLocaleIDs.html) to get more details about available values for this capability. If a test is executed on a Simulator then UI language is changed as well. You can also change Simulator language in runtime using [mobile: configureLocalization](#mobile-configurelocalization) extension.

docs/execute-methods.md

@@ -108,6 +108,7 @@ strategy | string | no | One of possible app installation strategies on real dev
 ### mobile: isAppInstalled
 
 Checks whether the given application is installed on the device under test.
+An [offload application]((https://discussions.apple.com/thread/254887240)) could be handled as not installed.
 
 #### Arguments
 
@@ -123,6 +124,9 @@ Either `true` or `false`
 
 Removes the given application from the device under test.
 
+An [offload application]((https://discussions.apple.com/thread/254887240)) also can be removed.
+Please check [Clear the application local data explicitly for real devices](troubleshooting.md#clear-the-application-local-data-explicitly-for-real-devices) to ensure the application local data cleanup.
+
 #### Arguments
 
 Name | Type | Required | Description | Example
@@ -205,6 +209,7 @@ bundleId | string | yes | The bundle identifier of the application to be activat
 
 List applications installed on the real device under test. This extension throws an error if called
 for a Simulator device.
+Offload applications will not be in the result.
 
 #### Arguments
 

docs/troubleshooting.md

@@ -12,6 +12,27 @@ title: Troubleshooting
     - This configuration is only necessary for XCUITest driver v4.3.0 or lower.
 * `shake` is implemented via AppleScript and works only on Simulator due to lack of support from Apple
 
+
+## Clear the application local data explicitly for real devices
+
+There might be a situation where application data is present on the real device although application itself is not installed. This could happen if:
+- The app is in [offload state](https://discussions.apple.com/thread/254887240)
+- The application state is cached
+- There was an unexpected failure while installing the app. An example of such failure is the `ApplicationVerificationFailed` which happens while installing an app signed with an invalid provisioning profile.
+
+Under the circumstances above the application identifier won't be listed in the [`mobile: listApps`](execute-methods.md#mobile-listapps) output neither deleted by [`mobile: isAppInstalled`](execute-methods.md#mobile-isappinstalled) command. Setting `appium:fullReset` or `appium:enforceAppInstall` capabilities to `true` won't help to clear this data too.
+
+The only way to completely get rid of the cached application data is to call the [`mobile: removeApp`](execute-methods.md#mobile-removeapp) command with the appropriate bundle identifier.
+
+The driver automatically tries to resolve application installs that failed because of `MismatchedApplicationIdentifierEntitlement`, although, if you explicitly ask it to not perform the application uninstall then consider calling [`mobile: removeApp`](execute-methods.md#mobile-removeapp) beforehand `MismatchedApplicationIdentifierEntitlement` error occurs only when the previously installed application's provisioning profile is different from what currently the driver is trying to install.
+
+Below are example steps of the manual approach mentioned above:
+
+1. Start a session without `appium:app` and `appium:bundleId`
+2. Call [`mobile: removeApp`](execute-methods.md#mobile-removeapp) for the target application's bundle id
+3. Install the test target with [`mobile: installApp`](execute-methods.md#mobile-installapp)
+4. Launch the application with [`mobile: launchApp`](execute-methods.md#mobile-launchapp) or [`mobile: activateApp`](execute-methods.md#mobile-activateapp)
+
 ## Weird state
 
 ### stop responding

lib/commands/app-management.js

@@ -39,6 +39,7 @@ export default {
   },
   /**
    * Checks whether the given application is installed on the device under test.
+   * Offload app is handled as not installed.
    *
    * @param {string} bundleId - The bundle identifier of the application to be checked
    * @returns {Promise<boolean>} `true` if the application is installed; `false` otherwise
@@ -52,6 +53,7 @@ export default {
   },
   /**
    * Removes/uninstalls the given application from the device under test.
+   * Offload app data could also be removed.
    *
    * @param {string} bundleId - The bundle identifier of the application to be removed
    * @returns {Promise<boolean>} `true` if the application has been removed successfully; `false` otherwise

lib/driver.js

@@ -1610,7 +1610,8 @@ class XCUITestDriver extends BaseDriver {
         };
       }
     } else {
-      this.log.info(`App '${bundleId}' is not installed on the device yet`);
+      this.log.info(`App '${bundleId}' is not installed yet or it has an offload and ` +
+        'cannot be detected, which might keep the local data.');
     }
     if (enforceAppInstall !== false || fullReset || !wasAppInstalled) {
       return {

lib/real-device-management.js

@@ -58,6 +58,7 @@ async function runRealDeviceReset(device, opts) {
  * @property {boolean} [skipUninstall] Whether to skip app uninstall before installing it
  * @property {'serial'|'parallel'|'ios-deploy'} [strategy='serial'] One of possible install strategies ('serial', 'parallel', 'ios-deploy')
  * @property {number} [timeout] App install timeout
+ * @property {boolean} [shouldEnforceUninstall] Whether to enforce the app uninstallation. e.g. fullReset, or enforceAppInstall is true
  */
 
 /**
@@ -67,20 +68,44 @@ async function runRealDeviceReset(device, opts) {
  * @param {InstallOptions} [opts]
  */
 async function installToRealDevice(device, app, bundleId, opts) {
-  if (!device.udid || !app) {
-    log.debug('No device id or app, not installing to real device.');
+  if (!device.udid || !app || !bundleId) {
+    log.debug('No device id, app or bundle id, not installing to real device.');
     return;
   }
 
   const {skipUninstall, strategy, timeout} = opts ?? {};
 
-  if (!skipUninstall && bundleId && (await device.isAppInstalled(bundleId))) {
-    log.debug(`Reset requested. Removing app with id '${bundleId}' from the device`);
+  if (!skipUninstall) {
+    log.info(`Reset requested. Removing app with id '${bundleId}' from the device`);
     await device.remove(bundleId);
   }
   log.debug(`Installing '${app}' on device with UUID '${device.udid}'...`);
-  await device.install(app, timeout, strategy);
-  log.debug('The app has been installed successfully.');
+
+  try {
+    await device.install(app, timeout, strategy);
+    log.debug('The app has been installed successfully.');
+  } catch (e) {
+    // Want to clarify the device's application installation state in this situation.
+
+    if (!skipUninstall || !e.message.includes('MismatchedApplicationIdentifierEntitlement')) {
+      // Other error cases that could not be recoverable by here.
+      // Exact error will be in the log.
+
+      // We cannot recover 'ApplicationVerificationFailed' situation since this reason is clearly the app's provisioning profile was invalid.
+      // [XCUITest] Error installing app '/path/to.app': Unexpected data: {"Error":"ApplicationVerificationFailed","ErrorDetail":-402620395,"ErrorDescription":"Failed to verify code signature of /path/to.app : 0xe8008015 (A valid provisioning profile for this executable was not found.)"}
+      throw e;
+    }
+
+    // If the error was by below error case, we could recover the situation
+    // by uninstalling the device's app bundle id explicitly regard less the app exists on the device or not (e.g. offload app).
+    // [XCUITest] Error installing app '/path/to.app': Unexpected data: {"Error":"MismatchedApplicationIdentifierEntitlement","ErrorDescription":"Upgrade's application-identifier entitlement string (TEAM_ID.com.kazucocoa.example) does not match installed application's application-identifier string (ANOTHER_TEAM_ID.com.kazucocoa.example); rejecting upgrade."}
+    log.info(`The application identified by '${bundleId}' cannot be installed because it might ` +
+      `be already cached on the device, probably with a different signature. ` +
+      `Will try to remove it and install a new copy. Original error: ${e.message}`);
+    await device.remove(bundleId);
+    await device.install(app, timeout, strategy);
+    log.debug('The app has been installed after one retrial.');
+  }
 }
 
 function getRealDeviceObj(udid) {

test/unit/real-device-management-specs.js

@@ -0,0 +1,122 @@
+import chai from 'chai';
+import chaiAsPromised from 'chai-as-promised';
+import {createSandbox} from 'sinon';
+import sinonChai from 'sinon-chai';
+import { installToRealDevice } from '../../lib/real-device-management';
+import IOSDeploy from '../../lib/ios-deploy';
+
+chai.should();
+chai.use(sinonChai).use(chaiAsPromised);
+
+const expect = chai.expect;
+
+describe('installToRealDevice', function () {
+  const udid = 'test-udid';
+  const app = '/path/to.app';
+  const bundleId = 'test.bundle.id';
+
+  /** @type {sinon.SinonSandbox} */
+  let sandbox;
+  beforeEach(function () {
+    sandbox = createSandbox();
+  });
+
+  afterEach(function () {
+    sandbox.restore();
+  });
+
+  it('nothing happen without app', async function () {
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').resolves();
+
+    await installToRealDevice(iosDeploy, undefined, bundleId, {});
+    expect(iosDeploy.remove).to.not.have.been.called;
+    expect(iosDeploy.install).to.not.have.been.called;
+  });
+
+  it('nothing happen without bundle id', async function () {
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').resolves();
+
+    await installToRealDevice(iosDeploy, app, undefined, {});
+    expect(iosDeploy.remove).to.not.have.been.called;
+    expect(iosDeploy.install).to.not.have.been.called;
+  });
+
+  it('should install without remove', async function () {
+    const opts = {
+      skipUninstall: true
+    };
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').resolves();
+
+    await installToRealDevice(iosDeploy, app, bundleId, opts);
+
+    expect(iosDeploy.remove).to.not.have.been.called;
+    expect(iosDeploy.install).to.have.been.calledOnce;
+  });
+
+  it('should install after remove', async function () {
+    const opts = {
+      skipUninstall: false
+    };
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').resolves();
+
+    await installToRealDevice(iosDeploy, app, bundleId, opts);
+
+    expect(iosDeploy.remove).to.have.been.calledOnce;
+    expect(iosDeploy.install).to.have.been.calledOnce;
+  });
+
+  it('should raise an error for invalid verification error after uninstall', async function () {
+    const opts = {
+      skipUninstall: false
+    };
+    const err_msg = `{"Error":"ApplicationVerificationFailed","ErrorDetail":-402620395,"ErrorDescription":"Failed to verify code signature of /path/to.app : 0xe8008015 (A valid provisioning profile for this executable was not found.)"}`;
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').throws(err_msg);
+
+    await installToRealDevice(iosDeploy, app, bundleId, opts).should.be.rejectedWith('ApplicationVerificationFailed');
+    expect(iosDeploy.remove).to.have.been.calledOnce;
+    expect(iosDeploy.install).to.have.been.calledOnce;
+  });
+
+  it('should install after removal once because of MismatchedApplicationIdentifierEntitlement error', async function () {
+    // This situation could happen when the app exists as offload, or cached state
+    // with different application identifier
+    const opts = {
+      skipUninstall: true
+    };
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install')
+      .onCall(0).throws(`{"Error":"MismatchedApplicationIdentifierEntitlement","ErrorDescription":"Upgrade's application-identifier entitlement string (TEAM_ID.com.kazucocoa.example) does not match installed application's application-identifier string (ANOTHER_TEAM_ID.com.kazucocoa.example); rejecting upgrade."}`)
+      .onCall(1).resolves();
+
+    await installToRealDevice(iosDeploy, app, bundleId, opts);
+
+    expect(iosDeploy.remove).to.have.been.calledOnce;
+    expect(iosDeploy.install).to.have.been.calledTwice;
+  });
+
+  it('should raise an error in the install ApplicationVerificationFailed error because it is not recoverable', async function () {
+    const opts = {
+      skipUninstall: true
+    };
+    const err_msg = `{"Error":"ApplicationVerificationFailed","ErrorDetail":-402620395,"ErrorDescription":"Failed to verify code signature of /path/to.app : 0xe8008015 (A valid provisioning profile for this executable was not found.)"}`;
+    const iosDeploy = new IOSDeploy(udid);
+    sandbox.stub(iosDeploy, 'remove').resolves();
+    sandbox.stub(iosDeploy, 'install').throws(err_msg);
+    sandbox.stub(iosDeploy, 'isAppInstalled').resolves(true);
+
+    await installToRealDevice(iosDeploy, app, bundleId, opts).should.be.rejectedWith('ApplicationVerificationFailed');
+    expect(iosDeploy.remove).to.not.have.been.called;
+    expect(iosDeploy.install).to.have.been.calledOnce;
+  });
+});