Writing End to End Tests using AzureTest

AzureTest is a static library written to help Azure SDK developers write tests for Swift that can be recorded and played back without network connectivity. It leverages a customized fork of the Venmo/DVR framework, which itself is based on the popular Ruby VCR framework.

Prerequisites

You will need the following things installed:

• Powershell
• Bicep CLI (if using bicep files instead of ARM templates)
• XCode
• Swift 5+

Getting Started

1. Within your framework's test targets, add libAzureTest.a under Build Phases > Link Binary With Libraries.
2. In Package.swift, add AzureTest to the dependencies for your test target.
3. Within the test scheme for your target, ensure you have created and set the following environment variables:

• TEST_MODE: Set to either "record" or "playback".
• SDK_REPO_ROOT: The local path to your clone of azure-sdk-for-ios. This is used only during recording. All files during playback must be included in the app bundle.

Creating Resources

Most tests require some Azure resources to be set up prior to running, and these resources should be cleaned up after the test completes. Mechanisms exists to facilitate this automatically.

1. Within the service folder for your framework (the folder directly beneath sdk), add resources to either the test-resources.bicep file or test-resources.json. This file will be used to deploy resources to run your tests live (such as for recording).
2. If you need to do further setup with the resources created in step 1, add or modify the test-resources-pre.ps1 or test-resources-post.ps1 scripts to perform any necessary setup. By default, you can only use Powershell cmdlets. The necessary outputs from these scripts should be dumped into a file called test-settings.plist (see below).
3. Run the following common engineering script from the root of the SDK repo to deploy your test resources: ./eng/common/TestResources/New-TestResources.ps1 <SERVICE_DIR>

Storing Credentials

1. Create a class which implements AzureTest.TestSettingsProtocol. This model should consist of string-based properties which contain the placeholder values that will be used during playback. These values must be deterministic. NEVER put actual credentials, tokens, connection strings, etc in this file!
2. Within your Tests folder, add the plist file created in step 3 above to the project. The file MUST MUST be named test-settings.plist, as this filename is part of the .gitignore and thus will not be committed as a change in source control. This file contains the actual values necessary to run your test live.
3. Ensure that test-settings.plist has target membership in the relevant test target(s).

Writing Tests

1. Within the Tests folder, create a new "Unit Test Case Class" in Xcode and import AzureTest along with any other necessary imports.
2. Change the test class to inherit from RecordableXCTestCase instead of XCTestCase and supply the name of your settings class as the generic argument (ex: class MyFirstTest: RecordableXCTestCase<MyTestSettings> { ... }).
3. Implement the setUpTest and/or setUpTestWithError method to load any values from your settings and set up your client in order to make service calls.
4. Use your client to make service calls like you normally would. After you first run your tests in record mode, you must add the recording files to your project. Otherwise, the files will not be found when you attempt to play them back. Once the files have been added initially, re-recording tests will automatically update them.

Scrubbing Values

AzureTest contains a class called RequestURLFilter which can be used to replace non-deterministic values in the request URL and body with deterministic ones to facilitate reliable playback. As an example:

    private var urlFilter: RequestURLFilter {
        let defaults = TestSettings()
        let textFilter = RequestURLFilter()
        textFilter.register(replacement: defaults.endpoint, for: settings.endpoint)
        return textFilter
    }

The textFilter.register(replacement:for:) method is used to register replacements. You can register as many replacements as necessary.

Then, in the setUpTest or setUpTestWithError method, you can register this filter with add(filter: urlFilter).

Playing Back Tests

1. Change the TEST_MODE environment variable in your target's Test scheme to "playback".
2. Run the tests. They should complete much faster and without any network connectivity.
3. Remove the test-settings.plist reference from your test folder before committing it or you will get a build error that the input file is not found.

Troubleshooting

• If playback mode fails, ensure you have added the recording files to the project.
• If you get a build error when creating a PR, ensure you have removed the reference to test-setting.plist from your project.