Improve the testability of emulator support

[larkee]

The emulator support testing is largely untested for all products due to difficulties with testability. This should be fixed!

[ava12]

Not sure what the task is. I see system tests for Spanner client that are not supported by the emulator are skipped. @larkee, is that what we need - skip unsupported tests? Or is is there something else to do?

[ava12]

Looks like Spanner tests work fine: unsupported tests are skipped, others pass (just don't forget to create test instance). As for other libraries:

• BigTable: all tests with real server pass, all tests with emulator throw an "UNIMPLEMENTED" error.
• Datastore: all tests pass.
• Firestore: all tests pass. The only problem found is flaky FirestoreSessionHandlerTest::testGarbageCollection().
• PubSub: all tests with real server pass. All REST client tests with emulator throw an "UNIMPLEMENTED" error. gRPC client tests with emulator pass, except for:
  • tests using IAM, subscription detach and topic update throw an "UNIMPLEMENTED" error;
  • tests trying to change pushConfig.attributes or retryPolicy.maximumBackoff throw "INVALID_ARGUMENT";
  • default value for deadLetterPolicy.maxDeliveryAttempts is 0 instead of 5;
  • filtering by message attributes does not work.

[meredithslota]

@ava12 It's unclear what else remains to be done here — do you know?

[vishwarajanand]

I am closing this issue since we have emulator based tests already setup for FS/DS/Spanner/Bigtable/PubSub.
ref has test workflows which includes starting an emulator for all 5 products listed above.