run_examples script not deterministic

[daviddrysdale]

Two runs of ./scripts/docker_run ./scripts/run_examples don't always give the same result, both locally and on Cloudbuild. For example, Cloudbuild runs 0b774ec3 and 2163a524 are run at the same commit but one fails and one succeeds.

The failure log (below) makes me wonder whether there's some parallelism at fault – maybe one job is rebuilding the .proto generated code at the same moment another job is trying to include it?

+ cargo build --release --target=wasm32-unknown-unknown --manifest-path=examples/hello_world/module/rust/Cargo.toml
2020-01-22 15:13:38  INFO  asylo_oak_main.cc : 64 : gRPC server started on port 8888
   Compiling log v0.4.7
   Compiling syn v0.15.42
   Compiling protoc v2.8.0
   Compiling protoc-rust v2.8.0
   Compiling oak_abi v0.1.0 (/workspace/oak/server/rust/oak_abi)
   Compiling oak v0.1.0 (/workspace/sdk/rust/oak)
   Compiling protoc-rust-grpc v0.7.0 (/workspace/third_party/grpc-rust/protoc-rust-grpc)
error[E0432]: unresolved import `proto::oak_api::ChannelReadStatus`
  --> /workspace/oak/server/rust/oak_abi/src/lib.rs:19:9
   |
19 | pub use proto::oak_api::ChannelReadStatus;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `ChannelReadStatus` in `proto::oak_api`

error[E0432]: unresolved import `proto::oak_api::OakStatus`
  --> /workspace/oak/server/rust/oak_abi/src/lib.rs:20:9
   |
20 | pub use proto::oak_api::OakStatus;
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^ no `OakStatus` in `proto::oak_api`

error: aborting due to 2 previous errors

[tiziano88]

Is this still the case, even after #649 ?

/cc @ipetr0v

[daviddrysdale]

I've not seen this for a while so I'll close this out.