Intermittent test failure: test___streaming_measurement_execution___cancel___cancels_measurement

[bkeryan]

Bug Report

test___streaming_measurement_execution___cancel___cancels_measurement failed in the PR build.

Repro or Code Sample

Run PR workflow.

Expected Behavior

Tests pass.

Current Behavior

https://github.com/ni/measurement-plugin-python/actions/runs/10966811168/job/30455562171

================================== FAILURES ===================================
____ test___streaming_measurement_execution___cancel___cancels_measurement ____

measurement_plugin_client_module = <module 'test_measurement_client.py' from 'C:\\Users\\ec2-user\\AppData\\Local\\Temp\\pytest-of-unknown\\pytest-5\\measurement_plugin_client_files7\\test_measurement_client.py'>

    def test___streaming_measurement_execution___cancel___cancels_measurement(
        measurement_plugin_client_module: ModuleType,
    ) -> None:
        test_measurement_client_type = getattr(measurement_plugin_client_module, "TestMeasurement")
        measurement_plugin_client = test_measurement_client_type()
    
        with pytest.raises(grpc.RpcError) as exc_info:
            with concurrent.futures.ThreadPoolExecutor() as executor:
                measure = executor.submit(lambda: list(measurement_plugin_client.stream_measure()))
                measurement_plugin_client.cancel()
>               measure.result()
E               Failed: DID NOT RAISE <class 'grpc.RpcError'>

C:\actions-runner\1\_work\measurement-plugin-python\measurement-plugin-python\packages\generator\tests\acceptance\test_streaming_measurement_client.py:96: Failed

Possible Solution

If the root cause is that the RPC is canceled before it starts, then it may help to have the test measurement signal an event and have the test wait for the event, as previously suggested: #870 (comment)

Context

Test failed for unrelated PR.

AB#2863572

[Jotheeswaran-Nandagopal]

@bkeryan, I am not sure of any direct way to get signal from an event from the test measurement before cancelling the measurement.
The proposed approach for this issue is to cancel the stream_measure call during the first iteration.

with pytest.raises(grpc.RpcError) as exc_info:
        measurement_outputs = measurement_plugin_client.stream_measure()
        for output in measurement_outputs:
            measurement_plugin_client.cancel()

Could you please share your thoughts on this approach?

[bkeryan]

@Jotheeswaran-Nandagopal First, find the root cause and explain it, then propose a fix.

@bkeryan, I am not sure of any direct way to get signal from an event from the test measurement before cancelling the measurement.

Two options:

• Wait for initial metadata. Example: https://github.com/ni/grpc-device/blob/main/examples/nidaqmx/analog-input-every-n-samples.py#L107
• Have the test directly interact with the test measurement. It's a Python module loaded in the same process, so you could set variables or call functions.

[dixonjoel]

@bkeryan I ran the test locally 5000 times with no failures, so it will be hard to validate the fix locally. Maybe we just implement one of the solutions and watch the test for a while on the PR? Would you be able to take this bug and implement a fix?

I'm not sure if there's any easy way to see the history for this test so we can watch for it to stop failing or see exactly how often it's failing on the test machines.

[bkeryan]

If we can't reproduce it, then let's close it as not reproducible.

[dixonjoel]

Cannot reproduce