Pin Azure Functions version used in CI to get the integration tests running (#6203 => v2) #6227

andrewlock · 2024-11-01T17:10:23Z

Summary of changes

Pins Azure functions to 4.0.6280 to fix CI

Reason for change

The "latest" Azure Functions installer installs version 4.0.6543. This causes the Azure Functions integration tests to fail with:

Status: 400 (The REST version of this request is not supported by this release of the Storage Emulator. Please upgrade the storage emulator to the latest version. Refer to the following URL for more information: http://go.microsoft.com/fwlink/?LinkId=392237)

However, there isn't_ a later version AFAICT...

My assumption is we need to use the azurite tool, because the storage emulator is no longer supported. However, my efforst to get that working have been... frustrating... as you can see from the commit list...

Essentially, azurite is installed in the VMs, and you can start it by hitting "%ProgramFiles%\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Microsoft\Azure Storage Emulator\azurite.exe", and it will start listening on the expected ports:

Azurite Blob service is starting at http://127.0.0.1:10000/
Azurite Blob service is successfully listening at http://127.0.0.1:10000/
Azurite Queue service is starting at http://127.0.0.1:10001/
Azurite Queue service is successfully listening at http://127.0.0.1:10001/
Azurite Table service is starting at http://127.0.0.1:10002/
Azurite Table service is successfully listening at http://127.0.0.1:10002/

But there's no way to "properly" run it in the background on Windows other than using start /b which fundamentally doesn't seem to work (azure functions fails to connect to it):

13:28:31 [DBG]  [2024-10-25T13:19:09.878Z] The listener for function 'TriggerAllTimer' was unable to start. Azure.Core: Retry failed after 6 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)). Azure.Core: No connection could be made because the target machine actively refused it. (127.0.0.1:10000). System.Net.Http: No connection could be made because the target machine actively refused it. (127.0.0.1:10000). System.Net.Sockets: No connection could be made because the target machine actively refused it.

Implementation details

I gave up and just pinned the version of Azure functions we install. We should update this to the latest when we can.

FWIW, https://github.com/Azure/azure-functions-core-tools/releases doesn't even list the new, breaking, version yet

Test coverage

The bit downside is we are no longer testing with the latest version of Azure Functions tools 🙁

Other details

Backport of

Pin Azure Functions version used in CI to get the integration tests running #6203

…unning (#6203) ## Summary of changes Pins Azure functions to `4.0.6280` to fix CI ## Reason for change The "latest" Azure Functions installer installs version `4.0.6543`. [This causes the Azure Functions integration tests to fail](https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=166249&view=logs&j=b915f7fe-737b-590c-2437-5d9fd23477ed&t=b6a81d99-5699-541f-919b-ca36a45374e0) with: ``` Status: 400 (The REST version of this request is not supported by this release of the Storage Emulator. Please upgrade the storage emulator to the latest version. Refer to the following URL for more information: http://go.microsoft.com/fwlink/?LinkId=392237) ``` However, there isn't_ a later version AFAICT... My assumption is we need to use the azurite tool, because the storage emulator is no longer supported. However, my efforst to get that working have been... frustrating... as you can see from the commit list... Essentially, azurite is installed in the VMs, and you can start it by hitting `"%ProgramFiles%\Microsoft Visual Studio\2022\Enterprise\Common7\IDE\Extensions\Microsoft\Azure Storage Emulator\azurite.exe"`, and it will start listening on the expected ports: ``` Azurite Blob service is starting at http://127.0.0.1:10000/ Azurite Blob service is successfully listening at http://127.0.0.1:10000/ Azurite Queue service is starting at http://127.0.0.1:10001/ Azurite Queue service is successfully listening at http://127.0.0.1:10001/ Azurite Table service is starting at http://127.0.0.1:10002/ Azurite Table service is successfully listening at http://127.0.0.1:10002/ ``` But [there's no way to "properly" run it in the background on Windows](Azure/Azurite#589) other than using `start /b` which fundamentally doesn't seem to work (azure functions fails to connect to it): ``` 13:28:31 [DBG] [2024-10-25T13:19:09.878Z] The listener for function 'TriggerAllTimer' was unable to start. Azure.Core: Retry failed after 6 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)) (No connection could be made because the target machine actively refused it. (127.0.0.1:10000)). Azure.Core: No connection could be made because the target machine actively refused it. (127.0.0.1:10000). System.Net.Http: No connection could be made because the target machine actively refused it. (127.0.0.1:10000). System.Net.Sockets: No connection could be made because the target machine actively refused it. ``` ## Implementation details I gave up and just pinned the version of Azure functions we install. We should update this to the latest when we can. FWIW, https://github.com/Azure/azure-functions-core-tools/releases doesn't even list the new, breaking, version yet ## Test coverage The bit downside is we are no longer testing with the latest version of Azure Functions tools 🙁

datadog-ddstaging · 2024-11-01T17:27:41Z

Datadog Report

Branch report: backport/andrew/ci/fix-azure-functions
Commit report: c37d243
Test service: dd-trace-dotnet

✅ 0 Failed, 342866 Passed, 1784 Skipped, 13h 59m 53.6s Total Time

andrewlock · 2024-11-01T17:41:16Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (74ms)  : 66, 83
     .   : milestone, 74,
    master - mean (71ms)  : 68, 74
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (1,023ms)  : 1004, 1041
     .   : milestone, 1023,
    master - mean (1,117ms)  : 1097, 1138
     .   : milestone, 1117,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (111ms)  : 107, 114
     .   : milestone, 111,
    master - mean (110ms)  : 105, 115
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (704ms)  : 685, 722
     .   : milestone, 704,
    master - mean (780ms)  : 764, 796
     .   : milestone, 780,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (94ms)  : 90, 99
     .   : milestone, 94,
    master - mean (93ms)  : 91, 96
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (665ms)  : 642, 687
     .   : milestone, 665,
    master - mean (734ms)  : 721, 747
     .   : milestone, 734,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (191ms)  : 188, 195
     .   : milestone, 191,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (1,114ms)  : 1090, 1138
     .   : milestone, 1114,
    master - mean (1,221ms)  : 1199, 1243
     .   : milestone, 1221,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (277ms)  : 272, 281
     .   : milestone, 277,
    master - mean (277ms)  : 270, 284
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (877ms)  : 859, 894
     .   : milestone, 877,
    master - mean (949ms)  : 929, 969
     .   : milestone, 949,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6227) - mean (266ms)  : 262, 269
     .   : milestone, 266,
    master - mean (264ms)  : 261, 268
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6227) - mean (864ms)  : 845, 882
     .   : milestone, 864,
    master - mean (932ms)  : 911, 952
     .   : milestone, 932,

andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages area:tests unit tests, integration tests labels Nov 1, 2024

andrewlock requested a review from a team as a code owner November 1, 2024 17:10

lucaspimentel approved these changes Nov 1, 2024

View reviewed changes

andrewlock merged commit 9bc2ebe into release/2.x Nov 4, 2024
70 of 73 checks passed

andrewlock deleted the backport/andrew/ci/fix-azure-functions branch November 4, 2024 09:27

github-actions bot added this to the vNext-v2 milestone Nov 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pin Azure Functions version used in CI to get the integration tests running (#6203 => v2) #6227

Pin Azure Functions version used in CI to get the integration tests running (#6203 => v2) #6227

andrewlock commented Nov 1, 2024

datadog-ddstaging bot commented Nov 1, 2024 •

edited

Loading

andrewlock commented Nov 1, 2024

Pin Azure Functions version used in CI to get the integration tests running (#6203 => v2) #6227

Pin Azure Functions version used in CI to get the integration tests running (#6203 => v2) #6227

Conversation

andrewlock commented Nov 1, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Nov 1, 2024 • edited Loading

Datadog Report

andrewlock commented Nov 1, 2024

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Nov 1, 2024 •

edited

Loading