-
Notifications
You must be signed in to change notification settings - Fork 141
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace re-directed dotnet-install.ps1
#5759
Conversation
dotnet-install.ps1
Datadog ReportBranch report: ❌ 4 Failed (0 Known Flaky), 340175 Passed, 1613 Skipped, 14h 12m 55.99s Total Time ❌ Failed Tests (4)
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (71ms) : 65, 78
. : milestone, 71,
master - mean (74ms) : 62, 85
. : milestone, 74,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (900ms) : 875, 925
. : milestone, 900,
master - mean (945ms) : 815, 1075
. : milestone, 945,
gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (109ms) : 107, 111
. : milestone, 109,
master - mean (120ms) : 111, 130
. : milestone, 120,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (635ms) : 614, 656
. : milestone, 635,
master - mean (741ms) : 650, 832
. : milestone, 741,
gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (94ms) : 90, 97
. : milestone, 94,
master - mean (122ms) : 119, 125
. : milestone, 122,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (591ms) : 571, 611
. : milestone, 591,
master - mean (671ms) : 495, 848
. : milestone, 671,
gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (191ms) : 188, 195
. : milestone, 191,
master - mean (191ms) : 187, 196
. : milestone, 191,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (1,002ms) : 977, 1027
. : milestone, 1002,
master - mean (1,001ms) : 974, 1028
. : milestone, 1001,
gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (280ms) : 275, 285
. : milestone, 280,
master - mean (276ms) : 273, 280
. : milestone, 276,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (817ms) : 787, 847
. : milestone, 817,
master - mean (823ms) : 798, 849
. : milestone, 823,
gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat X
axisFormat %s
todayMarker off
section Baseline
This PR (5759) - mean (265ms) : 261, 269
. : milestone, 265,
master - mean (265ms) : 262, 269
. : milestone, 265,
section CallTarget+Inlining+NGEN
This PR (5759) - mean (807ms) : 779, 835
. : milestone, 807,
master - mean (809ms) : 780, 839
. : milestone, 809,
|
Throughput/Crank Report:zap:Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5759) (11.571M) : 0, 11571038
master (11.842M) : 0, 11842369
benchmarks/2.9.0 (12.049M) : 0, 12048905
section Automatic
This PR (5759) (7.769M) : 0, 7768600
master (7.999M) : 0, 7998666
benchmarks/2.9.0 (8.464M) : 0, 8463997
section Trace stats
master (8.337M) : 0, 8336641
section Manual
This PR (5759) (9.957M) : 0, 9957383
master (10.173M) : 0, 10173108
section Manual + Automatic
This PR (5759) (7.254M) : 0, 7253620
master (7.350M) : 0, 7349541
section Version Conflict
master (6.703M) : 0, 6702940
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5759) (9.611M) : 0, 9610718
master (9.512M) : 0, 9511621
benchmarks/2.9.0 (9.590M) : 0, 9590349
section Automatic
This PR (5759) (6.482M) : 0, 6482339
master (6.515M) : 0, 6515399
section Trace stats
master (6.862M) : 0, 6861752
section Manual
This PR (5759) (8.338M) : 0, 8337696
master (8.299M) : 0, 8298527
section Manual + Automatic
This PR (5759) (6.166M) : 0, 6165997
master (6.262M) : 0, 6262373
section Version Conflict
master (5.588M) : 0, 5588112
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5759) (10.160M) : 0, 10160398
master (10.162M) : 0, 10161696
benchmarks/2.9.0 (10.465M) : 0, 10465033
section Automatic
This PR (5759) (7.110M) : 0, 7110075
master (7.293M) : 0, 7293498
benchmarks/2.9.0 (7.635M) : 0, 7635248
section Trace stats
master (7.497M) : 0, 7496532
section Manual
This PR (5759) (8.778M) : crit ,0, 8778087
master (9.243M) : 0, 9243291
section Manual + Automatic
This PR (5759) (6.710M) : 0, 6709696
master (6.976M) : 0, 6975655
section Version Conflict
master (6.267M) : 0, 6266553
|
Benchmarks Report for tracer 🐌Benchmarks for #5759 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.ElasticsearchBenchmark - Slower
|
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 | 1.131 | 1,275.94 | 1,443.59 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | CallElasticsearch |
net6.0 | 1.24μs | 0.458ns | 1.78ns | 0.0137 | 0 | 0 | 976 B |
master | CallElasticsearch |
netcoreapp3.1 | 1.51μs | 0.45ns | 1.62ns | 0.0128 | 0 | 0 | 976 B |
master | CallElasticsearch |
net472 | 2.64μs | 1.83ns | 6.84ns | 0.157 | 0 | 0 | 995 B |
master | CallElasticsearchAsync |
net6.0 | 1.28μs | 0.653ns | 2.44ns | 0.0134 | 0 | 0 | 952 B |
master | CallElasticsearchAsync |
netcoreapp3.1 | 1.73μs | 3.48ns | 13.5ns | 0.0137 | 0 | 0 | 1.02 KB |
master | CallElasticsearchAsync |
net472 | 2.5μs | 1.43ns | 5.34ns | 0.166 | 0 | 0 | 1.05 KB |
#5759 | CallElasticsearch |
net6.0 | 1.31μs | 0.639ns | 2.47ns | 0.0138 | 0 | 0 | 976 B |
#5759 | CallElasticsearch |
netcoreapp3.1 | 1.6μs | 0.631ns | 2.44ns | 0.0129 | 0 | 0 | 976 B |
#5759 | CallElasticsearch |
net472 | 2.57μs | 2.18ns | 8.44ns | 0.157 | 0 | 0 | 995 B |
#5759 | CallElasticsearchAsync |
net6.0 | 1.44μs | 0.686ns | 2.66ns | 0.0129 | 0 | 0 | 952 B |
#5759 | CallElasticsearchAsync |
netcoreapp3.1 | 1.65μs | 1.58ns | 5.9ns | 0.0141 | 0 | 0 | 1.02 KB |
#5759 | CallElasticsearchAsync |
net472 | 2.64μs | 1.02ns | 3.52ns | 0.167 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | ExecuteAsync |
net6.0 | 1.26μs | 0.528ns | 2.05ns | 0.0132 | 0 | 0 | 952 B |
master | ExecuteAsync |
netcoreapp3.1 | 1.73μs | 5.54ns | 21.5ns | 0.013 | 0 | 0 | 952 B |
master | ExecuteAsync |
net472 | 1.84μs | 0.645ns | 2.5ns | 0.145 | 0 | 0 | 915 B |
#5759 | ExecuteAsync |
net6.0 | 1.29μs | 0.852ns | 3.19ns | 0.0134 | 0 | 0 | 952 B |
#5759 | ExecuteAsync |
netcoreapp3.1 | 1.62μs | 0.606ns | 2.19ns | 0.0122 | 0 | 0 | 952 B |
#5759 | ExecuteAsync |
net472 | 1.83μs | 1.01ns | 3.89ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendAsync |
net6.0 | 4.18μs | 1.82ns | 6.81ns | 0.0314 | 0 | 0 | 2.22 KB |
master | SendAsync |
netcoreapp3.1 | 5.01μs | 2.42ns | 9.36ns | 0.0379 | 0 | 0 | 2.76 KB |
master | SendAsync |
net472 | 7.71μs | 8.49ns | 32.9ns | 0.498 | 0 | 0 | 3.15 KB |
#5759 | SendAsync |
net6.0 | 4.18μs | 1.21ns | 4.18ns | 0.0313 | 0 | 0 | 2.22 KB |
#5759 | SendAsync |
netcoreapp3.1 | 5.02μs | 2.17ns | 8.41ns | 0.0377 | 0 | 0 | 2.76 KB |
#5759 | SendAsync |
net472 | 7.58μs | 1.4ns | 5.23ns | 0.497 | 0 | 0 | 3.15 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 1.45μs | 0.588ns | 2.2ns | 0.0231 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
netcoreapp3.1 | 2.23μs | 1.75ns | 6.79ns | 0.0223 | 0 | 0 | 1.64 KB |
master | EnrichedLog |
net472 | 2.53μs | 1.43ns | 5.37ns | 0.249 | 0 | 0 | 1.57 KB |
#5759 | EnrichedLog |
net6.0 | 1.51μs | 0.628ns | 2.35ns | 0.0232 | 0 | 0 | 1.64 KB |
#5759 | EnrichedLog |
netcoreapp3.1 | 2.07μs | 0.783ns | 2.71ns | 0.0217 | 0 | 0 | 1.64 KB |
#5759 | EnrichedLog |
net472 | 2.57μs | 1.04ns | 4.04ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 112μs | 63.9ns | 239ns | 0 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
netcoreapp3.1 | 118μs | 134ns | 502ns | 0 | 0 | 0 | 4.28 KB |
master | EnrichedLog |
net472 | 146μs | 95.7ns | 358ns | 0.658 | 0.219 | 0 | 4.46 KB |
#5759 | EnrichedLog |
net6.0 | 117μs | 158ns | 613ns | 0.0563 | 0 | 0 | 4.28 KB |
#5759 | EnrichedLog |
netcoreapp3.1 | 121μs | 230ns | 830ns | 0.0606 | 0 | 0 | 4.28 KB |
#5759 | EnrichedLog |
net472 | 147μs | 57.7ns | 223ns | 0.659 | 0.22 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.98μs | 2.08ns | 8.05ns | 0.0313 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
netcoreapp3.1 | 4.28μs | 2.74ns | 10.3ns | 0.0278 | 0 | 0 | 2.2 KB |
master | EnrichedLog |
net472 | 4.82μs | 2.66ns | 10.3ns | 0.319 | 0 | 0 | 2.02 KB |
#5759 | EnrichedLog |
net6.0 | 3.01μs | 1.47ns | 5.69ns | 0.03 | 0 | 0 | 2.2 KB |
#5759 | EnrichedLog |
netcoreapp3.1 | 4.25μs | 1.15ns | 4.29ns | 0.0302 | 0 | 0 | 2.2 KB |
#5759 | EnrichedLog |
net472 | 4.76μs | 4.33ns | 16.2ns | 0.319 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | SendReceive |
net6.0 | 1.3μs | 0.483ns | 1.87ns | 0.0162 | 0 | 0 | 1.14 KB |
master | SendReceive |
netcoreapp3.1 | 1.72μs | 1.1ns | 4.25ns | 0.0156 | 0 | 0 | 1.14 KB |
master | SendReceive |
net472 | 2.09μs | 1.32ns | 5.13ns | 0.183 | 0 | 0 | 1.16 KB |
#5759 | SendReceive |
net6.0 | 1.27μs | 1.1ns | 4.13ns | 0.0159 | 0 | 0 | 1.14 KB |
#5759 | SendReceive |
netcoreapp3.1 | 1.73μs | 0.7ns | 2.71ns | 0.0149 | 0 | 0 | 1.14 KB |
#5759 | SendReceive |
net472 | 2.08μs | 0.526ns | 1.9ns | 0.183 | 0 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | EnrichedLog |
net6.0 | 2.82μs | 0.832ns | 3.22ns | 0.0227 | 0 | 0 | 1.6 KB |
master | EnrichedLog |
netcoreapp3.1 | 3.99μs | 3.08ns | 11.9ns | 0.022 | 0 | 0 | 1.65 KB |
master | EnrichedLog |
net472 | 4.46μs | 2.9ns | 11.2ns | 0.322 | 0 | 0 | 2.04 KB |
#5759 | EnrichedLog |
net6.0 | 2.76μs | 0.499ns | 1.87ns | 0.0221 | 0 | 0 | 1.6 KB |
#5759 | EnrichedLog |
netcoreapp3.1 | 3.92μs | 1.67ns | 6.26ns | 0.0216 | 0 | 0 | 1.65 KB |
#5759 | EnrichedLog |
net472 | 4.27μs | 2.25ns | 8.71ns | 0.324 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️
Slower ⚠️ in #5759
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0
1.122
391.51
439.30
Faster 🎉 in #5759
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0
1.138
549.73
483.21
Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 | 1.122 | 391.51 | 439.30 |
Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
---|---|---|---|---|
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 | 1.138 | 549.73 | 483.21 |
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | StartFinishSpan |
net6.0 | 391ns | 0.306ns | 1.19ns | 0.00814 | 0 | 0 | 576 B |
master | StartFinishSpan |
netcoreapp3.1 | 585ns | 0.328ns | 1.18ns | 0.00798 | 0 | 0 | 576 B |
master | StartFinishSpan |
net472 | 707ns | 0.37ns | 1.39ns | 0.0916 | 0 | 0 | 578 B |
master | StartFinishScope |
net6.0 | 550ns | 0.307ns | 1.19ns | 0.00963 | 0 | 0 | 696 B |
master | StartFinishScope |
netcoreapp3.1 | 742ns | 0.806ns | 2.91ns | 0.00932 | 0 | 0 | 696 B |
master | StartFinishScope |
net472 | 868ns | 0.845ns | 3.27ns | 0.104 | 0 | 0 | 658 B |
#5759 | StartFinishSpan |
net6.0 | 439ns | 0.111ns | 0.43ns | 0.00813 | 0 | 0 | 576 B |
#5759 | StartFinishSpan |
netcoreapp3.1 | 593ns | 0.354ns | 1.37ns | 0.00766 | 0 | 0 | 576 B |
#5759 | StartFinishSpan |
net472 | 648ns | 0.817ns | 3.16ns | 0.0917 | 0 | 0 | 578 B |
#5759 | StartFinishScope |
net6.0 | 483ns | 0.0936ns | 0.35ns | 0.00972 | 0 | 0 | 696 B |
#5759 | StartFinishScope |
netcoreapp3.1 | 672ns | 0.39ns | 1.35ns | 0.00947 | 0 | 0 | 696 B |
#5759 | StartFinishScope |
net472 | 849ns | 0.416ns | 1.61ns | 0.104 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
---|---|---|---|---|---|---|---|---|---|
master | RunOnMethodBegin |
net6.0 | 634ns | 0.122ns | 0.455ns | 0.00984 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
netcoreapp3.1 | 912ns | 2.94ns | 11.4ns | 0.0092 | 0 | 0 | 696 B |
master | RunOnMethodBegin |
net472 | 1.06μs | 0.971ns | 3.76ns | 0.104 | 0 | 0 | 658 B |
#5759 | RunOnMethodBegin |
net6.0 | 578ns | 0.264ns | 1.02ns | 0.00968 | 0 | 0 | 696 B |
#5759 | RunOnMethodBegin |
netcoreapp3.1 | 895ns | 0.194ns | 0.698ns | 0.00928 | 0 | 0 | 696 B |
#5759 | RunOnMethodBegin |
net472 | 1.11μs | 0.203ns | 0.785ns | 0.104 | 0 | 0 | 658 B |
Superseded by #5770 |
## Summary of changes - Vendor the `dotnet-install.sh` and `dotnet-install.ps1` scripts into the repo - Replace the centos7 repo references with vault based ones ## Reason for change - https://dotnet.microsoft.com/ went down recently, which meant we couldn't download `dotnet-install.sh` or `dotnet-install.ps1`, which broke a bunch of things, so vendor it. - Centos7 recently shut down their repo feed, which means you can no longer pull packages. Use vault repo instead until we deprecate centos7 entirely ## Implementation details - Vendor the scripts - Replace downloading of the script with direct reference - do some `sed` to replace the centos7 repo ## Test coverage Largely, this is the test, if it all works, I think we're good ## Other details Supersedes #5759 Requires updating the VMs <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
## Summary of changes - Vendor the `dotnet-install.sh` and `dotnet-install.ps1` scripts into the repo - Replace the centos7 repo references with vault based ones ## Reason for change - https://dotnet.microsoft.com/ went down recently, which meant we couldn't download `dotnet-install.sh` or `dotnet-install.ps1`, which broke a bunch of things, so vendor it. - Centos7 recently shut down their repo feed, which means you can no longer pull packages. Use vault repo instead until we deprecate centos7 entirely ## Implementation details - Vendor the scripts - Replace downloading of the script with direct reference - do some `sed` to replace the centos7 repo ## Test coverage Largely, this is the test, if it all works, I think we're good ## Other details Supersedes #5759 Requires updating the VMs <!--⚠️ Note: where possible, please obtain 2 approvals prior to merging. Unless CODEOWNERS specifies otherwise, for external teams it is typically best to have one review from a team member, and one review from apm-dotnet. Trivial changes do not require 2 reviews. -->
Summary of changes
Replace current URL for
dotnet-install.ps1
to use the raw GitHub contentReason for change
https://dotnet.microsoft.com is down, which breaks this download, so using the URL it redirects to
Implementation details
Use the direct GitHub link instead
Test coverage
This is the test
Other details
dotnet/runtime#104230
When I'm feeling stronger, we can download this ahead of time and update all the docker images and VMs too...