Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PR #10101/678993a4 backport][3.11] Fix race in FileResponse if file is replaced during prepare #10105

Conversation

patchback[bot]
Copy link
Contributor

@patchback patchback bot commented Dec 4, 2024

This is a backport of PR #10101 as merged into master (678993a).

There was a race in FileResponse where the stat would be incorrect if the file was changed out between the stat and open syscalls. This would lead to various unexpected behaviors such as trying to read beyond the length of the file or sending a partial file. This problem is likely to occour when files are being renamed/linked into place.

An example of how this can happen with a system that provides weather data every 60s:

An external process writes .weather.txt at the top of each minute, and than renames it into place as weather.txt. In this case FileResponse.prepare may stat the old weather.txt, and than open the new weather.txt, and use the os.stat_result from the original file.

To fix this we now fstat the open file on operating systems where fstat is available.

This change looks much larger than it actually is because most of the function now needs to be wrapped in the try/finally to ensure that we close the file cleanly as we open it a lot sooner now. Its best to compare without white space as well.

fixes #8013

@bdraco bdraco enabled auto-merge (squash) December 4, 2024 20:13
Copy link

codspeed-hq bot commented Dec 4, 2024

CodSpeed Performance Report

Merging #10105 will improve performances by 15.19%

Comparing patchback/backports/3.11/678993a4bd071200fbcc14fa121c6673221aa540/pr-10101 (fe03729) with 3.11 (23a4b31)

Summary

⚡ 2 improvements
✅ 44 untouched benchmarks

Benchmarks breakdown

Benchmark 3.11 patchback/backports/3.11/678993a4bd071200fbcc14fa121c6673221aa540/pr-10101 Change
test_simple_web_file_response[pyloop] 95.6 ms 83 ms +15.19%
test_simple_web_file_sendfile_fallback_response[pyloop] 101 ms 88.9 ms +13.59%

Copy link

codecov bot commented Dec 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.06%. Comparing base (23a4b31) to head (fe03729).
Report is 1 commits behind head on 3.11.

✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             3.11   #10105      +/-   ##
==========================================
- Coverage   98.06%   98.06%   -0.01%     
==========================================
  Files         123      123              
  Lines       37098    37113      +15     
  Branches     4459     4466       +7     
==========================================
+ Hits        36381    36395      +14     
  Misses        540      540              
- Partials      177      178       +1     
Flag Coverage Δ
CI-GHA 97.95% <100.00%> (+<0.01%) ⬆️
OS-Linux 97.65% <100.00%> (+<0.01%) ⬆️
OS-Windows 94.57% <100.00%> (-0.01%) ⬇️
OS-macOS 96.77% <100.00%> (-0.01%) ⬇️
Py-3.10.11 96.62% <100.00%> (+<0.01%) ⬆️
Py-3.10.15 97.16% <100.00%> (-0.01%) ⬇️
Py-3.11.10 97.22% <100.00%> (-0.02%) ⬇️
Py-3.11.9 96.69% <100.00%> (-0.03%) ⬇️
Py-3.12.7 97.72% <100.00%> (-0.01%) ⬇️
Py-3.13.0 96.55% <100.00%> (-0.01%) ⬇️
Py-3.13.1 97.69% <100.00%> (-0.01%) ⬇️
Py-3.9.13 96.55% <93.93%> (-0.01%) ⬇️
Py-3.9.20 97.12% <93.93%> (-0.01%) ⬇️
Py-pypy7.3.16 96.69% <93.93%> (+<0.01%) ⬆️
VM-macos 96.77% <100.00%> (-0.01%) ⬇️
VM-ubuntu 97.65% <100.00%> (+<0.01%) ⬆️
VM-windows 94.57% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bdraco bdraco merged commit 07d1759 into 3.11 Dec 4, 2024
34 checks passed
@bdraco bdraco deleted the patchback/backports/3.11/678993a4bd071200fbcc14fa121c6673221aa540/pr-10101 branch December 4, 2024 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant