Restrict POST query size

[ikreymer]

Description

Current indexing reads an entire POST request, which can be quite large, and attempts to append to urlkey query. Now restrict reading POST request to 16384, skip reading the remainder.

Motivation and Context

Many sites include large logs in POST during regular browsing (eg. instagram).
Indexing can get quite slow if reading the entire POST request, and is unnecessary.
Supersedes #482, truncating POST request at read time.
It is unlikely that such large queries would ever match exactly and only ever match via fuzzy lookup.

Screenshots (if appropriate):

Types of changes

• Replay fix (fixes a replay specific issue)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added or updated tests to cover my changes.
• All new and existing tests passed.

[codecov]

Codecov Report

Merging #519 into develop will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           develop    #519      +/-   ##
==========================================
+ Coverage    88.09%   88.1%   +<.01%     
==========================================
  Files           64      64              
  Lines         7922    7926       +4     
  Branches      1420    1420              
==========================================
+ Hits          6979    6983       +4     
  Misses         607     607              
  Partials       336     336

Impacted Files	Coverage Δ
pywb/warcserver/inputrequest.py	85.56% <100%> (+0.3%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d819aa...8090955. Read the comment docs.