-
Notifications
You must be signed in to change notification settings - Fork 866
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf: Optimize xrefmap.json file deserialization performance #9824
Conversation
a123a30
to
1279c7a
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #9824 +/- ##
==========================================
+ Coverage 74.31% 79.02% +4.71%
==========================================
Files 536 536
Lines 23189 23195 +6
Branches 4056 4029 -27
==========================================
+ Hits 17234 18331 +1097
+ Misses 4853 3740 -1113
- Partials 1102 1124 +22 ☔ View full report in Codecov by Sentry. |
1279c7a
to
062ab3c
Compare
Is file-based database (like sqlite) possible as an option? query might be faster. |
I though faster xrefmap loading is important. So it might be better to use file-based database or some key-value storage as suggested. Some of the required characteristics are.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice, looking forward to the switch to S.T.J.
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [docfx](https://togithub.com/dotnet/docfx) | `2.75.3` -> `2.76.0` | [![age](https://developer.mend.io/api/mc/badges/age/nuget/docfx/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/nuget/docfx/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/nuget/docfx/2.75.3/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/nuget/docfx/2.75.3/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>dotnet/docfx (docfx)</summary> ### [`v2.76.0`](https://togithub.com/dotnet/docfx/releases/tag/v2.76.0) <!-- Release notes generated using configuration in .github/release.yml at main --> #### What's Changed ##### 🎉 New Features - feat: Add DefineConstants supports for source file based build by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9707](https://togithub.com/dotnet/docfx/pull/9707) - feat: Add additional layout options to modern template by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9737](https://togithub.com/dotnet/docfx/pull/9737) - feat: Add pdfPrintBackground setting by [@​cjlotz](https://togithub.com/cjlotz) in [https://github.com/dotnet/docfx/pull/9742](https://togithub.com/dotnet/docfx/pull/9742) - feat: Update roslyn packages by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9753](https://togithub.com/dotnet/docfx/pull/9753) - feat: Publish nightly pre-release packages by [@​yufeih](https://togithub.com/yufeih) in [https://github.com/dotnet/docfx/pull/9788](https://togithub.com/dotnet/docfx/pull/9788) - feat: Add DLL references support for file-based API metadata generation by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9825](https://togithub.com/dotnet/docfx/pull/9825) - feat: Support DOCFX_SOURCE_REPOSITORY_URL by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9759](https://togithub.com/dotnet/docfx/pull/9759) - feat: Add feature to configure markdig extension by docfx.json configs. by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9820](https://togithub.com/dotnet/docfx/pull/9820) - feat: Support for all emphasis supported by MarkDig by [@​EMaderbacher](https://togithub.com/EMaderbacher) in [https://github.com/dotnet/docfx/pull/9762](https://togithub.com/dotnet/docfx/pull/9762) ##### 🐞 Bug Fixes - fix: `docfx download` command behaviors by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9721](https://togithub.com/dotnet/docfx/pull/9721) - fix: Resolved .NET SDK version problem on source-based API metadata generation by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9713](https://togithub.com/dotnet/docfx/pull/9713) - fix: Don't overwrite `_disabledToc` when metadata is explicitly defined by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9741](https://togithub.com/dotnet/docfx/pull/9741) - fix: Outputted PDF displays garbled characters when name contains non-ASCII chars by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9769](https://togithub.com/dotnet/docfx/pull/9769) - fix: Move script tags into head in modern template by [@​fuzzykiller](https://togithub.com/fuzzykiller) in [https://github.com/dotnet/docfx/pull/9768](https://togithub.com/dotnet/docfx/pull/9768) - fix: Broken examples problems when using inheritdoc tag by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9754](https://togithub.com/dotnet/docfx/pull/9754) - fix: NullReferenceException occurred when both include and exclude are null by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9786](https://togithub.com/dotnet/docfx/pull/9786) - fix: Change JsonSchema EvaluationOptions instance to shared object by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9807](https://togithub.com/dotnet/docfx/pull/9807) - fix: Bug fix Invalid documentation on type parameter [#​9755](https://togithub.com/dotnet/docfx/issues/9755) by [@​soniyaprasad77](https://togithub.com/soniyaprasad77) in [https://github.com/dotnet/docfx/pull/9806](https://togithub.com/dotnet/docfx/pull/9806) ##### 🚀 Performance Improvements - perf: Optimize xrefmap.json file deserialization performance by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9824](https://togithub.com/dotnet/docfx/pull/9824) ##### 🔧 Engineering - chore: percy snapshot by [@​yufeih](https://togithub.com/yufeih) in [https://github.com/dotnet/docfx/pull/9747](https://togithub.com/dotnet/docfx/pull/9747) - chore: Add detailed error message for specific exceptions by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9746](https://togithub.com/dotnet/docfx/pull/9746) - chore: update pdfpig nuget package version by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9803](https://togithub.com/dotnet/docfx/pull/9803) ##### 📄 Documentation - docs: Split docfx CLI reference docs based on commands by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9708](https://togithub.com/dotnet/docfx/pull/9708) - docs: Update markdown documentation regarding MarkDig extension by [@​tbolon](https://togithub.com/tbolon) in [https://github.com/dotnet/docfx/pull/9810](https://togithub.com/dotnet/docfx/pull/9810) - docs: Update `docfx.json` config reference document by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9813](https://togithub.com/dotnet/docfx/pull/9813) - docs: Correct `docfx template` page title by [@​peitschie](https://togithub.com/peitschie) in [https://github.com/dotnet/docfx/pull/9812](https://togithub.com/dotnet/docfx/pull/9812) - docs: Add a warning about relative path resolution. by [@​voroninp](https://togithub.com/voroninp) in [https://github.com/dotnet/docfx/pull/9828](https://togithub.com/dotnet/docfx/pull/9828) #### New Contributors - [@​EMaderbacher](https://togithub.com/EMaderbacher) made their first contribution in [https://github.com/dotnet/docfx/pull/9762](https://togithub.com/dotnet/docfx/pull/9762) - [@​fuzzykiller](https://togithub.com/fuzzykiller) made their first contribution in [https://github.com/dotnet/docfx/pull/9768](https://togithub.com/dotnet/docfx/pull/9768) - [@​soniyaprasad77](https://togithub.com/soniyaprasad77) made their first contribution in [https://github.com/dotnet/docfx/pull/9806](https://togithub.com/dotnet/docfx/pull/9806) - [@​tbolon](https://togithub.com/tbolon) made their first contribution in [https://github.com/dotnet/docfx/pull/9810](https://togithub.com/dotnet/docfx/pull/9810) - [@​peitschie](https://togithub.com/peitschie) made their first contribution in [https://github.com/dotnet/docfx/pull/9812](https://togithub.com/dotnet/docfx/pull/9812) - [@​voroninp](https://togithub.com/voroninp) made their first contribution in [https://github.com/dotnet/docfx/pull/9828](https://togithub.com/dotnet/docfx/pull/9828) **Full Changelog**: dotnet/docfx@v2.75.3...v2.76.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 9pm,before 6am" in timezone Europe/Zurich, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/buehler/dotnet-operator-sdk). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [docfx](https://togithub.com/dotnet/docfx) | `2.75.3` -> `2.76.0` | [![age](https://developer.mend.io/api/mc/badges/age/nuget/docfx/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/nuget/docfx/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/nuget/docfx/2.75.3/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/nuget/docfx/2.75.3/2.76.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>dotnet/docfx (docfx)</summary> ### [`v2.76.0`](https://togithub.com/dotnet/docfx/releases/tag/v2.76.0) <!-- Release notes generated using configuration in .github/release.yml at main --> #### What's Changed ##### 🎉 New Features - feat: Add DefineConstants supports for source file based build by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9707](https://togithub.com/dotnet/docfx/pull/9707) - feat: Add additional layout options to modern template by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9737](https://togithub.com/dotnet/docfx/pull/9737) - feat: Add pdfPrintBackground setting by [@​cjlotz](https://togithub.com/cjlotz) in [https://github.com/dotnet/docfx/pull/9742](https://togithub.com/dotnet/docfx/pull/9742) - feat: Update roslyn packages by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9753](https://togithub.com/dotnet/docfx/pull/9753) - feat: Publish nightly pre-release packages by [@​yufeih](https://togithub.com/yufeih) in [https://github.com/dotnet/docfx/pull/9788](https://togithub.com/dotnet/docfx/pull/9788) - feat: Add DLL references support for file-based API metadata generation by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9825](https://togithub.com/dotnet/docfx/pull/9825) - feat: Support DOCFX_SOURCE_REPOSITORY_URL by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9759](https://togithub.com/dotnet/docfx/pull/9759) - feat: Add feature to configure markdig extension by docfx.json configs. by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9820](https://togithub.com/dotnet/docfx/pull/9820) - feat: Support for all emphasis supported by MarkDig by [@​EMaderbacher](https://togithub.com/EMaderbacher) in [https://github.com/dotnet/docfx/pull/9762](https://togithub.com/dotnet/docfx/pull/9762) ##### 🐞 Bug Fixes - fix: `docfx download` command behaviors by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9721](https://togithub.com/dotnet/docfx/pull/9721) - fix: Resolved .NET SDK version problem on source-based API metadata generation by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9713](https://togithub.com/dotnet/docfx/pull/9713) - fix: Don't overwrite `_disabledToc` when metadata is explicitly defined by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9741](https://togithub.com/dotnet/docfx/pull/9741) - fix: Outputted PDF displays garbled characters when name contains non-ASCII chars by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9769](https://togithub.com/dotnet/docfx/pull/9769) - fix: Move script tags into head in modern template by [@​fuzzykiller](https://togithub.com/fuzzykiller) in [https://github.com/dotnet/docfx/pull/9768](https://togithub.com/dotnet/docfx/pull/9768) - fix: Broken examples problems when using inheritdoc tag by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9754](https://togithub.com/dotnet/docfx/pull/9754) - fix: NullReferenceException occurred when both include and exclude are null by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9786](https://togithub.com/dotnet/docfx/pull/9786) - fix: Change JsonSchema EvaluationOptions instance to shared object by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9807](https://togithub.com/dotnet/docfx/pull/9807) - fix: Bug fix Invalid documentation on type parameter [#​9755](https://togithub.com/dotnet/docfx/issues/9755) by [@​soniyaprasad77](https://togithub.com/soniyaprasad77) in [https://github.com/dotnet/docfx/pull/9806](https://togithub.com/dotnet/docfx/pull/9806) ##### 🚀 Performance Improvements - perf: Optimize xrefmap.json file deserialization performance by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9824](https://togithub.com/dotnet/docfx/pull/9824) ##### 🔧 Engineering - chore: percy snapshot by [@​yufeih](https://togithub.com/yufeih) in [https://github.com/dotnet/docfx/pull/9747](https://togithub.com/dotnet/docfx/pull/9747) - chore: Add detailed error message for specific exceptions by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9746](https://togithub.com/dotnet/docfx/pull/9746) - chore: update pdfpig nuget package version by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9803](https://togithub.com/dotnet/docfx/pull/9803) ##### 📄 Documentation - docs: Split docfx CLI reference docs based on commands by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9708](https://togithub.com/dotnet/docfx/pull/9708) - docs: Update markdown documentation regarding MarkDig extension by [@​tbolon](https://togithub.com/tbolon) in [https://github.com/dotnet/docfx/pull/9810](https://togithub.com/dotnet/docfx/pull/9810) - docs: Update `docfx.json` config reference document by [@​filzrev](https://togithub.com/filzrev) in [https://github.com/dotnet/docfx/pull/9813](https://togithub.com/dotnet/docfx/pull/9813) - docs: Correct `docfx template` page title by [@​peitschie](https://togithub.com/peitschie) in [https://github.com/dotnet/docfx/pull/9812](https://togithub.com/dotnet/docfx/pull/9812) - docs: Add a warning about relative path resolution. by [@​voroninp](https://togithub.com/voroninp) in [https://github.com/dotnet/docfx/pull/9828](https://togithub.com/dotnet/docfx/pull/9828) #### New Contributors - [@​EMaderbacher](https://togithub.com/EMaderbacher) made their first contribution in [https://github.com/dotnet/docfx/pull/9762](https://togithub.com/dotnet/docfx/pull/9762) - [@​fuzzykiller](https://togithub.com/fuzzykiller) made their first contribution in [https://github.com/dotnet/docfx/pull/9768](https://togithub.com/dotnet/docfx/pull/9768) - [@​soniyaprasad77](https://togithub.com/soniyaprasad77) made their first contribution in [https://github.com/dotnet/docfx/pull/9806](https://togithub.com/dotnet/docfx/pull/9806) - [@​tbolon](https://togithub.com/tbolon) made their first contribution in [https://github.com/dotnet/docfx/pull/9810](https://togithub.com/dotnet/docfx/pull/9810) - [@​peitschie](https://togithub.com/peitschie) made their first contribution in [https://github.com/dotnet/docfx/pull/9812](https://togithub.com/dotnet/docfx/pull/9812) - [@​voroninp](https://togithub.com/voroninp) made their first contribution in [https://github.com/dotnet/docfx/pull/9828](https://togithub.com/dotnet/docfx/pull/9828) **Full Changelog**: dotnet/docfx@v2.75.3...v2.76.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 9pm,before 6am" in timezone Europe/Zurich, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/buehler/dotnet-operator-sdk). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4yNjkuMiIsInVwZGF0ZWRJblZlciI6IjM3LjI2OS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiJ9--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
…9824) perf: optimize xrefmap.json file deserialization
What's changed in this PR
When XrefMap is loaded from local file or network.
The current implementation using
YamlDotNet
's deserialize method.For JSON file deserialization.
It seems to be faster to use
Newtonsoft.json
for JSON deserialization.Benchmark