Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(db): Consider hot and cold dbs archivals #10960

Merged
merged 1 commit into from
Apr 5, 2024

Conversation

posvyatokum
Copy link
Member

This reflect my understanding of DBKinds. Archive, Hot, and Cold dbs are only present in archival nodes. RPC is the only non-archival db kind.
This change will make it harder for people to open Hot storage as a single DB in non-archival mode. They will first have to change DBKind of hot db to RPC using the database tool of neard. Then, if they want to open this db as part of split storage again, they will need to change DBKind back using the same tool.

This whole part of code is due to a refactoring. I will think about the best project for this tasks on Monday.

@posvyatokum posvyatokum requested a review from a team as a code owner April 5, 2024 13:10
@posvyatokum posvyatokum added this pull request to the merge queue Apr 5, 2024
Copy link

codecov bot commented Apr 5, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 71.57%. Comparing base (f42345a) to head (ffef08c).
Report is 1 commits behind head on master.

Files Patch % Lines
core/store/src/lib.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10960      +/-   ##
==========================================
+ Coverage   67.13%   71.57%   +4.44%     
==========================================
  Files         746      758      +12     
  Lines      150860   152127    +1267     
  Branches   150860   152127    +1267     
==========================================
+ Hits       101274   108890    +7616     
+ Misses      45288    38737    -6551     
- Partials     4298     4500     +202     
Flag Coverage Δ
backward-compatibility 0.24% <0.00%> (?)
db-migration 0.24% <0.00%> (?)
genesis-check 1.42% <0.00%> (?)
integration-tests 37.17% <0.00%> (?)
linux 70.03% <0.00%> (+3.53%) ⬆️
linux-nightly 71.07% <0.00%> (+4.46%) ⬆️
macos 54.55% <0.00%> (?)
pytests 1.65% <0.00%> (?)
sanity-checks 1.44% <0.00%> (?)
unittests 67.19% <0.00%> (+0.06%) ⬆️
upgradability 0.29% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Merged via the queue into near:master with commit 4cea196 Apr 5, 2024
27 of 29 checks passed
@posvyatokum posvyatokum deleted the fix-archive branch April 5, 2024 13:50
mooori pushed a commit to mooori/nearcore that referenced this pull request Apr 16, 2024
<p>This PR was automatically created by Snyk using the credentials of a
real user.</p><br /><h3>Snyk has created this PR to upgrade react-router
from 6.17.0 to 6.18.0.</h3>

:information_source: Keep your dependencies up-to-date. This makes it
easier to fix existing vulnerabilities and to more quickly identify and
fix newly disclosed vulnerabilities when they affect your project.
<hr/>

- The recommended version is **3 versions** ahead of your current
version.
- The recommended version was released **21 days ago**, on 2023-10-31.


<details>
<summary><b>Release notes</b></summary>
<br/>
  <details>
    <summary>Package name: <b>react-router</b></summary>
    <ul>
      <li>
<b>6.18.0</b> - <a
href="https://snyk.io/redirect/github/remix-run/react-router/releases/tag/react-router-native%406.18.0">2023-10-31</a></br><p>[email protected]</p>
      </li>
      <li>
<b>6.18.0-pre.1</b> - <a
href="https://snyk.io/redirect/github/remix-run/react-router/releases/tag/react-router-native%406.18.0-pre.1">2023-10-30</a></br><p>[email protected]</p>
      </li>
      <li>
        <b>6.18.0-pre.0</b> - 2023-10-26
      </li>
      <li>
        <b>6.17.0</b> - 2023-10-16
      </li>
    </ul>
from <a
href="https://snyk.io/redirect/github/remix-run/react-router/releases">react-router
GitHub release notes</a>
  </details>
</details>


<details>
  <summary><b>Commit messages</b></summary>
  </br>
  <details>
    <summary>Package name: <b>react-router</b></summary>
    <ul>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/667f9360759925adfc0e052b1c56e43447521f54">667f936</a>
chore: Update version for release (near#10981)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/3c6e27c14668efec276a5b1cacc2572cc5a76d6c">3c6e27c</a>
Exit prerelease mode</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/67af262448f66e08d794e2418699f7c1a7392a52">67af262</a>
chore: Update version for release (pre) (near#10978)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/150028859ef8c3e116dc840c8bde043cc3060051">1500288</a>
Updates for v7_fetcherPersist post-processing logic (near#10977)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/cacc90b83cb2aa0f4b60f6ec5454f237633d9478">cacc90b</a>
chore: Update version for release (pre) (near#10966)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/77402de4cb6d4a8468def168a96c9992d54cbbdb">77402de</a>
Enter prerelease mode</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/1a8265ccfce0005ba588288650707a1eba39afd8">1a8265c</a>
Merge branch &#x27;main&#x27; into release-next</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/19af0cf7652f309a6a68c3c698f8eb36e733ce72">19af0cf</a>
Add future.v7_fetcherPersist flag (near#10962)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/cb2d911d20b6d3268cde61e5828097ce5166f05c">cb2d911</a>
Add fetcher data layer (near#10961)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/c0dbcd256df3d58cb07713e947ab8014886e76c7">c0dbcd2</a>
Add useFetcher(key) and &lt;Form navigate&#x3D;{false}&gt; (near#10960)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/805924dd2fd9003a3e9cb5e3534778e61ede6bbf">805924d</a>
Revert &quot;Ensure Form contains splat portion of pathname when no
action is specified (near#10933)&quot; (near#10965)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/677d6c8915ef5f834700bcf5cc5ed90a6c55b200">677d6c8</a>
Support optional path segments in &#x60;matchPath&#x60; (near#10768)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/a23f017ec16b1eb3c622a5549645227066c49e3a">a23f017</a>
docs: Clarify fetcher load revalidation behavior</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/908a40a25382612b869638664e19a4aa7a977e53">908a40a</a>
Ensure Form contains splat portion of pathname when no action is
specified (near#10933)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/2041d9c7ebc8575ff99ed127d3c9ab8b79347b1e">2041d9c</a>
chore: sort contributors list</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/a71b4e296776ef3b4fcada1f2a0b9fb52f03cfed">a71b4e2</a>
Merge branch &#x27;release-next&#x27; into dev</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/779536cbfb25122264f9c89f2aef63ae2531f9fd">779536c</a>
Merge branch &#x27;release-next&#x27;</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/e2c0d828131d27f5e34b81327d30f7afd4ffea90">e2c0d82</a>
Split up router-test.ts (near#10929)</li>
<li><a
href="https://snyk.io/redirect/github/remix-run/react-router/commit/e74b935d4d162f89766f9c97be7797b6e9afa97c">e74b935</a>
Fix lint errors</li>
    </ul>

<a
href="https://snyk.io/redirect/github/remix-run/react-router/compare/edd9ad4957321cfb260cee21ad98aab2becfe250...667f9360759925adfc0e052b1c56e43447521f54">Compare</a>
  </details>
</details>
<hr/>

**Note:** *You are seeing this because you or someone else with access
to this repository has authorized Snyk to open upgrade PRs.*

For more information: <img
src="https://api.segment.io/v1/pixel/track?data=eyJ3cml0ZUtleSI6InJyWmxZcEdHY2RyTHZsb0lYd0dUcVg4WkFRTnNCOUEwIiwiYW5vbnltb3VzSWQiOiIzNWI0YmJiNy0wZTdiLTQ0MzAtOWI5YS1lZjhlODg4MjJkYWIiLCJldmVudCI6IlBSIHZpZXdlZCIsInByb3BlcnRpZXMiOnsicHJJZCI6IjM1YjRiYmI3LTBlN2ItNDQzMC05YjlhLWVmOGU4ODgyMmRhYiJ9fQ=="
width="0" height="0"/>

🧐 [View latest project
report](https://app.snyk.io/org/ecp88/project/98480bdc-d80b-4fd1-89d7-c4c56a706763?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr)

🛠 [Adjust upgrade PR
settings](https://app.snyk.io/org/ecp88/project/98480bdc-d80b-4fd1-89d7-c4c56a706763/settings/integration?utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr)

🔕 [Ignore this dependency or unsubscribe from future upgrade
PRs](https://app.snyk.io/org/ecp88/project/98480bdc-d80b-4fd1-89d7-c4c56a706763/settings/integration?pkg&#x3D;react-router&amp;utm_source&#x3D;github&amp;utm_medium&#x3D;referral&amp;page&#x3D;upgrade-pr#auto-dep-upgrades)

<!---
(snyk:metadata:{"prId":"35b4bbb7-0e7b-4430-9b9a-ef8e88822dab","prPublicId":"35b4bbb7-0e7b-4430-9b9a-ef8e88822dab","dependencies":[{"name":"react-router","from":"6.17.0","to":"6.18.0"}],"packageManager":"npm","type":"auto","projectUrl":"https://app.snyk.io/org/ecp88/project/98480bdc-d80b-4fd1-89d7-c4c56a706763?utm_source=github&utm_medium=referral&page=upgrade-pr","projectPublicId":"98480bdc-d80b-4fd1-89d7-c4c56a706763","env":"prod","prType":"upgrade","vulns":[],"issuesToFix":[],"upgrade":[],"upgradeInfo":{"versionsDiff":3,"publishedDate":"2023-10-31T14:24:16.706Z"},"templateVariants":[],"hasFixes":false,"isMajorUpgrade":false,"isBreakingChange":false,"priorityScoreList":[]})
--->

Co-authored-by: snyk-bot <[email protected]>
marcelo-gonzalez added a commit to marcelo-gonzalez/nearcore that referenced this pull request Jul 9, 2024
near#10906 made a change that will
create a database with store type set to `Hot` when starting a node
with archive=true with no existing database. Then near#10960
fixed a bug that would have made such a node crash when checking whether it's archival.
The result is that the assumption that we can proceed to garbase collect an archival database
if the type is `Hot` doesn't hold anymore, because nodes created like this but with no
"cold_store" set in the config will just have a hot database, but no cold one that data gets moved
to. This was causing block_sync_archival.py to fail because blocks were being garbage collected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants