Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[docs] Add documentation on fileset migrations under operational guides #2630

Merged
merged 1 commit into from
Sep 17, 2020
Merged

[docs] Add documentation on fileset migrations under operational guides #2630

merged 1 commit into from
Sep 17, 2020

Conversation

nbroyles
Copy link
Collaborator

What this PR does / why we need it:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

Does this PR require updating code package or user-facing documentation?:

@ryanhall07 ryanhall07 self-requested a review September 13, 2020 21:42
ryanhall07
ryanhall07 reviewed Sep 13, 2020
View reviewed changes
docs/operational_guide/fileset_migrations.md
## Migration Process
Migrations are executed during the initial stages of the bootstrap. When enabled, the filesystem bootstrapper will scan for filesets that should be migrated and migrate any filesets found. A fileset is determined to be in need of a migration based on the `MajorVersion` and `MinorVersion` found in the info file. If `MajorVersion.MinorVersion` is less than the target migration version, then that fileset will be scheduled for migration.

If migrations are deemed necessary, the bootstrap process pauses until the migrations complete. If a failure occurs while migrating, an error is logged and the process continues. If a fileset is not successfully migrated, the non-migrated version of the fileset is used going forward. In other words, whether they succeed or fail, migrations should leave filesets in a good state.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems unfortunate that migrations slow down the bootstrapping process, especially since it seems like the migration isn't required for boot up (i.e if it fails, the old fs is used). curious if there is any future plan to change this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The concurrency knobs help a bit to make them happen pretty quickly, but, yeah, not ideal for sure. This is something we can revisit if it starts to become a problem and we have more migrations. Given that m3 has been around for a while now and we're just implementing something like this, I'm inclined to wait and see before investing more time in optimizations.

@nbroyles nbroyles merged commit 0ef7aba into master Sep 17, 2020
@nbroyles nbroyles deleted the nb/mig-docs branch September 17, 2020 16:33
ChrisChinchilla pushed a commit that referenced this pull request Oct 5, 2020
Squashed commit of the following:
00c8613 
commit 4f3778dded83ad4d81aaeaad62608f6c7a0f9461
Author: ChrisChinchilla <[email protected]>
Date:   Mon Oct 5 12:53:40 2020 +0200

    fix code inclusion

    Signed-off-by: ChrisChinchilla <[email protected]>

commit daefb435243ed98041744c520d52ac4a92421694
Author: ChrisChinchilla <[email protected]>
Date:   Mon Oct 5 12:53:34 2020 +0200

    Add redirects

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 612cfbb319b30177501530f9cd7bdd626ea6b107
Author: ChrisChinchilla <[email protected]>
Date:   Mon Oct 5 12:09:40 2020 +0200

    Remove versioning for now

    Signed-off-by: ChrisChinchilla <[email protected]>

commit cda48e18dce956a4e684dd4d6d7943df808e6846
Author: ChrisChinchilla <[email protected]>
Date:   Fri Oct 2 18:09:35 2020 +0200

    Switch netlify directory

    Signed-off-by: ChrisChinchilla <[email protected]>

commit c5d0b1decc95d042a565cbe97d25c9eda94549a9
Author: ChrisChinchilla <[email protected]>
Date:   Fri Oct 2 18:00:34 2020 +0200

    Move theme to module

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 855d8c7af1bf45978ed180469ecb1b753ec1abdb
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 14:59:14 2020 +0200

    Netlify dev

    Signed-off-by: ChrisChinchilla <[email protected]>

commit a8a8eda0f2f3b8fa1b3aefa2316942ff692dc8e5
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 14:03:34 2020 +0200

    Update Hugo version

    Signed-off-by: ChrisChinchilla <[email protected]>

commit f19ffadb7caad18663dad99ec5230e8357125954
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 13:54:41 2020 +0200

    Random file to fix odd git issues

    Signed-off-by: ChrisChinchilla <[email protected]>

commit a6413db7891f2f741510e4b36300e3cb93904f1d
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 13:11:42 2020 +0200

    Update versions

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 9919f004633ace0f88bb9cb721a6373416f28781
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 12:45:25 2020 +0200

    Convert docs theme to module

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 3a1e013338b6671eb601df3f5cbdd89d770ca171
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 12:27:31 2020 +0200

    Remove subtree again, not working

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 282ca2d870eb656ba15e20ade3d8aec6493523fe
Author: ChrisChinchilla <[email protected]>
Date:   Thu Oct 1 12:20:11 2020 +0200

    Add versions to config

    Signed-off-by: ChrisChinchilla <[email protected]>

commit 73f03d34990b740d5bb1c57df1c5fbb328d0f701
Author: ChrisChinchilla <[email protected]>
Date:   Wed Sep 30 18:29:35 2020 +0200

    Consolidate all commits

commit f2ebf5c
Author: teddywahle <[email protected]>
Date:   Mon Sep 21 12:33:37 2020 -0700

    [query] Implemented the Graphite `integralByInterval` function  (#2596)

commit a66fb7d
Author: arnikola <[email protected]>
Date:   Mon Sep 21 14:33:57 2020 -0400

    [dbnode] Tile iterators for wide aggregations (#2646)

commit 9ea5682
Author: teddywahle <[email protected]>
Date:   Sun Sep 20 21:50:31 2020 -0700

    [query] Implemented the Graphite `divideSeriesLists` function (#2585)

commit 35cac59
Author: Rob Skillington <[email protected]>
Date:   Mon Sep 21 00:21:58 2020 -0400

    [coordinator] Update OpenAPI specs for namespace update endpoint (#2629)

commit ef83ec4
Author: Rob Skillington <[email protected]>
Date:   Sun Sep 20 21:57:26 2020 -0400

    [changelog] Add changelog for 0.15.15 (#2649)

commit 091f833
Author: Rob Skillington <[email protected]>
Date:   Fri Sep 18 11:30:57 2020 -0400

    [coordinator] Allow configuration of tag validation (#2647)

commit 3476b4e
Author: Gediminas Guoba <[email protected]>
Date:   Fri Sep 18 17:24:09 2020 +0300

    [dbnode] Streaming writer (#2618)

    * [dbnode] Large tiles writer

    * minor refactorings

    * minor refactoring

    * Skip tagsEncoder, use encodedTags directly

    * Rename LargeTilesWriter to StreamingWriter

    * Add FIXME wrt stegment.Tail.Finalize

    * Address PR feedback

    Co-authored-by: Linas Medziunas <[email protected]>
    Co-authored-by: Linas Medžiūnas <[email protected]>

commit 88164cf
Author: Ryan Hall <[email protected]>
Date:   Thu Sep 17 16:38:04 2020 -0700

    Only read the commit log once during bootstrapping (#2645)

    * Only read the commit log once during bootstrapping

    A recent refactoring of cold writes ( #2508) introduced a regression that increases the
    chances the commit log is read twice while bootstrapping. The referenced PR changed the commitlog
    bootstrapper to read all requested time ranges, even if a range had been fulfilled by a previous
    bootstrapper. This was necessary since the commitlog may have cold writes that were never commmited
    to a fileset. The fileystem bootstrapper would report a time range as fulfilled, but might be missing
    cold writes only in the commit log.

    It should be noted this bug was always theoretically possible, but unlikely since the commitlog bootstrapper
    typically wouldn't run in the first pass (cold time ranges) since the filesystem would fulfill all cold ranges
    and short circuit the first pass of the boostrapper.

    This change only reads the commit log on the first pass of the boostrapper and caches the result to skip reading
    it in subsequent passes. It doesn't actually matter which pass we read the commit log, the first was just chosen
    arbitrarily.

    Other attempts at fixing this bug attempted to disable the entire commit log bootstrapper during a pass, but that's
    not possible since the commit log bootstrapper is actually 2 bootstrappers in one, both the the commit log and snapshot
    files. To minimize the refactoring changes we still want to only read the snapshot files of the requested ranges.

commit 3d2915f
Author: nate <[email protected]>
Date:   Thu Sep 17 13:03:48 2020 -0400

    [dbnode] Make caching after block retrieval a configuration option (#2613)

commit 0ef7aba
Author: nate <[email protected]>
Date:   Thu Sep 17 12:33:53 2020 -0400

    [docs] Add documentation on fileset migrations (#2630)

Signed-off-by: ChrisChinchilla <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ryanhall07 ryanhall07 ryanhall07 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nbroyles @ryanhall07