Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

plugin-ext: Change PluginsKeyValueStorage to not continuously access the disk #12236

Merged
merged 17 commits into from
May 25, 2023
Merged

plugin-ext: Change PluginsKeyValueStorage to not continuously access the disk #12236

merged 17 commits into from
May 25, 2023

Commits on May 1, 2023

  1. [plugin-ext] Add file locking to avoid race condition writing to glob… 

    …al plugin storage file

Fixes #8384

Added the use of a semaphore to lock access to the global plugin storage file, to avoid it
being accessed in a disorderly fashion, causing potential corruption.

As per the issue discussion, it's possible that there is also a potential race condition
caused by node's write function, not flushing its buffers to disk, but in this specific case,
it was obvious that parallel calls to set() and get() were the main cause of the problems.

Signed-off-by: Marc Dumais <[email protected]>
    @marcdumais-work @paul-marechal
    marcdumais-work authored and paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    41eecc4 View commit details
    Browse the repository at this point in the history

  2. use async-mutex + cleanup

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    4092d0c View commit details
    Browse the repository at this point in the history

  3. cleanup postConstruct 

    Inversify 4 doesn't really support async postConstruct methods.
    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    ee6bf49 View commit details
    Browse the repository at this point in the history

  4. make globalStateFileLock process-global 

    Also put a guard around the folder initialization before writing to the
`global-state.json` file.
    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    816dcae View commit details
    Browse the repository at this point in the history

  5. handle guarding access to various files 

    Turns out the `PluginsKeyValueStorage` implementation handles more than
just the one `global-state.json` file, so we need to make sure to guard
against concurrent access to these files too.
    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    3ba222a View commit details
    Browse the repository at this point in the history

  6. fix naming

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    ea2b16d View commit details
    Browse the repository at this point in the history

  7. fix typo

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    cb0cd71 View commit details
    Browse the repository at this point in the history

  8. [plugins-key-value-storage] add minimal tests 

    Signed-off-by: Marc Dumais <[email protected]>
    @marcdumais-work @paul-marechal
    marcdumais-work authored and paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    485c444 View commit details
    Browse the repository at this point in the history

  9. core: introduce FileSystemLocking 

    The native Node file system APIs are powerful but low level: Nothing
prevents bad code from concurrently writing to the same file.

`FileSystemLocking` is meant to be a central component for any method
handling files and wanting to prevent concurrent access by other parts
of the framework.
    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    7a7eaee View commit details
    Browse the repository at this point in the history

  10. add missing injectable decorator

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    857e1c3 View commit details
    Browse the repository at this point in the history

  11. fix tests

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    e2e46df View commit details
    Browse the repository at this point in the history

  12. use delayed file sync 

    Instead of reading/writing on each get/set, use an in-memory cache and
synchronize the data on disk every so often.
    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    8e1686c View commit details
    Browse the repository at this point in the history

  13. everything private

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    66680ae View commit details
    Browse the repository at this point in the history

  14. fix linter

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    ca2e324 View commit details
    Browse the repository at this point in the history

  15. get rid of proper-lockfile

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    655a942 View commit details
    Browse the repository at this point in the history

  16. remove left over

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    e8dfd42 View commit details
    Browse the repository at this point in the history

  17. cleanup

    @paul-marechal
    paul-marechal committed May 1, 2023
    Configuration menu
    Copy the full SHA
    4675ccb View commit details
    Browse the repository at this point in the history