Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SQLite for report storage to address concurrency issues #67

Open
thatvideoshopguy opened this issue Apr 25, 2023 · 1 comment

Comments

@thatvideoshopguy
Copy link

Description

When running WPWatcher with multiple instances or scheduled scans, there is a risk of data corruption and failures due to concurrent access to the JSON report database file. This issue impacts users who have multiple scans running at the same time or who use a combination of cron jobs and the Linux service.

Proposed Solution

To address these concurrency issues, I propose migrating from JSON file storage to SQLite for storing scan reports. SQLite provides built-in support for concurrent access and ensures data consistency even when multiple processes access the database simultaneously. By using SQLite, WPWatcher can run multiple instances without the risk of data corruption due to concurrent access.

  • Update the DataBase class to use SQLite instead of a JSON file.
  • Create a table named reports in the SQLite database to store report data.
  • Modify the write and find methods to work with the SQLite database.
  • Ensure the updated implementation maintains backward compatibility with existing configurations and usage.
  • Test the updated implementation with multiple instances of WPWatcher running concurrently to confirm that the concurrency issues are resolved.

Benefit

By implementing SQLite for report storage, WPWatcher will be able to handle multiple instances and concurrent scans without the risk of data corruption or failures. This change will greatly enhance the reliability and robustness of the tool for users who need to perform frequent or overlapping scans.

@tristanlatr
Copy link
Owner

Yes good idea, though I don't want to bother writing SQL queries, so we could use SQLObject or something like this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants