feat(core): add sqlite db for nx

[FrozenPandaz]

Current Behavior

Nx has some persistent storage managed as separate files on disk. For example, the local cache queries the file system for existing directories.

Expected Behavior

Nx has a new more performant persistent storage via SQLite database. The db is used for the following purposes now:

1. Storing task details of different hashes (This serves as reference for other tables to get more information about a hash)
2. Storing a record of cached artifacts
3. Storing a history of tasks which have run

The cache in particular has the following benefits:

• It's faster, YMMV but it's definitely faster because it writes and reads less from disk.
• It's able to track access of different cached artifacts
• It purges cached artifacts more intelligently by looking at when artifacts were last ACCESSED rather than when they were CREATED. This will also eliminate cache misses due to the cached artifacts being purged simply because they were CREATED.

This is opt-in for now, under NX_DB_CACHE=true

Related Issue(s)

Fixes #

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Updated (UTC)
nx-dev	⬜️ Ignored (Inspect)	Visit Preview	Aug 21, 2024 5:10pm

[nx-cloud]

☁️ Nx Cloud Report

CI is running/has finished running commands for commit f05e681. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

---

✅ Successfully ran 4 targets

• nx affected --targets=lint,test,build,e2e,e2e-ci --base=7e158385fdcd1c52fb5348354d6dbe267f03159f --head=f05e68154c1ad6b7977bda7648efe131b70556ae --parallel=3
• nx run-many -t check-imports check-commit check-lock-files check-codeowners documentation --parallel=1 --no-dte
• nx-cloud record -- nx format:check --base=7e158385fdcd1c52fb5348354d6dbe267f03159f --head=f05e68154c1ad6b7977bda7648efe131b70556ae
• nx documentation --no-dte

---

Sent with 💌 from NxCloud.

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-2baa5df

Release details	📑
Published version	0.0.0-pr-26891-2baa5df
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	2baa5df
Workflow run	9965155095

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

Failed to publish a PR release of this pull request, triggered by @FrozenPandaz.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/9977820782

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-a1d375a

Release details	📑
Published version	0.0.0-pr-26891-a1d375a
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	a1d375a
Workflow run	9977820782

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-1cb1598

Release details	📑
Published version	0.0.0-pr-26891-1cb1598
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	1cb1598
Workflow run	9985709569

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-048ee5a

Release details	📑
Published version	0.0.0-pr-26891-048ee5a
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	048ee5a
Workflow run	9993019041

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

Failed to publish a PR release of this pull request, triggered by @FrozenPandaz.
See the failed workflow run at: https://github.com/nrwl/nx/actions/runs/9993344345

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-a7e4733

Release details	📑
Published version	0.0.0-pr-26891-a7e4733
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	a7e4733
Workflow run	10372914229

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-0a70597

Release details	📑
Published version	0.0.0-pr-26891-0a70597
Triggered by	@FrozenPandaz
Branch	db-cache
Commit	0a70597
Workflow run	10409362584

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

🐳 We have a release for that!

This PR has a release associated with it. You can try it out using this command:

npx [email protected] my-workspace

Or just copy this version and use it in your own command:

0.0.0-pr-26891-cf56f7b

Release details	📑
Published version	0.0.0-pr-26891-cf56f7b
Triggered by	@Cammisuli
Branch	db-cache
Commit	cf56f7b
Workflow run	10492552663

To request a new release for this pull request, mention someone from the Nx team or the @nrwl/nx-pipelines-reviewers.

[github-actions]

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.