Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 BUG: Wrong local KV binding since [email protected] #4011

Closed
SimonDegraeve opened this issue Sep 22, 2023 · 4 comments
Closed

🐛 BUG: Wrong local KV binding since [email protected] #4011

SimonDegraeve opened this issue Sep 22, 2023 · 4 comments
Labels
bug Something that isn't working miniflare Relating to Miniflare

Comments

@SimonDegraeve
Copy link
Contributor

Which Cloudflare product(s) does this pertain to?

KV, Wrangler core

What version(s) of the tool(s) are you using?

3.9.0 [Wrangler]

What version of Node are you using?

No response

What operating system are you using?

Mac

Describe the Bug

When I run wrangler dev I used to be able to save KV values locally in the cache using the following path
[cache-dir]/v3/kv/[kv-namespace-id]/db.sqlite

Since [email protected] the KV values are stored at
[cache-dir]/v3/kv/miniflare-KVNamespaceObject/6b57c7448544d428a1bfbbc5130e97beeaeeaa2bcce16f2018ee2f282ef60644.sqlite

How is the db name generated ? How can I reliably access the cache for a given namespace ?

I am asking those questions because I work for a big tech company and we are heavily relying on CloudFlare for our frontend pipeline. We used to bundle and serve our apps with fab (https://fab.dev/), however since the project has been abandoned we replicated the same features on our own.
So we have a frontend repo with CI building a frontend app with webpack, we bundle the output with our own custom worker replicating fab deployment logic with hono (https://hono.dev/). Meaning the files generated by webpack are stored in a KV store and serve by a Worker.
During development, we run e2e tests, hosted in a different repo by using CI triggers. The e2e tests repo CI will download artifacts from our frontend repo, the bundle we generated with worker and assets, and serve it locally inside the container using wrangler dev. The custom serve method we created would unpack the bundle and move assets inside [cache-dir]/v3/kv/[kv-namespace-id]/db.sqlite before starting wrangler, and then the KV binding is able to serve our assets locally just like it would in prod or from a squad environment.

Sorry for the long message, available for further questions if you have any.

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

No response

@SimonDegraeve SimonDegraeve added the bug Something that isn't working label Sep 22, 2023
@github-project-automation github-project-automation bot moved this to Untriaged in workers-sdk Sep 22, 2023
@SimonDegraeve
Copy link
Contributor Author

cc @mrbbot You might be able to answers those questions since you implemented the changes. Thank you in advance.

@admah admah added the miniflare Relating to Miniflare label Sep 22, 2023
@sdarnell
Copy link

https://discord.com/channels/595317990191398933/891052295410835476/1156147705282834484
MrBBot wrote: As of v3, Miniflare’s persistent storage layout should no longer be considered a public interface. I’d encourage you to use the wrangler d1 execute --local and wrangler kv:key --local commands to access and mutate data locally.
If you’re interested though, that big hex string is the ID of the D1 database’s Durable Object. Essentially DurableObjectNamespace#idFromName(databaseName). See cloudflare/miniflare#656 for more details.

@SimonDegraeve
Copy link
Contributor Author

Thank you for the answer. Unfortunately I cannot use the wrangler kv:key put --local as it does not seems to support blob, I will find another way.

@github-project-automation github-project-automation bot moved this from Untriaged to Done in workers-sdk Sep 27, 2023
@mrbbot
Copy link
Contributor

mrbbot commented Sep 28, 2023

@SimonDegraeve, another option would be to use Miniflare’s API and the getKVNamespace()/getD1Database() methods: https://github.com/cloudflare/miniflare/blob/tre/packages/miniflare/README.md#class-miniflare. These give you the same bindings API as inside a Worker, allowing you to access/mutate data programmatically.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that isn't working miniflare Relating to Miniflare
Projects
None yet
Development

No branches or pull requests

4 participants