Platform context fallbacks #4292
Comments
hmnd
changed the title
|
relatated #2304 ideally adapters come with their own dev setup where needed, eg miniflare for cf |
|
Yeah, this is one facet of a much larger topic that also includes #3535. The |
what do you think if developer specify the contents of platform? for example I need a database for my project. for Deno I can set database in platform object that can use Mongodb driver from deno.land like below: // adapter.deno.js
// PRODUCTION mode
// we have access to Deno,WebSocket,crypto.... in deno
const platform = {
db: {
get(id) {...},
insert(data) {...}
update(id, data) {...},
remove(id) {...}
}
}
export default platform;then I can import and use this file in server.js and for dev-mode I can simulate this functionality in handle (hooks.js) using in-memory or filesystem based database. // hooks.js
export async function handle ({event, resolve }) {
if(!platform.db) { // DEV mode
platform.db = {
get(id) {...},
insert(data) {...},
update(id, data) {...},
remove(id) {...}
}
}
....
}this way our sveltekit project is not dependent to cloudflare/deno and we can always move from one provider to another because the developer is the creator of this abstraction.
yes, this way we can manage platform object from svelte.config.js instead of hooks.js and we can provide different implementations for different adapters |
|
My proposal would be, having adapters optionally provide an I have managed fix this issue when using in hooks I use: //src/hooks/index.ts
export const interceptPlatform: Handle = async ({ event, resolve }) => {
event.platform = await cloudflareAdapterPlatform(event.platform)
return resolve(event)
}
...
export const handle: Handle = sequence(interceptPlatform, ...)Every request is intercepted by following code: //src/cloudflareAdapterPlatform.ts
import { dev } from '$app/env';
import type { CFENV } from '../../app';
let context:ExecutionContext
const exposeCFGlobals = (globalObjects:object,ctx:ExecutionContext)=>{
Object.entries(globalObjects).forEach(([key,val])=>{
global[key]=val;
})
context = ctx;
}
const fn = (ctx:ExecutionContext) => {
exposeCFGlobals({crypto},ctx)
return;
}
export default async (_platform:App.Platform) => {
if(!dev){
return _platform;
}
if(_platform){
return _platform;
}
console.log("!!!!!INITIALIZED!!!!!")
const dotenv = await import("dotenv");
const esbuild = await import("esbuild")
const path = await import("path")
const toml = await import("toml")
const fs = await import("fs");
const sourcefile = path.join(process.cwd(),"/durable_objects/src/objects.ts")
const tsconfpath = path.join(process.cwd(),"/tsconfig.json")
const wranglerPath = path.join(process.cwd(),"/wrangler.toml")
const sourceCode = fs.readFileSync(sourcefile).toString('utf8')
const tsconfigRaw = fs.readFileSync(tsconfpath).toString('utf8')
const wranglerConfigRaw = fs.readFileSync(wranglerPath).toString('utf8')
const wranglerConfig = toml.parse(wranglerConfigRaw)
const bindings = wranglerConfig?.vars ??{}
const durableObjects = (wranglerConfig?.durable_objects?.bindings ?? []).reduce((p,{name,class_name})=>{
p[name] = class_name;
return p;
},{})
const {code } = esbuild.transformSync(
`
const fn = ${fn.toString()};
export default {
fetch: async (request, env2, ctx2) => {
fn(ctx2);
return new Response("Hello Miniflare!");
}
};
${sourceCode}
`
,{
loader: 'ts',
sourcefile,
tsconfigRaw,
sourcemap:"inline"
});
const {parsed} = dotenv.config()
const miniflare = await (await import("miniflare")).Miniflare;
const mf = new miniflare({
modules:true,
durableObjectsPersist:true,
wranglerConfigPath:false,
envPath:path.join(process.cwd(),"/.env"),
script: code,
durableObjects,
bindings,
globals:{exposeCFGlobals}
})
await mf.dispatchFetch("https://host.tld")
const env = {...parsed, ...bindings} as unknown as CFENV;
for await (const [k,_] of Object.entries(durableObjects)){
env[k] = await mf.getDurableObjectNamespace(k) as DurableObjectNamespace;
}
const platform:App.Platform = {env,context}
return platform;
}In brief
This logic should be part of adapter and every adapter should fulfil its platform specific requirements. I understand that there is a desire for keeping codebase free from adapter specific logic. However, I don't see this happening when endpoint events expose adapter specific platform. |
|
Hi @denizkenan, your looks great! Would you by any chance have an public repo with this implementation available? |
It's not perfect but here's what I've been using for my base repo: |
Ah nice, I see that you have wrapped it as boilerplate(also added cache). |
|
👍🏼 |
|
Hi there, I understand the frustration expressed in the original post regarding the inability to use platform specific context when in development mode. While there are some workarounds available, such as running the building process in watch mode and using the development software provided by the platform on the build output, they fall short of providing a seamless developer experience, especially with regards to features like HMR. As someone who has developed applications that rely on the platform attribute, I can attest to the pain of not being able to use it in development mode. It's disappointing that Svelte, which is known for its developer-friendliness, has not yet found a solution to this issue, especially when other frameworks have already done so. I would like to voice my support for finding a solution to this issue and urge the Svelte team to address it as soon as possible. Please let me know when there are any kind of solutions to this yet. Thank you. |
|
For Cloudflare Pages / Workers it would be possible to use Miniflare to archive this. I build a basic sample showcasing how fallbacks can already be implemented using this, it would still be awesome to have this already povided by // /lib/server/miniflare.ts
type StorageOptionsMemory = {
type: 'memory';
};
type StorageOptionsFile = {
type: 'file';
path: string;
};
export type StorageOptions = StorageOptionsMemory | StorageOptionsFile;
export const createCache = async (storageOptions: StorageOptions) => {
const { Cache } = await import('@miniflare/cache');
if (storageOptions.type === 'memory') {
const { MemoryStorage } = await import('@miniflare/storage-memory');
return new Cache(new MemoryStorage());
} else if (storageOptions.type === 'file') {
const { FileStorage } = await import('@miniflare/storage-file');
return new Cache(new FileStorage(storageOptions.path));
}
throw new Error('StorageType not found');
};
export const createD1 = async (storageOptions: StorageOptions) => {
const { createSQLiteDB } = await import('@miniflare/shared');
const { D1Database, D1DatabaseAPI } = await import('@miniflare/d1');
if (storageOptions.type === 'memory') {
const sqliteDb = await createSQLiteDB(':memory:');
return new D1Database(new D1DatabaseAPI(sqliteDb));
} else if (storageOptions.type === 'file') {
const sqliteDb = await createSQLiteDB(storageOptions.path);
return new D1Database(new D1DatabaseAPI(sqliteDb));
}
throw new Error('StorageType not found');
};
export const createR2 = async (storageOptions: StorageOptions) => {
const { R2Bucket } = await import('@miniflare/r2');
if (storageOptions.type === 'memory') {
const { MemoryStorage } = await import('@miniflare/storage-memory');
return new R2Bucket(new MemoryStorage());
} else if (storageOptions.type === 'file') {
const { FileStorage } = await import('@miniflare/storage-file');
return new R2Bucket(new FileStorage(storageOptions.path));
}
throw new Error('StorageType not found');
};
export const createKV = async (storageOptions: StorageOptions) => {
const { KVNamespace } = await import('@miniflare/kv');
if (storageOptions.type === 'memory') {
const { MemoryStorage } = await import('@miniflare/storage-memory');
return new KVNamespace(new MemoryStorage());
} else if (storageOptions.type === 'file') {
const { FileStorage } = await import('@miniflare/storage-file');
return new KVNamespace(new FileStorage(storageOptions.path));
}
throw new Error('StorageType not found');
};
export const createDOStorage = async (storageOptions: StorageOptions) => {
const { DurableObjectStorage } = await import('@miniflare/durable-objects');
if (storageOptions.type === 'memory') {
const { MemoryStorage } = await import('@miniflare/storage-memory');
return new DurableObjectStorage(new MemoryStorage());
} else if (storageOptions.type === 'file') {
const { FileStorage } = await import('@miniflare/storage-file');
return new DurableObjectStorage(new FileStorage(storageOptions.path));
}
throw new Error('StorageType not found');
};// /src/hooks.server.ts
import { dev } from '$app/environment';
import { createKV, createD1 } from '$lib/server/miniflare';
export const handle = ({ event, resolve }) => {
if (dev) {
// We fake the platform for local development.
event.platform ??= {
env: {
COUNTER: createKV({ type: 'file', path: '.mf/kv-counter' }),
DATABASE: createD1({ type: 'file', path: '.mf/d1-database.sqlite3' }),
},
};
}
return resolve(event);
}; |
|
I used the same technique to fill out the platform in dev for a custom adapter I was working with: export async function handle({ event }) {
if (dev) {
const dev_platform = await import('./dev/platform.server.js');
event.platform = dev_platform.platform;
}
} |
|
What's the proper way to define classes for durable objects using SvelteKit and the modular @miniflare libraries? |
|
@UnlimitedBytes thanks for sharing. Would it also be possible to access remote D1 dbs from wrangler pages dev (assuming I just use another remote D1 as dev instance next to my production D1)? |
I wonder about this also. Following. |
Yes, it would be possible. Although D1 is intended to be only used from It is worth nothing though that Cloudflare currently (as D1 is only intended to be used from workers/pages and it's in alpha) does not document their D1 API Endpoints. So you will need to "reverse engineer" it from the Sadly I cannot provide an example for this specific use-case as it's not mine and I don't have enough time to spear to develop things I don't need myself. |
|
|
THIS has to be included in the docs. I can't explain how confused I was, just learning sveltekit but not being able to get it running locally. Somehow weird for being the "most-loved" framework. Simply add the above to the docs, it'll save sooo many people hours of research. |
|
Ideally, we'd just pass the namespaces to the adapter options such as: // svelte.config.js
import adapter from '@sveltejs/adapter-cloudflare';
/** @type {import('@sveltejs/kit').Config} */
const config = {
kit: {
adapter: adapter({
kvNamespaces: ['YOUR_KV_NAMESPACE'] // populates platform for us
})
}
};
export default config;Otherwise, we could export helper methods as demonstrated in #4292 (comment) As a third option (or something we can add now), we can document the pattern below on using // src/lib/dev/miniflare.js
import { Miniflare, Log, LogLevel } from 'miniflare';
// See https://latest.miniflare.dev/
// for further configuration
/** @type {import('miniflare').MiniflareOptions} */
const opts = {
log: new Log(LogLevel.WARN),
modules: true,
script: `
export default {
fetch () {
return new Response(null, { status: 404 });
}
}
`,
// These namespaces should also be added to
// `app.d.ts` `App.Platform.env`
// (and `wrangler.toml` if using wrangler)
kvNamespaces: ['YOUR_KV_NAMESPACE'],
kvPersist: './.mf/kv'
};
/**
* @returns {Promise<App.Platform>}
*/
export async function getPlatform() {
const mf = new Miniflare(opts);
/** @type {App.Platform} */
const platform = {
env: await mf.getBindings()
};
return platform;
}// src/hooks.server.js
import { dev } from '$app/environment';
/** @type {import('@sveltejs/kit').Handle} */
export async function handle({ event, resolve }) {
if (dev) {
const { getPlatform } = await import('$lib/dev/miniflare');
event.platform ??= await getPlatform();
}
const response = await resolve(event);
return response;
} |
|
I originally used a version based on: @UnlimitedBytes code #4292 (comment)
See https://github.com/sdarnell/cf-svelte/blob/main/src/lib/server/miniflare.ts But for the more general issue, it would be really good if adapters could hook into the server hooks processing, and obviously if the adapter-cloudflare could include the above it would relieve a lot of pain when people try to use CF and Sveltekit together. |
I've been trying to get this working all morning and finally found my way to this post. However, I get the following error when running I originally had a bunch of errors for missing basic modules, but i was able to |
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
This comment was marked as off-topic.
|
I found something that works for me, at least, it'll let me continue developing and testing locally with the same code that'll run on Cloudflare in production. I created two NPM commands, one that just does
Not ideal since i'll need to keep my package.json and wrangler.toml in sync if i add/change any bindings, but it'd be a much better developer experience if the adapter could take care of these details. |
|
@ssbm-oro So I actually got all this working yesterday with a local db through mini flare and wrangler using dev build. Be minded I don't have KV setup, so removed this parts of the code. Here is my start script to get it all running
I also needed to populate my local DB, I use this script: |
|
@zwergius Thanks for the suggestion, this is similar to how I was trying to run it before. Maybe it's because I'm using npm instead of pnpm, but when I run the command like this instead of looking at the build output directory, the DB and KV objects are undefined. |
|
For the many others that will end up with this issue: There is pull request on svelte-demo-d1 which let you test this quickly: |
|
I was able to get this working today locally see: cloudflare/workers-sdk#3622 (comment) |
|
@longrunningprocess , yes, that's possible, but it's not "dev" space.. that is testing of a build.. |
|
#11730 has been merged which opens the door for adapters to populate |
and others
Describe the problem
svelte-kit dev, theevent.platformobject is always empty, with no great way to mock it.event.platformis empty for prerendering.Describe the proposed solution
Ability to provide a
platformobject insvelte.config.jsthat isevent.platformwhenevent.platformis undefined.event.platform, useful for specifying optional/default platform values.Alternatives considered
Perhaps allowing a 'transform' function that accepts an event may be better, in case platform needs to change based on inputs?
Importance
would make my life easier
Additional Information
No response
The text was updated successfully, but these errors were encountered: