Make dev environment match production environment more closely

[swyxio]

Describe the problem

I'm not sure if this is a bug so much as a discussion I'd like to have. One basic premise of sveltekit is that we have a local dev experience that matches closely to the production experience.

I find that one area where this promise is broken is when you try to render-on-demand rather than prerender, because it is very common for serverless environments (netlify, vercel et al) to separately bundle and run the serverless function. So, if I have a page that is only rendered on demand, its very easy for me to code a file read that works in dev, but then breaks when done in prod:

import { resolve } from 'path';
import { promises as fs } from 'fs';

const x = await fs.readFile(resolve('./podcasts.yml'), 'utf8')

this code works fine in dev, but then deploy it and...

{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: ENOENT: no such file or directory, open '/var/task/podcasts.yml'","trace":["Runtime.UnhandledPromiseRejection: Error: ENOENT: no such file or directory, open '/var/task/podcasts.yml'"," at process.<anonymous> (/var/runtime/index.js:35:15)"," at process.emit (events.js:400:28)"," at processPromiseRejections (internal/process/promises.js:245:33)"," at processTicksAndRejections (internal/process/task_queues.js:96:32)"]}

Describe the proposed solution

i think we should try to shift this problem left - if the page is serverless, actually compile the function and make it properly isolated as it would be in prod

Alternatives considered

do nothing, let people fumble around with this new serverless rendering paradigm :(

for what its worth i'm a veteran Netlify user, spent multiple hours trying to figure out how to get around this and still havent got anything i'm really happy with (apart from brute force copying the file postbuild into the built folder, which really feels super janky and not great)

Importance

would make my life easier

Additional Information

No response

[swyxio]

i just found netlify/zip-it-and-ship-it#392 which is a feature netlify shipped to get around that.. but still really much rather discover this stuff during dev than debug during production

[Rich-Harris]

I want this too. It's a huge can of worms though - where does it begin and end? Presumably we could hack vite to control access to node built-in modules (but allow it for prerenderable pages, or if using adapter-static) but there are doubtless other environmental differences beyond availability of node built-ins and filesystem access (eg all the web APIs Deno provides, presence/absence of process, etc), and it's unclear how faithfully we could emulate those environments. Lot of stuff we need to figure out

[swyxio]

inb4 the answer is some wasm microvm thingy…

[Rich-Harris]

Just leaving a note-to-self that we also need this before we can realistically create any sort of Electron/Tauri/whatever adapter

[antony]

This also depends on doing something platform specific for the proposed adapter, i.e. Vercel handles things a little differently to say, Netlify.

Even if we work around that difference by looking at the installed adapter, we don't have a clear picture of what to do when somebody has adapter-auto installed, for example.

[Rich-Harris]

If the dev server was started via vc dev or netlify dev etc then the problem becomes more solvable. I'm just not sure what that looks like

[Rich-Harris]

As part of #4296, we're thinking about how to prevent client-side code from accidentally importing secret environment variables. The current plan is to traverse the module graph from client-side entry points (i.e. pages, plus maybe hooks/client.js in future) in both dev and build and throwing an error if a server-only module is imported. (For now, that just means $app/env/private, but it could include endpoints, or modules with a certain naming convention like utils.server.js or whatever.)

I digress. The point is that if we have the ability to say that some files can't import certain modules, then it should be possible to say (for example) that if an endpoint imports node:fs or other Node built-ins and isn't marked as prerenderable (#4093) then it's an error.

This isn't a complete solution to the problem of making the dev server environment a high fidelity simulation of the production environment — it doesn't address platform-specific things like Durable Objects, or differences in the runtimes. But it would at least solve the specific problem that motivated this issue, namely that it's confusing to be able to use Node built-ins in dev and preview but not in production.

[yzn-h]

is this still in the roadmap ?

[NetOpWibby]

Popping in here to say I'd love if I could use package SvelteKit into a Deno app.