Function file changes ignored by `netlify dev --framework "#static"` while using `vite build --watch`

[mi-na-bot]

Describe the bug

Problem

I am attempting to use Vite to bundle function files and preview them using netlify dev as the node server. When using the emptyOutDir option I find that the functions are not correctly updated/refreshed as they are changed and rebuilt.

Discussion

The error is caused because vite removes the directory and the almost instantly recreates a new one with fresh versions of the files. Since the event handlers are debounced the order of the adds and removes becomes jumbled causing the function registry to enter a corrupted state inconsistent with the filesystem. In this code block, simply removing the debounce also solves the issue.

cli/src/utils/command-helpers.ts

Lines 253 to 289 in bc18db2

	let onChangeQueue: string[] = []
	let onAddQueue: string[] = []
	let onUnlinkQueue: string[] = []
	const debouncedOnChange = debounce(() => {
	onChange(onChangeQueue)
	onChangeQueue = []
	}, DEBOUNCE_WAIT)
	const debouncedOnAdd = debounce(() => {
	onAdd(onAddQueue)
	onAddQueue = []
	}, DEBOUNCE_WAIT)
	const debouncedOnUnlink = debounce(() => {
	onUnlink(onUnlinkQueue)
	onUnlinkQueue = []
	}, DEBOUNCE_WAIT)
	watcher
	.on('change', (path) => {
	// @ts-expect-error
	decache(path)
	onChangeQueue.push(path)
	debouncedOnChange()
	})
	.on('unlink', (path) => {
	// @ts-expect-error
	decache(path)
	onUnlinkQueue.push(path)
	debouncedOnUnlink()
	})
	.on('add', (path) => {
	// @ts-expect-error
	decache(path)
	onAddQueue.push(path)
	debouncedOnAdd()
	})

This code resolves the issue

export const watchDebounced = async (target, { depth, ignored = [], onAdd = noOp, onChange = noOp, onUnlink = noOp }) => {
    const baseIgnores = [/\/(node_modules|.git)\//];
    const watcher = chokidar.watch(target, { depth, ignored: [...baseIgnores, ...ignored], ignoreInitial: true, atomic:false });
    await once(watcher, 'ready');
    watcher
    .on('change', (path) => {
      // @ts-expect-error
      decache(path)
      onChange([path])
    })
    .on('unlink', (path) => {
      // @ts-expect-error
      decache(path)
      onUnlink([path])
    })
    .on('add', (path) => {
      // @ts-expect-error
      decache(path)
      onAdd([path])
    })
    return watcher;
};

Solutions with debounce?

Is debounce actually helpful here?

Steps to reproduce

1. Clone this minimal error reproduction. https://github.com/bgw-io/netlify-debounce-error-example.git
2. Install dependencies. pnpm install
3. Start the dev server pnpm dev
4. Change the console.log message in src/functions/api-background.ts
5. Observe that vite rebuilds it but netlify dev does not mention reloading it.
6. Navigate to localhost:8888/.netlify/functions/api-background and observe that the old console.log message is shown, not the new one.
7. Close the dev server with ctrl+c

Configuration

No netlify.toml is required. Other configurations are all in minimal reproduction repo.

Environment

@minervabot ➜ /workspaces/netlify-debounce-error-example (main) $ pnpm dlx envinfo --system --binaries --npmPackages netlify-cli --npmGlobalPackages netlify-cli
Packages: +1
+
Progress: resolved 1, reused 0, downloaded 1, added 1, done

  System:
    OS: Linux 6.5 Ubuntu 20.04.6 LTS (Focal Fossa)
    CPU: (2) x64 AMD EPYC 7763 64-Core Processor
    Memory: 5.52 GB / 7.74 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 20.14.0 - ~/nvm/current/bin/node
    Yarn: 1.22.22 - /usr/bin/yarn
    npm: 10.7.0 - ~/nvm/current/bin/npm
    pnpm: 9.1.4 - ~/nvm/current/bin/pnpm
  npmPackages:
    netlify-cli: ^17.26.0 => 17.26.0 

@minervabot ➜ /workspaces/netlify-debounce-error-example (main) $ 

[mi-na-bot]

I wrote an alternative version of the debounce that will not let the add/unlink/change get jumbled and even it fails when setTimeout is used (works if flushLater runs flushNow immediately), I believe that even yielding causes some kind of timing issue.

export const watchDebounced = async (target, { depth, ignored = [], onAdd = noOp, onChange = noOp, onUnlink = noOp }) => {
    const baseIgnores = [/\/(node_modules|.git)\//];
    const watcher = chokidar.watch(target, { depth, ignored: [...baseIgnores, ...ignored], ignoreInitial: true, atomic:false });
    await once(watcher, 'ready');
    let flushTimeout;
    let onChangeQueue = [];
    let onAddQueue = [];
    let onUnlinkQueue = [];

    const flushNow = ()=>{
        if (flushTimeout) {
            clearTimeout(flushTimeout);
            flushTimeout=undefined;
        }
        if (onChangeQueue.length>0) {
            onChange(onChangeQueue);
            onChangeQueue=[]
        }
        if (onAddQueue.length>0) {
            onAdd(onAddQueue);
            onAddQueue = [];
        }
        if (onUnlinkQueue.length>0) {
            onUnlink(onUnlinkQueue);
            onUnlinkQueue = [];
        }
    }

    const flushLater = ()=>{
        if (flushTimeout) {
            clearTimeout(flushTimeout);
            flushTimeout=undefined;
        }
        setTimeout(flushNow,1500)
    }


    const debouncedOnChange = (path) => {
        if (onAddQueue.length>0 || onUnlinkQueue.length>0) flushNow();
        onChangeQueue.push(path);
        flushLater();
    }
    const debouncedOnAdd = (path) => {
        if (onChangeQueue.length>0 || onUnlinkQueue.length>0) flushNow();
        onAddQueue.push(path);
        flushLater();
    }
    const debouncedOnUnlink = (path) => {
        if (onAddQueue.length>0 || onChangeQueue.length>0) flushNow();
        onUnlinkQueue.push(path);
        flushLater();
    }

    watcher
        .on('change', (path) => {
        // @ts-expect-error
        decache(path);
        
        debouncedOnChange(path);
    })
        .on('unlink', (path) => {
        // @ts-expect-error
        decache(path);
        
        debouncedOnUnlink(path);
    })
        .on('add', (path) => {
        // @ts-expect-error
        decache(path);
        
        debouncedOnAdd(path);
    });
    return watcher;
};

[mi-na-bot]

Honestly I am just going to use another solution here. It is unfortunate the file watching is so buggy as emulating the node environment using the CLI would be sooooo convenient.