nlkitai/nlux

Make defaultDelayInMsBeforeComplete configurable

Closed this issue · 3 comments

aspirantzhang commented

Description

Currently, the stream is automatically marked as complete if no data is received for more than 2 seconds (defaultDelayInMsBeforeComplete). In some cases, the interval between data chunks can exceed this limit, causing the stream to be prematurely marked as complete.

https://github.com/nlkitai/nlux/blob/c564e4105bd99a399bf51b6fb9b8e1e3a704b9ac/packages/shared/src/markdown/stream/streamParser.ts#L85C1-L91C10

const defaultDelayInMsBeforeComplete = 2000;
...

        if (buffer.length === 0) {
            if (streamIsComplete || nowTime - parsingContext.timeSinceLastProcessing > defaultDelayInMsBeforeComplete) {
                completeParsing();
            }

            return;
        }

I believe this should be configurable to accommodate streams with longer gaps between data chunks.

I am working with Alibaba's Tongyi Qianwen model, and I have observed that the interval between data chunks can be 10 seconds or longer (possibly due to network issues). This causes the stream to be marked complete too early.

Suggestion

Can we introduce an option to configure this timeout? This flexibility would help prevent premature completion of streams in various use cases.

As I am not a professional frontend developer, I am unsure how to implement this change myself and would greatly appreciate assistance from others.

Thank you!

salmenus commented

Nice suggestion. Quick one to implement. Working on that.

  • 👍1
  • ❤️1
salmenus commented

Alright new option added in v2.5.6, just published to NPM ✔️

Details: 👇

waitTimeBeforeStreamCompletion?: number | 'never';

Usage example:

<AiChat
    adapter={customSlowAdapter}
    messageOptions={{
        waitTimeBeforeStreamCompletion: 5000,
    }}
/>

Code Example:
https://github.com/nlkitai/nlux/blob/latest/samples/aiChat/react/src/App.tsx#L305

Documentation:
https://docs.nlkit.com/nlux/reference/ui/ai-chat#message-options

Give it a try @aspirantzhang and let us know what do you think.
And please support NLUX by sharing/tweeting — You can mention @nluxai if you do so 🙌

  • 👍1
salmenus commented

I'll be closing this @aspirantzhang if no further feedback.