Performance Issue with Next.js 13.4.7 Edge Functions in AI Chatbot Application

[abhayc-main]

Title: Performance Issue with Next.js 13 Edge Functions in AI Chatbot Application

Description

I am encountering significant performance issues with my Next.js 13 application that uses edge functions and edge runtime for integrating an AI chatbot using OpenAI's API. Despite using the same code as in the vercel-labs/ai-chatbot example repository, my application's response times are considerably slower, both when running locally and when deployed on Vercel Pro.

Expected Behavior

Given that the codebase is identical to the vercel-labs/ai-chatbot repository, I expect similar performance in terms of response time and streaming efficiency.

Actual Behavior

The chatbot responses in my application are substantially slower compared to the vercel-labs/ai-chatbot example, despite being run under similar conditions. This slow performance persists both on a local setup and when the application is deployed on Vercel Pro.

Steps to Reproduce

1. Clone the vercel-labs/ai-chatbot repository from GitHub.
2. Set up the application to run with Next.js 13 using edge functions and edge runtime.
3. Implement the chatbot functionality using the OpenAI API as per the example in the repository.
4. Compare the response time of the chatbot in this setup with the original repository's deployment on chat.vercel.ai.

Additional Context

• The primary function in question is the edge runtime route responsible for handling chatbot interactions. The code is as follows:

  import { kv } from '@vercel/kv'
  import { OpenAIStream, StreamingTextResponse } from 'ai'
  import { Configuration, OpenAIApi } from 'openai-edge'
  
  import { auth } from '@/auth'
  import { nanoid } from '@/lib/utils'
  
  export const runtime = 'edge'
  
  const configuration = new Configuration({
    apiKey: process.env.OPENAI_API_KEY
  })
  
  const openai = new OpenAIApi(configuration)
  
  export async function POST(req: Request) {
    const json = await req.json()
    const { messages, previewToken } = json
    const userId = (await auth())?.user.id
  
    if (!userId) {
      return new Response('Unauthorized', {
        status: 401
      })
    }
  
    if (previewToken) {
      configuration.apiKey = previewToken
    }
  
  
  
  
    const res = await openai.createChatCompletion({
      model: 'gpt-3.5-turbo-16k',
      messages,
      temperature: 0.7,
      stream: true
    })
  
    const stream = OpenAIStream(res, {
      async onCompletion(completion) {
        const title = json.messages[0].content.substring(0, 100)
        const id = json.id ?? nanoid()
        const createdAt = Date.now()
        const path = `/chat/${id}`
        const payload = {
          id,
          title,
          userId,
          createdAt,
          path,
          messages: [
            ...messages,
            {
              content: completion,
              role: 'assistant'
            }
          ]
        }
        await kv.hmset(`chat:${id}`, payload)
        await kv.zadd(`user:chat:${userId}`, {
          score: createdAt,
          member: `chat:${id}`
        })
      }
    })
  
    return new StreamingTextResponse(stream)
  }

• There are no apparent differences in the code or setup that could account for this performance discrepancy.
• The issue occurs regardless of whether the application is running locally or is deployed on Vercel Pro.

Environment

• Next.js version: 13
• Deployment platform: Vercel Pro
• Local environment OS and version: [Insert your local OS and version]
• Node.js version: [Insert your Node.js version]

Possible Causes

• Network latency or configuration issues specific to my environment.
• Potential differences in resource allocation between my setup and the original example.
• Unidentified bottlenecks in the edge runtime or edge functions implementation.

I am seeking guidance or suggestions on how to diagnose and resolve this performance issue. Any insights or recommendations would be greatly appreciated.