How to consume a stream from an external API in my Express.js server and send it to my client at the same time? Do I need something like Kafka?

by

I am sending OpenAI completions from my express.js server to my iOS swift client. I can send the streams as I receive them using the pipeline and iOS client can easily access the events.

import { pipeline } from 'node:stream/promises'

const completion = await fetch('https://api.openai.com/v1/chat/completions', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`
    },
    body: JSON.stringify({
      model: 'gpt-3.5-turbo',
      messages,
      stream: true
    })
  })

  await pipeline(completion.body as any, res)

If I try to read the stream in my endpoint in any way and write the values to the stream, the iOS client cannot access the events and the request resolves like a normal REST response.

What should I do? Do I need to something like Kafka do accomplish this?

  • “If I try to read the stream in my endpoint in any way and write the values to the stream,”- please clarify, what do you mean by ‘the values’ ? Also it will be good if you provide the code where you tried to do the processing and adding values to the stream.

    – 

Leave a Comment