Building a GPT-3 app with Next.js and streaming

This post is historical and describes Edge Functions as they existed at the time. For new projects, use with the Node.js runtime and .Note:Vercel FunctionsFluid compute

The field of artificial intelligence continues to take the world by storm. Huge strides have been made in text and image generation through tools like ChatGPT, GPT-3, DALL-E, and Stable Diffusion. It’s spawned a wave of exciting AI startups, many of which we’re seeing .built with Vercel and Next.js

One of the most exciting developments in the AI space is GPT-3, a cutting-edge natural language processing model developed by . With its ability to understand and generate human-like text, GPT-3 has the potential to disrupt how we perform many of our tasks.OpenAI

This historical walkthrough uses to show how streaming changed the user experience for GPT-3 apps. For new projects, build this pattern with , , and streaming responses.twitterbio.comVercel FunctionsFluid compute

The Next.js frontend consists of a few elements:

Here’s what the code for our looks like. We have a few pieces of state that correspond to the elements mentioned above. We’re also defining a prompt—like ChatGPT, we need to send a prompt to GPT-3 to instruct it to generate the new bios. Finally, we ask GPT-3 to generate two bios clearly labeled (so we can parse them correctly) using the user-provided bio and vibe as context.index page

The rest of our index page is comprised of the UI elements themselves: our text box, dropdown, submit button, and two containers on the bottom that we display when we get the generated bios. There's also some loading logic for the button to show a loading indicator when clicked.

In addition to the UI elements and the loading logic, we have a function that’s called when the user clicks the submit button. This sends a request to our API route with the prompt in the body.generateBioPOST/api/generate

We get the generated bios back from the API route, save it to the state, then display it to the user. Because we asked GPT-3 to return the text in a specific numbered format, we can split it based on the “2.” to show the user the two bios separated nicely into containers as seen below.generatedBios

A great advantage of using Next.js is being able to handle both our frontend and backend in a single application. We can spin up an API route just by creating a file called in our folder. Let’s take a look at our API Route.generate.tsapi/api/generate

We get the prompt from the request body that’s passed in on the frontend, then construct a payload to send to OpenAI. In this payload, we specify some important information like the exact model (GPT-3) and how many tokens we want OpenAI to respond with (a token is approximately 4 characters). In this case, we’re limiting the max tokens because Twitter bios have a character constraint.

After the payload is constructed, we send it in a request to OpenAI, await the result to get back the generated bios, then send them back to the client as JSON.POST

There we have it! We built the first version of our application. Feel free to check out the and for this approach.codedemo

The original version of this tutorial migrated the example to Edge Functions to get streaming and avoid timeouts. For new AI applications on Vercel, use with the default Node.js runtime and . This gives you streaming support, longer durations, and full Node.js API compatibility.Vercel FunctionsFluid compute

If you are rebuilding this example today:

See and for current implementation guidance.Vercel FunctionsStreaming Functions

We hope this walkthrough helps you build incredible GPT-3 powered applications. We’ve already seen several sites built with this template such as , , and —some of which have thousands of users. Visit the to see everything we talked about in action, check out our , or start optimizing prompts across various models with Vercel's .RephraserGenzTranslatorChefGPTTwitter Bio siteother AI templatesAI Playground

Read more

The frontend

The backend

Current guidance: use Vercel Functions with streaming

Resources

• A text box for users to copy their current bio or write a few sentences about themselves

• A dropdown where they can select the tone of the bio they want to be generated

• A submit button for generating their bio, which when clicked calls an API route that uses OpenAI’s GPT-3 model and returns two generated bios

• Two containers to display the generated bios after we get them back from the API route

• Keep the API route on the Node.js runtime.

• Stream model output from the route to the client.

• Use Fluid compute for long-running model calls and efficient concurrency.

• Choose a function region close to the upstream AI provider when latency matters.