Node.js
Use the OpenAI Node SDK or the AI SDK to talk to OwnLLM from JavaScript.
OwnLLM works with two popular JavaScript clients. Pick whichever your codebase already uses.
Option 1 — OpenAI SDK
npm install openaiimport OpenAI from "openai";
const client = new OpenAI({
baseURL: process.env.OPENAI_BASE_URL, // https://acme-prod.ownllm.app/v1
apiKey: process.env.OPENAI_API_KEY,
});
const stream = await client.chat.completions.create({
model: "llama-3.3-70b",
messages: [{ role: "user", content: "Hello" }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}Tool calling and the rest of the OpenAI surface work unchanged.
Option 2 — Vercel AI SDK
npm install ai @ai-sdk/openai-compatibleimport { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";
const ownllm = createOpenAICompatible({
name: "ownllm",
baseURL: process.env.OPENAI_BASE_URL!,
apiKey: process.env.OPENAI_API_KEY!,
});
const result = streamText({
model: ownllm("llama-3.3-70b"),
messages: [{ role: "user", content: "Why is the sky blue?" }],
});
for await (const chunk of result.textStream) {
process.stdout.write(chunk);
}The AI SDK provides a uniform shape across providers — useful if
your app talks to several. createOpenAICompatible handles tool
calling, structured output, and streaming.
Audit attribution
Both SDKs let you pass user:
// OpenAI SDK
await client.chat.completions.create({
model: "llama-3.3-70b",
messages: [...],
user: "alice@acme.com",
});// AI SDK — `experimental_telemetry` records the user id
streamText({
model: ownllm("llama-3.3-70b"),
messages: [...],
experimental_telemetry: {
isEnabled: true,
metadata: { user: "alice@acme.com" },
},
});Edge runtimes
The OwnLLM API is plain HTTPS, so it works from Cloudflare Workers,
Vercel Edge Functions, Deno Deploy, and so on. The OpenAI SDK
requires globalThis.fetch, which all those runtimes ship.
React hooks (assistant-ui / useChat)
For a chat UI in a Next.js app, drive the <Thread /> component from
assistant-ui (or the AI SDK's useChat) with a server route that
forwards to OwnLLM:
// app/api/chat/route.ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { convertToModelMessages, streamText } from "ai";
const ownllm = createOpenAICompatible({
name: "ownllm",
baseURL: process.env.OPENAI_BASE_URL!,
apiKey: process.env.OPENAI_API_KEY!,
});
export async function POST(req: Request) {
const { messages, model } = await req.json();
const result = streamText({
model: ownllm(model ?? "llama-3.3-70b"),
messages: convertToModelMessages(messages),
});
return result.toUIMessageStreamResponse();
}