Overview

Styrr supports streaming responses via server-sent events (SSE). The fallback logic still applies — if the first model rate-limits or fails during streaming, Styrr can restart with the next model.

Usage

import { StyrRouter } from '@carloscortezcloud/styrr-llm';

const router = new StyrRouter({
  apiKey: process.env.OPENROUTER_API_KEY,
  models: [
    { id: 'nvidia/nemotron-3-ultra-550b:free' },
    { id: 'google/gemma-4-31b-it:free' },
  ],
});

const stream = await router.stream([
  { role: 'user', content: 'Tell me a story' },
]);

for await (const chunk of stream) {
  // chunk: { type: 'token' | 'done' | 'fallback', data: string }
  if (chunk.type === 'token') {
    process.stdout.write(chunk.data);
  }
}

Stream Events

Event	Description
token	A text token from the LLM
fallback	Model failed, switching to next
done	Stream complete
error	Stream error

Retry Behavior

Streaming can fail midway if:

• The model rate-limits mid-response
• A network error occurs

Styrr handles these cases by falling back to the next model and restarting the stream. The fallback event notifies you when this happens.

Integration with Tinkuy

When using Tinkuy’s agent.stream(), streaming is handled automatically:

for await (const chunk of agent.stream('Tell me a story')) {
  // Tokens, tool calls, and fallback events
  // all unified into a single stream
}