Streaming

APis in toddle lets you stream the response data so your users spend less time staring at a loading spinner. This can, for example, be useful if you're integrating an AI chatbot onto your website and you want to stream the response to the user as the AI generates it.

Types of streaming

Two types of streaming can be done in toddle:

1. Server-sent events (SSE)
2. JSON streaming

Server-sent events (SSE aka Event Stream)

This type of stream is using the "text/event-stream" content type. The server sends a stream of events to the client. Each event consists of up to 4 fields:

• "id": The event ID
• "event": The event type
• "data": The event data
• "retry": The time in milliseconds before the client should try to reconnect

When toddle receives an entire event, it will emit an "On message"-event with the event data. This is a "CustomEvent" where the "detail" is the event data containing the abovementioned fields.

You can read more about the specs of SSE here

JSON streaming

This type of stream is using the "application/stream+json" or "application/x-ndjson" content type. The server sends a stream of JSON objects to the client.

When toddle receives a complete JSON object, it will emit an "On message"-event with the JSON object as the event data. This is a "CustomEvent" where the "detail" is the JSON object.

You can read more about the specs here

On Message Event

The "On message"-event is emitted when toddle receives a complete event or JSON object. You can act on this event by adding an "On message"-event in the Events tab in the API. To access the data of the event, refer to the streaming type described above.

Change the response behavior

Toddle uses the "Content-Type"-header of the response to determine how the response will be parsed.
If the header is set to "text/event-stream," toddle will parse it as a stream of server-side events.
Likewise, if the header is set to "application/stream+json" or "application/x-ndjson" toddle will parse it as a stream of JSON objects.
You might run into a case where the server is not setting the Content-Type header correctly. In these cases, you can override the default parsing behavior to match your expectations. This is done by setting the "Response Parsing" configuration in the advanced tab of the API to "Event Stream" or "JSON stream." This will cause toddle to ignore the Content-Type header and parse the response as a stream of server-side events or JSON objects, respectively.