Skip to main content
The Realtime API enables you to transform live video streams with minimal latency using WebRTC. Perfect for building iOS camera effects, video conferencing filters, AR applications, and interactive live streaming.

Quick Start

Client-Side Authentication

For iOS and macOS applications, use ephemeral keys instead of embedding your permanent API key in the app bundle. Ephemeral keys are short-lived tokens safe to include in client applications.
Learn more about client tokens and why they’re important for security.

Fetching an Ephemeral Key

Your app should fetch an ephemeral key from your backend server before connecting:

Connecting with an Ephemeral Key

Never hardcode your permanent API key in iOS apps. App bundles can be decompiled, exposing embedded secrets.

Camera Capture

Realtime media uses LiveKit tracks. Build a LocalVideoTrack from LiveKit’s camera-track factory, size it from the model registry, and attach a MirroringVideoProcessor to pre-flip the front camera.

Setting Up Capture

CameraCaptureOptions parameters:
  • position.front or .back (LiveKit’s AVCaptureDevice.Position)
  • dimensionsDimensions(width:height:). For portrait output, pass model.height as width and model.width as height (swap for landscape).
  • fps — target framerate; use model.fps
LocalVideoTrack.createCameraTrack(name:options:processor:) starts capture immediately. There’s no separate startCapture() call.

Switching Cameras

LiveKit’s CameraCapturer toggles between front and back. Keep the MirroringVideoProcessor in sync so .auto mirroring follows the active camera:

Stopping Capture

Use model.fps, model.width, and model.height to size the capture so the encoder doesn’t have to rescale.
Camera capture requires a real iOS device. The simulator does not support camera access.

Front-camera mirroring

Pre-flipping the selfie input keeps server-baked pixels (watermarks, overlays) correctly oriented when you render the remote stream as-is. MirrorMode values:
  • .off (default) — never mirror.
  • .auto — mirror only when the active camera is .front. Update mirror.cameraPosition = cameraCapturer.position on camera switch.
  • .on — always mirror.
With mirroring enabled, render both the local preview and the remote stream with RTCMLVideoViewWrapper(track:) — no mirror: argument.

Connecting

Create a DecartRealtimeManager and connect with your local media stream:
RealtimeConfiguration parameters:
  • model (required) - Realtime model from Models.realtime()
  • resolution (optional) - .p720 or .p1080. Omit for the server’s 720p default; pass .p1080 to request a 1080p remote stream from supported models.
  • initialPrompt (optional) - Initial transformation prompt
    • text - Prompt text
    • enrich - Whether to auto-enhance the prompt
    • referenceImageData - Optional reference image Data
  • connection (optional) - Connection configuration
    • iceServers - STUN/TURN server URLs (default: Google STUN)
    • connectionTimeout - Connection timeout in seconds (default: 15)
    • rtcConfiguration - Custom RTCConfiguration for advanced WebRTC tuning
  • media (optional) - Media configuration
    • video.maxBitrate - Max bitrate in bps (default: 2,500,000)
    • video.minBitrate - Min bitrate in bps (default: 300,000)
    • video.maxFramerate - Max framerate (default: 30)
    • video.preferredCodec - Preferred video codec (default: “VP8”)
Returns: RealtimeMediaStream — the transformed remote stream containing an optional videoTrack you can render For image-capable models, pass the reference image data on DecartPrompt:
Set initialPrompt (with referenceImageData for image-capable models) so the first frame is already transformed — otherwise viewers briefly see the raw camera feed.

Managing Prompts

Change the transformation style dynamically without reconnecting:
DecartPrompt parameters:
  • text (required) - Text description of desired style
  • referenceImageData (optional) - Reference image data (used with lucy-2.1)
  • enrich (optional) - Whether to enhance the prompt (default: false)
Prompt enhancement uses Decart’s AI to expand simple prompts for better results. Only the lucy-2.1 model supports reference images.

Connection State

Monitor connection state, service status, generation ticks, and session ID using the events AsyncStream:
DecartRealtimeState properties:
  • connectionState.idle, .connecting, .connected, .generating, .reconnecting, .disconnected, .error
  • serviceStatus.unknown, .enteringQueue, .ready
  • queuePosition — Current position in queue (nil if not queued)
  • queueSize — Total queue size (nil if not queued)
  • generationTick — Seconds elapsed during generation (nil when not generating)
  • sessionId — Current session identifier (nil before session established)
DecartRealtimeConnectionState helpers:
  • .isConnectedtrue when connected or generating
  • .isInSessiontrue when connected, connecting, generating, or reconnecting

Auto-Reconnect

The SDK automatically reconnects when an unexpected disconnection occurs (e.g., network interruption). During auto-reconnect, the connection state transitions to .reconnecting while the SDK retries with exponential backoff (up to 5 attempts, max 10s delay). When auto-reconnect succeeds, a new RealtimeMediaStream is emitted via remoteStreamUpdates. You must rebind your UI to the new stream’s video track:
Auto-reconnect is not triggered on user-initiated disconnect(), permanent errors (401/403, invalid key, expired session), or after all retries are exhausted. If all retries fail, the state moves to .error.

Connection Quality

Two layers report network health on a shared .good | .fair | .poor | .critical scale: a preflight check before connecting, and an in-session signal while connected.

Preflight

A fast, network-only reachability check. No session, no cost. Never throws — degrades to .critical / .failed on any error:

In-session quality

While connected, the SDK derives a smoothed verdict from live connection stats (latency, packet loss, upstream bandwidth, frame rate) and tells you the limiting factor. The stream yields on debounced level changes; getConnectionQuality() returns the live snapshot whose metrics refresh on every poll:
In-session quality is on by default. Opt out by passing observability: .init(connectionQualityEnabled: false) in RealtimeConfiguration.

Glass-to-glass latency (opt-in)

Network RTT alone doesn’t reflect the latency users actually feel — a session can read .good while still feeling laggy. Set debugQuality: true to measure the real camera→display latency: the SDK stamps a pixel marker into each outgoing frame and reads it back off the rendered output, surfacing startup (ttffMs), steady-state (g2gMs), and end-to-end frame drops (g2gDropRatio). When present, glass-to-glass drives the latency verdict instead of RTT.
Diagnostic only. The marker is visible (bottom-left of the published and rendered video) and adds per-frame pixel work — don’t enable it for production / end-user sessions. The debugQuality flag must match on both createLocalCameraStream and RealtimeConfiguration.

Deep preflight

For a measured verdict before connecting, use the deep probe — it briefly opens a real session with a synthetic source, measures glass-to-glass, then tears it down. Requires a model and costs a short session:

Error Handling

Errors are thrown from async methods and can also arrive through the events stream when the connection state becomes .error:
Error Cases:
  • .invalidAPIKey - API key is invalid or missing
  • .invalidBaseURL(String?) - Base URL is malformed
  • .webRTCError(String) - WebRTC connection failed
  • .websocketError(String) - WebSocket connection error
  • .connectionTimeout - Connection timed out
  • .serverError(String) - Server returned an error
  • .processingError(String) - Processing failed
  • .invalidInput(String) - Invalid input parameters
  • .modelNotFound(String) - Specified model doesn’t exist
  • .networkError(Error) - Network request failed
  • .queueError(String) - Queue operation failed

Cleanup

Always stop the camera track and disconnect the manager when done:
Failing to disconnect can leave the LiveKit room open and waste resources.

Complete SwiftUI Example

Here’s a full SwiftUI application using the SDK’s built-in RTCMLVideoViewWrapper:

Best Practices

Size CameraCaptureOptions from the model registry so the encoder doesn’t have to rescale.
For best results, set enrich: true to let Decart’s AI enhance your prompts. Only disable it if you need exact prompt control.
Always observe remoteStreamUpdates to rebind your video track when the SDK auto-reconnects after a network interruption.
Use for await state in manager.events to track connection state, generation ticks, session ID, and queue position in a structured concurrency context.
Always call manager.disconnect() and capture.stopCapture() when done to avoid memory leaks and unnecessary resource usage.
Always test camera features on real iOS devices, as the simulator does not support WebRTC camera access.
Add camera and microphone usage descriptions to your Info.plist and handle permission denials gracefully in your UI.

API Reference

DecartClient.createRealtimeManager(options:)

Creates a realtime manager for a WebRTC session. Parameters:
  • options: RealtimeConfiguration - Configuration for the realtime session
    • model: ModelDefinition - Realtime model from Models.realtime()
    • initialPrompt: DecartPrompt - Initial transformation prompt (default: empty)
      • text: String - Prompt text
      • enrich: Bool - Whether to auto-enhance the prompt
      • referenceImageData: Data? - Optional reference image data
    • connection: ConnectionConfig - Connection settings (default: standard)
    • media: MediaConfig - Media settings (default: standard)
Returns: DecartRealtimeManager Throws: DecartError if the signaling URL cannot be constructed

DecartRealtimeManager.connect(localStream:)

Connects to the realtime transformation service. Parameters:
  • localStream: RealtimeMediaStream - Local media stream with camera video track
Returns: RealtimeMediaStream — the transformed remote stream Throws: DecartError if connection fails or times out

DecartRealtimeManager.setPrompt(_:)

Changes the transformation style. Parameters:
  • prompt: DecartPrompt - Prompt with text, optional reference image, and enrich flag

DecartRealtimeManager.disconnect()

Closes the connection and cleans up WebRTC resources.

DecartRealtimeManager.events

An AsyncStream<DecartRealtimeState> that emits state changes.

DecartRealtimeManager.remoteStreamUpdates

An AsyncStream<RealtimeMediaStream> that emits new remote streams after auto-reconnect.

LocalVideoTrack.createCameraTrack(name:options:processor:)

LiveKit factory that opens the camera and produces a LocalVideoTrack. Capture starts immediately — there is no separate startCapture() call. Parameters:
  • name: String - Track identifier (e.g. "video0")
  • options: CameraCaptureOptions - Position, dimensions, fps
  • processor: VideoProcessor? - Optional MirroringVideoProcessor (or any LiveKit VideoProcessor)

CameraCapturer.switchCameraPosition()

Toggles between front and back cameras. Access via videoTrack.capturer as? CameraCapturer. After switching, update mirror.cameraPosition = cameraCapturer.position to keep MirrorMode.auto in sync.

LocalVideoTrack.stop()

Async. Stops capture and releases the track. Call before manager.disconnect() during cleanup.

Next Steps

SDK Overview

Learn about installation, setup, and Swift SDK fundamentals

GitHub

Browse the SDK source code and contribute