> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.decart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Mobile App

> Build a realtime AI video app with Expo and the Decart SDK

Transform live camera video with AI on mobile devices. This walkthrough covers the architecture and key integration points of an open-source Expo app that connects to Decart's realtime models — including Lucy 2.1 for character transformation.

<Info>
  This example uses Expo with LiveKit's React Native WebRTC stack. React Native realtime requires the LiveKit packages and an early `registerGlobals()` call — plain `react-native-webrtc` is not supported. See [LiveKit setup for React Native](#livekit-setup-for-react-native) below.
</Info>

**Source code:** [github.com/DecartAI/decart-example-expo-realtime](https://github.com/DecartAI/decart-example-expo-realtime)

## What you'll build

<CardGroup cols={3}>
  <Card title="Model switching" icon="toggle-on">
    Switch between Lucy Restyle Live, Lucy editing, and — after a quick upgrade — Lucy 2.1 for character transformation.
  </Card>

  <Card title="Style presets" icon="palette">
    Swipe through curated looks: anime characters, tuxedos, superhero costumes, and more. Each preset is a text prompt sent to the model.
  </Card>

  <Card title="Camera controls" icon="camera-rotate">
    Toggle front/back camera, switch view modes (fullscreen, picture-in-picture, split), and see your original and transformed feeds side by side.
  </Card>
</CardGroup>

## Prerequisites

* Node.js 18+
* Expo (`npx expo` — no global install needed)
* A Decart API key from [platform.decart.ai](https://platform.decart.ai)
* A physical iOS or Android device (WebRTC requires real camera hardware)

## Project setup

<Steps>
  <Step title="Clone the repository">
    ```bash theme={null}
    git clone https://github.com/DecartAI/decart-example-expo-realtime.git
    cd decart-example-expo-realtime
    ```
  </Step>

  <Step title="Install dependencies">
    ```bash theme={null}
    npm install
    ```
  </Step>

  <Step title="Add your API key">
    Create a `.env` file in the project root:

    ```bash theme={null}
    EXPO_PUBLIC_DECART_API_KEY=your-api-key-here
    ```
  </Step>

  <Step title="Run on a device">
    ```bash theme={null}
    npx expo run:ios
    # or
    npx expo run:android
    ```
  </Step>
</Steps>

<Warning>
  WebRTC requires a physical device. The iOS Simulator and Android Emulator do not support camera streaming.
</Warning>

## LiveKit setup for React Native

React Native realtime runs on [LiveKit's React Native packages](https://github.com/livekit/client-sdk-react-native). Plain `react-native-webrtc` is **not** supported. The cloned example already wires this up — here's what makes it work.

**1. Install the LiveKit packages** (already in the example's `package.json`):

```bash theme={null}
npm install @decartai/sdk livekit-client@2.20.1 \
  @livekit/react-native@2.11.1 \
  @livekit/react-native-webrtc@144.1.1
```

**2. Register LiveKit's native globals before the app starts.** The SDK reads the WebRTC globals that LiveKit installs, so `registerGlobals()` must run before your router or root component is imported:

```typescript theme={null}
// livekit-bootstrap.ts
import { registerGlobals } from "@livekit/react-native";

registerGlobals();
```

```typescript theme={null}
// index.ts — import the bootstrap before the app entrypoint
import "./livekit-bootstrap";
import "expo-router/entry"; // or your root App component
```

**3. Add the Expo config plugins.** Install them and register them in `app.json`:

```bash theme={null}
npm install @livekit/react-native-expo-plugin@1.0.2 \
  @config-plugins/react-native-webrtc
```

```json theme={null}
{
  "expo": {
    "plugins": [
      "@livekit/react-native-expo-plugin",
      "@config-plugins/react-native-webrtc"
    ]
  }
}
```

Then run `npx expo prebuild` and rebuild the native app. LiveKit does not run in Expo Go — you need a development build on a physical device.

<Note>
  Using **bare React Native (CLI)** instead of Expo? The realtime code is identical, but you need two Metro/Babel config changes (Expo applies them for you). See [React Native CLI (bare projects)](/sdks/javascript-realtime#react-native-cli-bare-projects).
</Note>

## Architecture overview

The app uses Expo Router with three screens:

| Screen      | File                        | Purpose                                                  |
| ----------- | --------------------------- | -------------------------------------------------------- |
| Entry       | `app/index.tsx`             | Checks camera/microphone permissions, routes accordingly |
| Permissions | `app/permission-screen.tsx` | Requests camera and microphone access                    |
| Camera      | `app/camera.tsx`            | Renders the main `<Camera />` component                  |

Key directories:

```
components/camera/
  hooks/useWebRTC.ts      — Core Decart SDK integration
  ui/ModelSelector.tsx     — Model toggle UI
  ui/VideoRenderer.tsx     — Displays local + remote streams
  ui/style-carousel/       — Swipeable style picker

livekit-bootstrap.ts       — registerGlobals() before the app entrypoint

lib/realtime/
  media-streams.ts         — Camera capture via LiveKit's mediaDevices

lib/skins/
  lucy-skin-list.ts        — 21 text prompts for Lucy editing presets
  lucy-restyle-live-skin-list.ts      — Style presets with optional audio
```

## Core integration: the WebRTC hook

The `useWebRTC` hook handles the entire Decart SDK lifecycle — creating a client, capturing the camera, connecting to a realtime model, and managing cleanup.

```typescript theme={null}
import { useCallback, useRef, useState } from "react";
import { createDecartClient, type ModelDefinition, type RealTimeClient, resolveFpsNumber } from "@decartai/sdk";
import type { MediaStream } from "@livekit/react-native-webrtc";
import { getMediaStream } from "@/lib/realtime/media-streams";

export function useWebRTC({ model, facingMode }: { model: ModelDefinition; facingMode: string }) {
  const [localMediaStream, setLocalMediaStream] = useState<MediaStream | null>(null);
  const [remoteMediaStream, setRemoteMediaStream] = useState<MediaStream | null>(null);
  const [connectionState, setConnectionState] = useState<string>("disconnected");
  const realtimeClientRef = useRef<RealTimeClient | null>(null);

  const initializeWebRTC = useCallback(
    async ({ model, facingMode }: { model: ModelDefinition; facingMode: string }) => {
      try {
        // 1. Create the Decart client
        const decartClient = createDecartClient({
          apiKey: process.env.EXPO_PUBLIC_DECART_API_KEY as string,
        });

        // 2. Capture camera with model-specific constraints.
        //    resolveFpsNumber() turns the model's FPS descriptor into a number.
        const stream = await getMediaStream(facingMode, {
          fps: resolveFpsNumber(model.fps),
          width: model.width,
          height: model.height,
        });
        setLocalMediaStream(stream);
        setConnectionState("connecting");

        // 3. Connect to the realtime model. Pass LiveKit's MediaStream type as the
        //    generic — no casts — and select VP8 with preferredVideoCodec.
        realtimeClientRef.current = await decartClient.realtime.connect<MediaStream>(stream, {
          model,
          preferredVideoCodec: "vp8",
          onRemoteStream: (transformedStream) => {
            setRemoteMediaStream(transformedStream);
            setConnectionState("connected");
          },
        });
      } catch (error) {
        console.error("Connection failed:", error);
        setConnectionState("failed");
      }
    },
    [],
  );

  const cleanupWebRTC = useCallback(() => {
    realtimeClientRef.current?.disconnect();
    setRemoteMediaStream(null);
    setConnectionState("disconnected");
  }, []);

  return {
    localMediaStream,
    remoteMediaStream,
    connectionState,
    realtimeClientRef,
    initializeWebRTC,
    cleanupWebRTC,
  };
}
```

Three things to note:

1. **Model constraints drive camera setup.** Each model defines its own `fps`, `width`, and `height`. Resolve the FPS with `resolveFpsNumber(model.fps)` (it can be a descriptor object rather than a plain number), and `getMediaStream` requests exactly those dimensions from LiveKit's `mediaDevices`.

2. **`preferredVideoCodec: "vp8"` selects the codec.** React Native works best with VP8. The SDK negotiates it for you — there's no manual SDP rewriting (the old `customizeOffer` / `forceCodecInSDP` approach is gone).

3. **No `MediaStream` casts.** `connect<MediaStream>(...)` takes LiveKit's React Native `MediaStream` type as a generic, so you pass the captured stream straight through — no `as unknown as` bridging.

## Switching models

Each model offers different creative capabilities:

| Model                 | ID               | What it does                                                         |
| --------------------- | ---------------- | -------------------------------------------------------------------- |
| **Lucy 2.1**          | `lucy-2.1`       | Character transformation and text-guided editing in one model        |
| **Lucy Restyle Live** | `lucy-restyle-2` | Full video restyling — transform scenes into different visual styles |

Switch models by creating a new `ModelDefinition` and reconnecting:

```typescript theme={null}
import { models, type ModelDefinition } from "@decartai/sdk";

// Select a model
const restyleLive = models.realtime("lucy-restyle-2");
const lucy2 = models.realtime("lucy-2.1");

// To switch: disconnect, update state, reconnect
async function switchModel(newModel: ModelDefinition) {
  cleanupWebRTC();
  await new Promise((resolve) => setTimeout(resolve, 100)); // allow cleanup to complete
  await initializeWebRTC({ model: newModel, facingMode });
}
```

<Tip>
  Use Lucy 2.1 as the default for new mobile integrations. It supports both character reference and text-only editing.
</Tip>

## Applying style presets

The app includes curated prompt presets for each model. When a user selects a preset, the prompt is sent to the active model:

```typescript theme={null}
// Apply a style preset to the current model
if (realtimeClientRef.current) {
  realtimeClientRef.current.setPrompt(skin.prompt, {
    enhance: skin.enrich ?? false,
  });
}
```

Example prompts from the preset list:

```typescript theme={null}
const presets = [
  {
    title: "Anime Character",
    prompt:
      "Transform the person into a 2D anime character with smooth cel-shaded lines, " +
      "soft pastel highlights, large expressive eyes, clean contours, even lighting, " +
      "simplified textures, and a bright studio-style background for a polished anime look.",
  },
  {
    title: "Black Tuxedo",
    prompt:
      "Change the outfit to a sharp black tuxedo with satin lapels, crisp white shirt, " +
      "black bow tie, tailored fit, polished shoes, and soft indoor lighting reflecting " +
      "off a marble floor.",
  },
  {
    title: "Sunglasses",
    prompt:
      "Add a pair of dark tinted sunglasses resting naturally on the person's face, " +
      "smooth acetate frames, subtle reflections on the lenses, accurate nose " +
      "placement, and soft shadows across the cheeks.",
  },
];
```

For Lucy 2.1, use `set()` instead of `setPrompt()` to include a reference image:

```typescript theme={null}
// Lucy 2.1: transform into a character using a reference image
await realtimeClientRef.current.set({
  prompt: "Transform into this character",
  image: characterImage, // File, Blob, or URL
  enhance: true,
});
```

## Handling app lifecycle

Mobile apps frequently move between foreground and background. The example handles this by disconnecting when backgrounded and reconnecting when foregrounded:

```typescript theme={null}
import { AppState, type AppStateStatus } from "react-native";

useEffect(() => {
  const handleAppStateChange = (nextAppState: AppStateStatus) => {
    if (appStateRef.current.match(/inactive|background/) && nextAppState === "active") {
      // App returned to foreground — reconnect
      initializeWebRTC({ model, facingMode });
    } else if (appStateRef.current === "active" && nextAppState.match(/inactive|background/)) {
      // App going to background — disconnect to free resources
      cleanupWebRTC();
    }
    appStateRef.current = nextAppState;
  };

  const subscription = AppState.addEventListener("change", handleAppStateChange);
  return () => subscription?.remove();
}, [cleanupWebRTC, initializeWebRTC]);
```

<Tip>
  Always disconnect when backgrounded. WebRTC connections consume battery and network resources even when the app isn't visible.
</Tip>

## Camera controls

**Mirror the front camera.** The SDK's `mirror` option is browser-only and is rejected on React Native with `UNSUPPORTED_PLATFORM_FEATURE`. Mirror the local preview at the view layer instead — pass `mirror` to LiveKit's `<RTCView>`:

```typescript theme={null}
import { RTCView } from "@livekit/react-native-webrtc";

<RTCView
  mirror={facingMode === "user"}
  objectFit="cover"
  streamURL={localMediaStream.toURL()}
/>
```

**Switch facing mode.** The camera stream needs to be recaptured when toggling between front and back cameras, since each has different hardware capabilities.

## Adding Lucy 2.1 support

<Note>
  Adding Lucy 2.1 requires two small changes: updating the model selector and using the `set()` method for character reference images.
</Note>

**1. Add Lucy 2.1 to the model selector:**

```typescript theme={null}
import { models } from "@decartai/sdk";

// Add lucy-2.1 alongside the existing models
const handleTabPress = (modelName: "lucy-restyle-2" | "lucy-2.1") => {
  const newModel = models.realtime(modelName);
  onModelChange(newModel);
};
```

**2. Use `set()` for character reference images:**

Lucy 2.1's key feature is character transformation via reference images. Replace `setPrompt()` with `set()` when a reference image is available:

```typescript theme={null}
// For Lucy 2.1 with a reference image
await realtimeClientRef.current.set({
  prompt: "Transform into this character",
  image: selectedImage,
  enhance: true,
});

// For text-only editing (works with all models)
realtimeClientRef.current.setPrompt("Add sunglasses", { enhance: true });
```

## Next steps

<CardGroup cols={2}>
  <Card title="Lucy 2.1 guide" icon="sparkles" href="/models/realtime/lucy-2.1">
    Character transformation, reference images, and the `set()` API in detail.
  </Card>

  <Card title="JavaScript SDK" icon="js" href="/sdks/javascript-realtime">
    Full SDK reference for connection management, events, and client-side auth.
  </Card>

  <Card title="All realtime models" icon="bolt" href="/models/realtime/overview">
    Compare Lucy 2.1 and Lucy Restyle Live side by side.
  </Card>

  <Card title="Source code" icon="github" href="https://github.com/DecartAI/decart-example-expo-realtime">
    Clone the full example app and start building.
  </Card>
</CardGroup>