> For the complete documentation index, see [llms.txt](https://docs.flap.sh/flap/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.flap.sh/flap/developers/preview/flap-video-generator.md).

# Flap Video Generator

{% hint style="warning" %}
**Preview.** This document has not been reviewed yet. Content may be incomplete or inaccurate.
{% endhint %}

## 1. Overview

**Flap Video Generator** lets any smart contract (a Flap vault, a game, a story contract) generate AI video clips **on-chain**. You call `generateVideo()` on `FlapAIProvider` with a text prompt and a flat BNB fee. Flap's backend generates the clip, pins it to IPFS, and writes the result back on-chain.

Each requester (`msg.sender`) owns an append-only **video session**. Each fulfilled clip is appended to it, and the clips are encoded so that playing them back-to-back gives **one continuous stream**. A contract can grow a story clip by clip, and anyone can watch the whole session as a single video.

Clips can be conditioned on **reference images** (IPFS CIDs). That gives you two separate controls:

* **Continuity:** the next clip starts exactly where the previous one ended.
* **Character consistency:** the same character keeps the same look across clips.

Section 4 explains how to combine the two, using a real deployed consumer as the example.

{% hint style="success" %}
**Live demo:** [video.munch.cash](https://video.munch.cash/) plays real on-chain sessions on BSC Testnet and BNB Mainnet (switch at the top of the page). Connect a wallet to extend the story with your own prompt.
{% endhint %}

```mermaid
flowchart TD
    subgraph on-chain["🔗 On-Chain"]
        Consumer(["🏦 Your Contract<br/>(vault / story consumer)"])
        Provider(["⚙️ FlapAIProvider"])
        Session(["🎞️ Video Session<br/>(per requester, append-only)"])
    end

    subgraph off-chain["☁️ Off-Chain"]
        Backend(["🤖 Flap Video Backend"])
        Model(["🎬 Video Model"])
        IPFS(["📦 IPFS<br/>(clips + last frames)"])
    end

    Consumer -->|"① generateVideo(prompt, referenceType, referenceCids) + fee"| Provider
    Provider -->|"② emit FlapAIProviderVideoRequested"| Backend
    Backend -->|"③ prompt + reference images"| Model
    Model -->|"④ video"| Backend
    Backend -->|"⑤ upload clip + last frame"| IPFS
    Backend -->|"⑥ submit clip CIDs"| Provider
    Provider -->|"⑦ append clip"| Session
    Provider -->|"⑧ onVideoRequestSettled(result)"| Consumer
```

### Key properties

| Property              | Value                                                                                                   |
| --------------------- | ------------------------------------------------------------------------------------------------------- |
| Fee                   | Flat fee per clip, read it from `videoGenerateFee()`; `msg.value` must match **exactly**                |
| Concurrency           | **One pending request per requester.** A second call reverts until the first one is fulfilled or failed |
| Output                | MPEG-TS clip CID + last-frame JPEG CID + duration, stored on-chain in the requester's session           |
| Clip format (current) | \~8 s, 720p, 16:9                                                                                       |
| Typical latency       | \~1–2 minutes from request to fulfillment                                                               |
| Callback              | Optional best-effort `IFlapVideoReceiver.onVideoRequestSettled`, which can never block fulfillment      |
| Failure               | Terminal `FAILED` with a machine-readable reason. **No refund**                                         |

***

## 2. Deployed Addresses

The video generator lives on the same `FlapAIProvider` contract as the [Flap AI Oracle](/flap/developers/preview/flap-ai-oracle.md).

| Network     | Chain ID | FlapAIProvider                                                                                                                 | Fee per clip | Callback gas limit |
| ----------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------ | ------------------ |
| BSC Testnet | 97       | [`0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744`](https://testnet.bscscan.com/address/0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744) | 0.01 BNB     | 2,000,000          |
| BNB Mainnet | 56       | [`0xaEe3a7Ca6fe6b53f6c32a3e8407eC5A9dF8B7E39`](https://bscscan.com/address/0xaEe3a7Ca6fe6b53f6c32a3e8407eC5A9dF8B7E39)         | 0.01 BNB     | 2,000,000          |

{% hint style="warning" %}
**Always call `videoGenerateFee()` at request time. Never hardcode the fee.** It is configurable and may change. A wrong `msg.value` (too much or too little) reverts with `FlapAIProviderWrongVideoFee`.
{% endhint %}

{% hint style="info" %}
**Testnet note:** video generation is expensive, so the testnet backend runs on a limited budget and may stop fulfilling if it runs out. If you'd like to test on testnet, feel free to reach out to us.
{% endhint %}

### Demo consumers

Both are `ReferenceStoryVideoConsumer` (§4) and can be watched and extended in the [demo app](https://video.munch.cash/).

| Network     | Consumer                                                                                                                       | Character               | Story                           | Watch                                                                          |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | ------------------------------- | ------------------------------------------------------------------------------ |
| BSC Testnet | [`0xB7f4398BEECE5Ec0da06497e647107D697c55c66`](https://testnet.bscscan.com/address/0xB7f4398BEECE5Ec0da06497e647107D697c55c66) | Ice boy (2D cartoon)    | *The Ice Boy Who Melted Upward* | [video.munch.cash/?network=testnet](https://video.munch.cash/?network=testnet) |
| BNB Mainnet | [`0x8b527D3f104A1945BD68B391fBd6b4a00B7EA3a2`](https://bscscan.com/address/0x8b527D3f104A1945BD68B391fBd6b4a00B7EA3a2)         | Arthur (photorealistic) | *Arthur's Boat*, 8 clips, 64 s  | [video.munch.cash/?network=mainnet](https://video.munch.cash/?network=mainnet) |

***

## 3. Reference Types

`generateVideo` takes a `uint8 referenceType` that tells the backend how to use the reference image CIDs you pass:

| Code | Constant                         | `referenceCids`                               | How the image is used                                                                                        | Enabled             |
| ---- | -------------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | ------------------- |
| `0`  | `VIDEO_REF_NONE`                 | empty                                         | Text-to-video, no image                                                                                      | ✅                   |
| `1`  | `VIDEO_REF_PREV_LAST_FRAME`      | **empty** (the provider resolves it on-chain) | The previous clip's **last frame becomes the exact first frame** of the new clip                             | ✅                   |
| `2`  | `VIDEO_REF_FIRST_FRAME`          | exactly 1                                     | Your image becomes the exact first frame                                                                     | ✅                   |
| `3`  | `VIDEO_REF_FIRST_AND_LAST_FRAME` | exactly 2 `[first, last]`                     | Pinned first and last frames                                                                                 | ❌ (not enabled yet) |
| `4`  | `VIDEO_REF_CHARACTER`            | 1–3                                           | **Soft references** that guide the character's look, subject and style. They are **not** pinned to any frame | ✅                   |

Rules enforced on-chain:

* At most `MAX_VIDEO_REFERENCE_CIDS = 3` CIDs per request, each non-empty and ≤ `MAX_VIDEO_REFERENCE_CID_LENGTH = 128` bytes.
* A disabled type reverts with `FlapAIProviderUnsupportedReferenceType`. Check `isVideoReferenceTypeEnabled(type)` first.
* Type `1` reverts with `FlapAIProviderNoPreviousClip` if the requester has no clip yet. The first clip of a session must use another type.
* A wrong CID count reverts with `FlapAIProviderBadReferenceCidCount`.

{% hint style="info" %}
The legacy one-argument `generateVideo(prompt)` is still supported. It automatically uses type `0` for the first clip and type `1` for every later clip.
{% endhint %}

### Frame references vs. character references: you can't fully have both in one clip

This is the most important thing to understand when designing your consumer:

```mermaid
flowchart LR
    subgraph T1["Type 1: PREV_LAST_FRAME"]
        A1["Previous clip's<br/>last frame"] -->|"pinned as frame 0"| B1["New clip"]
    end
    subgraph T4["Type 4: CHARACTER"]
        A4["Character image"] -->|"identity guide"| B4["New clip"]
        C4["Previous last frame<br/>(optional)"] -->|"scene guide only"| B4
    end
```

|                                      | **Type 1 (previous last frame)**                                                                                                                              | **Type 4 (character + last frame as references)**                   |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- |
| Seamless join with the previous clip | ✅ **Pixel-exact.** The new clip opens on the old clip's final frame                                                                                           | ⚠️ Same scene, setting and style, but a **visible cut** at the join |
| Character consistency                | ⚠️ Carried only through the pixels of the starting frame. Details **drift** over many clips (e.g. the model adds clothing that wasn't in the original design) | ✅ **Strong.** The character image is re-applied on every clip       |

The current video model can't take a pinned starting frame **and** a soft character reference in the same clip. When a frame is pinned, soft references are ignored. That is why these are two separate types and not one combined mode.

**Recommended pattern: alternate the two types.**

* Use **type 4** `[character, previous last frame]` to (re)anchor the character, at the start of each "scene".
* Use **type 1** for the following beat(s) to keep motion seamless.

Every type-4 clip resets any drift from the type-1 clips before it, and the type-1 clips keep the story flowing without cuts.

**Prompt tips:**

* In type 4, **say which reference image is which** at the start of the prompt (e.g. *"Reference image 1 is the main character … Reference image 2 is the final frame of the previous clip …"*).
* Restate the character's key traits (colours, outfit, style) in the prompt. The model then follows them more closely, in both types.

***

## 4. Example: `ReferenceStoryVideoConsumer`

`ReferenceStoryVideoConsumer` is a small contract that builds a continuous story around **one fixed character**. Anyone can pay the fee to add the next clip and choose, per clip, between:

* **Type 1:** continue seamlessly from the previous clip.
* **Type 4:** re-anchor on the character, with the previous last frame as a scene hint.

The contract checks the reference CIDs and adds the "which image is which" prefix to the prompt itself, so the prompt always matches the images.

|                 |                                                                                                                                                  |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Network         | BSC Testnet                                                                                                                                      |
| Consumer        | [`0xB7f4398BEECE5Ec0da06497e647107D697c55c66`](https://testnet.bscscan.com/address/0xB7f4398BEECE5Ec0da06497e647107D697c55c66) (source verified) |
| Provider        | `0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744`                                                                                                     |
| Character image | `bafkreigo6g3mkveu5w3l7ud56qr4oq3sa62hawcdmybdhbi5agurqwm5ye` (an "ice boy", flat 2D cartoon style)                                              |

### Request flow

```mermaid
sequenceDiagram
    participant U as Caller (anyone)
    participant C as ReferenceStoryVideoConsumer
    participant P as FlapAIProvider
    participant B as Flap Video Backend

    U->>C: generateVideo(prompt, type, cids) + fee
    C->>P: getLastVideoFrameCid(this)
    alt type 1 (PREV_LAST_FRAME)
        C->>C: require a previous clip, cids == []
    else type 4 (CHARACTER)
        C->>C: require cids == [CHARACTER_CID] (first clip)<br/>or [CHARACTER_CID, lastFrame]
        C->>C: prefix prompt with "image 1 = character, image 2 = previous scene"
    end
    C->>P: generateVideo{value: fee}(fullPrompt, type, cids)
    P-->>B: FlapAIProviderVideoRequested
    B->>P: submit clip (videoCid, lastFrameCid, duration)
    P->>C: onVideoRequestSettled(result)
    C->>C: store latest clip / failure
```

### The contract

```solidity
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.13;

import {IFlapAIProvider, IFlapVideoReceiver} from "src/plugins/AIProvider/IFlapAIProvider.sol";

/// @dev Reference-type surface of FlapAIProvider used by this consumer.
interface IFlapAIProviderReferenceTypes {
    function generateVideo(string calldata prompt, uint8 referenceType, string[] calldata referenceCids)
        external
        payable
        returns (uint256 requestId);
    function getLastVideoFrameCid(address user) external view returns (string memory);
}

contract ReferenceStoryVideoConsumer is IFlapVideoReceiver {
    uint8 public constant VIDEO_REF_PREV_LAST_FRAME = 1;
    uint8 public constant VIDEO_REF_CHARACTER = 4;

    string internal constant CHARACTER_HINT =
        "Reference image 1 is the main character: keep the exact same character design, colors, outfit and art style. ";
    string internal constant SCENE_HINT =
        "Reference image 2 is the final frame of the previous clip: continue from that scene, same setting, lighting and camera framing, as a natural continuation. ";

    IFlapAIProvider public immutable provider;
    string public CHARACTER_CID;

    uint256 public lastRequestId;
    string public latestVideoCid;
    string public latestFrameCid;
    uint32 public latestDurationMs;
    bool public lastRequestFailed;
    IFlapAIProvider.VideoFailureReason public lastFailureReason;
    string public lastFailureDetail;

    event VideoGenerationRequested(
        uint256 indexed requestId, address indexed caller, string prompt, uint8 referenceType, string[] referenceCids
    );
    event VideoGenerationFulfilled(uint256 indexed requestId, string videoCid, string lastFrameCid, uint32 durationMs);
    event VideoGenerationFailed(uint256 indexed requestId, IFlapAIProvider.VideoFailureReason reason, string failureDetail);

    error WrongFee(uint256 sent, uint256 required);
    error EmptyPrompt();
    error OnlyProvider();
    error RequestAlreadyPending(uint256 requestId);
    error ReferenceTypeNotAllowed(uint8 referenceType);
    error NoPreviousClip();
    error BadCharacterReferences();

    constructor(address provider_, string memory characterCid_) {
        require(provider_ != address(0), "zero provider");
        require(bytes(characterCid_).length != 0, "empty character cid");
        provider = IFlapAIProvider(provider_);
        CHARACTER_CID = characterCid_;
    }

    /// @param referenceType VIDEO_REF_PREV_LAST_FRAME (1) or VIDEO_REF_CHARACTER (4).
    /// @param referenceCids Type 1: empty. Type 4: [CHARACTER_CID] for the first clip,
    ///                      [CHARACTER_CID, current last frame] afterwards.
    function generateVideo(string calldata prompt, uint8 referenceType, string[] calldata referenceCids)
        external
        payable
        returns (uint256 requestId)
    {
        uint256 fee = provider.videoGenerateFee();
        if (msg.value != fee) revert WrongFee(msg.value, fee);
        if (bytes(prompt).length == 0) revert EmptyPrompt();
        if (lastRequestId != 0) {
            IFlapAIProvider.VideoRequestView memory prior = provider.getVideoRequest(lastRequestId);
            if (prior.status == IFlapAIProvider.VideoRequestStatus.PENDING) revert RequestAlreadyPending(lastRequestId);
        }

        IFlapAIProviderReferenceTypes p = IFlapAIProviderReferenceTypes(address(provider));
        // No race: the provider's one-pending rule means the last frame can't change
        // before this request is recorded.
        string memory lastFrame = p.getLastVideoFrameCid(address(this));
        bool hasLastFrame = bytes(lastFrame).length != 0;

        string memory fullPrompt;
        if (referenceType == VIDEO_REF_PREV_LAST_FRAME) {
            // Continuity: the provider resolves the previous last frame on-chain.
            if (!hasLastFrame) revert NoPreviousClip();
            if (referenceCids.length != 0) revert BadCharacterReferences();
            fullPrompt = prompt;
        } else if (referenceType == VIDEO_REF_CHARACTER) {
            // Character consistency: [character] or [character, previous last frame].
            uint256 expected = hasLastFrame ? 2 : 1;
            if (referenceCids.length != expected) revert BadCharacterReferences();
            if (keccak256(bytes(referenceCids[0])) != keccak256(bytes(CHARACTER_CID))) revert BadCharacterReferences();
            if (hasLastFrame && keccak256(bytes(referenceCids[1])) != keccak256(bytes(lastFrame))) {
                revert BadCharacterReferences();
            }
            fullPrompt = hasLastFrame
                ? string.concat(CHARACTER_HINT, SCENE_HINT, prompt)
                : string.concat(CHARACTER_HINT, prompt);
        } else {
            revert ReferenceTypeNotAllowed(referenceType);
        }

        requestId = p.generateVideo{value: msg.value}(fullPrompt, referenceType, referenceCids);
        lastRequestId = requestId;
        emit VideoGenerationRequested(requestId, msg.sender, prompt, referenceType, referenceCids);
    }

    function onVideoRequestSettled(VideoRequestResult calldata result) external override {
        if (msg.sender != address(provider)) revert OnlyProvider();
        if (result.status == IFlapAIProvider.VideoRequestStatus.FULFILLED) {
            latestVideoCid = result.videoCid;
            latestFrameCid = result.lastFrameCid;
            latestDurationMs = result.durationMs;
            lastRequestFailed = false;
            emit VideoGenerationFulfilled(result.requestId, result.videoCid, result.lastFrameCid, result.durationMs);
        } else if (result.status == IFlapAIProvider.VideoRequestStatus.FAILED) {
            lastRequestFailed = true;
            lastFailureReason = result.failureReason;
            lastFailureDetail = result.failureDetail;
            emit VideoGenerationFailed(result.requestId, result.failureReason, result.failureDetail);
        }
    }

    // ---- views ----

    function getLastFrame() external view returns (string memory) {
        return IFlapAIProviderReferenceTypes(address(provider)).getLastVideoFrameCid(address(this));
    }

    function getFullSession() external view returns (IFlapAIProvider.VideoClip[] memory) {
        uint256 len = provider.getVideoSessionLength(address(this));
        return provider.getVideoSessionSlice(address(this), 0, len);
    }
}
```

**Design notes:**

1. **The session belongs to the consumer contract**, because it is the `msg.sender` of the provider call. Many callers can extend one shared story.
2. **The character CID is fixed in the constructor.** Type-4 calls must pass it as reference image 1, so no caller can swap the character.
3. **The last frame is read from the provider** (`getLastVideoFrameCid(address(this))`), not from the consumer's own callback storage. The value is still correct if a callback was skipped or ran out of gas.
4. **The prompt prefix matches the CID order the contract enforces.** "Image 1 is the character, image 2 is the previous scene" is always true.
5. **The callback only records state.** It is best-effort and gas-capped, so heavy logic doesn't belong there.

### Example: an 8-clip story

This 8-clip story, *"The Ice Boy Who Melted Upward"*, was generated on the testnet consumer above. It alternates the two types as recommended in §3:

| # | Type | `referenceCids`          | Purpose                        | Prompt (abridged)                                                                 |
| - | ---- | ------------------------ | ------------------------------ | --------------------------------------------------------------------------------- |
| 1 | 4    | `[character]`            | Establish the character        | The ice boy wakes up alone inside a giant snow globe on a kitchen table…          |
| 2 | 1    | `[]`                     | Seamless continuation          | The glass softens like jelly; he pushes through and drops onto the table…         |
| 3 | 4    | `[character, lastFrame]` | New scene, re-anchor character | He follows floating ice cubes out of the window into a sky where it snows upward… |
| 4 | 1    | `[]`                     | Seamless continuation          | A whale made of folded newspaper swims past; he rides its fin higher…             |
| 5 | 4    | `[character, lastFrame]` | New scene, re-anchor character | The whale delivers him to a floating island where tiny suns sunbathe…             |
| 6 | 1    | `[]`                     | Seamless continuation          | He melts, but his water flows upward into a cloud shaped like his face…           |
| 7 | 4    | `[character, lastFrame]` | Re-anchor character            | The cloud rains back down and rebuilds him, covered in crystal flowers…           |
| 8 | 1    | `[]`                     | Seamless ending                | He shakes a new snow globe; the camera reveals the whole world is inside one…     |

Calling it with `cast`:

```bash
CONSUMER=0xB7f4398BEECE5Ec0da06497e647107D697c55c66
CHAR=bafkreigo6g3mkveu5w3l7ud56qr4oq3sa62hawcdmybdhbi5agurqwm5ye
RPC=https://bsc-testnet-rpc.publicnode.com
FEE=$(cast call 0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744 "videoGenerateFee()(uint256)" --rpc-url $RPC | awk '{print $1}')

# Type 4: re-anchor the character, previous last frame as scene reference
LAST=$(cast call $CONSUMER "getLastFrame()(string)" --rpc-url $RPC | tr -d '"')
cast send $CONSUMER "generateVideo(string,uint8,string[])" \
  "The ice boy follows a trail of floating ice cubes into a sky where it snows upward." \
  4 "[$CHAR,$LAST]" --value $FEE --rpc-url $RPC --private-key <KEY>

# Type 1: seamless continuation (wait for the previous request to settle first)
cast send $CONSUMER "generateVideo(string,uint8,string[])" \
  "A whale made of folded newspaper swims past; the ice boy grabs its fin and rides it higher." \
  1 "[]" --value $FEE --rpc-url $RPC --private-key <KEY>
```

***

## 5. Playing a Session

A session is an ordered list of `VideoClip`s. Every clip is a separate, immutable **MPEG-TS** file on IPFS (`videoCid`). Together they are designed to play as **one continuous video**: load the clip list from the chain, give it to an MPEG-TS player as consecutive segments, and the viewer sees one timeline across all clips.

### Why MPEG-TS

Every `VideoClip` carries `startPtsMs`, its start time on the session's timeline: `0` for the first clip, then the previous clip's `startPtsMs + durationMs` (the cumulative duration of all earlier clips). The backend encodes each clip with its timestamps **already shifted by that offset**, so clip *N*'s first frame is stamped exactly where clip *N-1* ended.

This only works because of how MPEG-TS is built:

* **No global header.** An MPEG-TS file is a flat sequence of self-describing packets with absolute timestamps. Two TS files placed back-to-back (as player segments, or even byte-concatenated) are still a valid stream.
* **Timestamps live in the stream.** Each clip can be encoded once with its final position on the session timeline. Neither the player nor any server has to rewrite anything.
* **Clean splices.** Clips are encoded with **no B-frames** (so presentation and decode timestamps are always equal) and **start with a keyframe**. Decode time never jumps backwards at a clip boundary, and every clip can be decoded on its own from its first packet.

What this means for you:

|                         | MPEG-TS (what the session uses)                  | Plain MP4 per clip                                                 |
| ----------------------- | ------------------------------------------------ | ------------------------------------------------------------------ |
| Append a new clip       | Upload one more file; earlier clips never change | Each file has its own header (`moov`) and a timeline starting at 0 |
| Play clips back-to-back | One continuous timeline, no reset between clips  | Player resets or seeks at every clip, or you re-mux everything     |
| Infrastructure          | Static files on IPFS + the clip list on-chain    | Needs re-encoding or a server-generated HLS/DASH manifest          |
| Content addressing      | Each clip CID is final the moment it's on-chain  | A merged file would get a new CID every time it grows              |

In short: the session is **append-only and content-addressed**, and MPEG-TS is the format where "append one more immutable file" and "play everything as one video" are compatible. The on-chain clip list **is** the playlist.

{% hint style="info" %}
`durationMs` and `startPtsMs` are in **milliseconds** on-chain. mpegts.js segment durations are in **seconds**, so always pass `durationMs / 1000`.
{% endhint %}

### Reading the session

From a contract:

```solidity
uint256 len = provider.getVideoSessionLength(consumer);
IFlapAIProvider.VideoClip[] memory clips = provider.getVideoSessionSlice(consumer, 0, len);
// clips[i].videoCid      → MPEG-TS segment
// clips[i].durationMs    → segment duration (ms)
// clips[i].startPtsMs    → segment start on the session timeline (ms)
// clips[i].lastFrameCid  → thumbnail / reference for the next clip
```

From a frontend, with [viem](https://viem.sh). `user` is the address that called `generateVideo` (usually your consumer contract):

```ts
import { createPublicClient, http, type Address } from "viem";
import { bsc } from "viem/chains"; // use `bscTestnet` for testnet

// FlapAIProvider: BNB Mainnet. Testnet: 0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744
const FLAP_AI_PROVIDER: Address = "0xaEe3a7Ca6fe6b53f6c32a3e8407eC5A9dF8B7E39";

const abi = [
  {
    type: "function",
    name: "getVideoSessionLength",
    stateMutability: "view",
    inputs: [{ name: "user", type: "address" }],
    outputs: [{ name: "", type: "uint256" }],
  },
  {
    type: "function",
    name: "getVideoSessionSlice",
    stateMutability: "view",
    inputs: [
      { name: "user", type: "address" },
      { name: "start", type: "uint256" },
      { name: "count", type: "uint256" },
    ],
    outputs: [
      {
        name: "clips",
        type: "tuple[]",
        components: [
          { name: "requestId", type: "uint256" },
          { name: "videoCid", type: "string" },
          { name: "lastFrameCid", type: "string" },
          { name: "durationMs", type: "uint32" },
          { name: "startPtsMs", type: "uint64" },
          { name: "createdAt", type: "uint64" },
          { name: "referenceType", type: "uint8" },
        ],
      },
    ],
  },
] as const;

const client = createPublicClient({ chain: bsc, transport: http() });

/** Reads every fulfilled clip of `user`'s session, in timeline order. */
export async function fetchSession(user: Address) {
  const length = await client.readContract({
    address: FLAP_AI_PROVIDER,
    abi,
    functionName: "getVideoSessionLength",
    args: [user],
  });
  if (length === 0n) return [];

  return client.readContract({
    address: FLAP_AI_PROVIDER,
    abi,
    functionName: "getVideoSessionSlice",
    args: [user, 0n, length],
  });
  // → readonly { requestId: bigint; videoCid: string; lastFrameCid: string;
  //     durationMs: number; startPtsMs: bigint; createdAt: bigint; referenceType: number }[]
}

export type VideoClip = Awaited<ReturnType<typeof fetchSession>>[number];
```

For long sessions, page through with `getVideoSessionSlice(user, start, count)`. It clamps `count` to the end of the session.

### Playing with mpegts.js

[mpegts.js](https://github.com/xqq/mpegts.js) is an open-source HTML5 MPEG-TS player. It transmuxes TS to fragmented MP4 in the browser and plays it through **Media Source Extensions (MSE)**. Give it **all clips as one multi-segment source**: it loads and stitches the segments itself, and the native `<video>` seek bar covers the whole session.

```bash
npm install mpegts.js
```

Segment URLs point at any IPFS gateway: `https://<your-gateway>/ipfs/<videoCid>`. The gateway must send CORS headers (`Access-Control-Allow-Origin`), because mpegts.js downloads the segments with `fetch`. A dedicated gateway is recommended for anything beyond testing.

#### Minimal example (plain HTML, no build step)

A single HTML file that reads a session from BSC Testnet and plays it:

```html
<!doctype html>
<html>
<body>
  <video id="video" controls playsinline style="width: 100%; max-width: 960px"></video>

  <!-- mpegts.js UMD build: exposes a global `mpegts` -->
  <script src="https://cdn.jsdelivr.net/npm/mpegts.js@1.8.2/dist/mpegts.js"></script>
  <script type="module">
    import { createPublicClient, http, parseAbi } from "https://esm.sh/viem@2";
    import { bscTestnet } from "https://esm.sh/viem@2/chains";

    const PROVIDER = "0xFfddcE44e8cFf7703Fd85118524bfC8B2f70b744"; // FlapAIProvider (BSC Testnet)
    const USER = "0x...";                     // the requester whose session you want to play
    const GATEWAY = "https://<your-gateway>"; // IPFS gateway with CORS enabled

    const abi = parseAbi([
      "function getVideoSessionLength(address user) view returns (uint256)",
      "function getVideoSessionSlice(address user, uint256 start, uint256 count) view returns ((uint256 requestId, string videoCid, string lastFrameCid, uint32 durationMs, uint64 startPtsMs, uint64 createdAt, uint8 referenceType)[])",
    ]);
    const client = createPublicClient({ chain: bscTestnet, transport: http() });

    const len = await client.readContract({
      address: PROVIDER, abi, functionName: "getVideoSessionLength", args: [USER],
    });
    const clips = await client.readContract({
      address: PROVIDER, abi, functionName: "getVideoSessionSlice", args: [USER, 0n, len],
    });

    const video = document.getElementById("video");
    if (clips.length > 0 && mpegts.isSupported()) {
      const player = mpegts.createPlayer({
        type: "mse",
        isLive: false,
        segments: clips.map((c) => ({
          url: `${GATEWAY}/ipfs/${c.videoCid}`,
          duration: c.durationMs / 1000, // seconds
        })),
      });
      player.attachMediaElement(video);
      player.load();
      player.play()?.catch(() => {}); // autoplay may need a user click first
    } else {
      // Fallback: no MSE (or empty session). Offer the individual clips instead.
      video.replaceWith(...clips.map((c, i) => {
        const a = document.createElement("a");
        a.href = `${GATEWAY}/ipfs/${c.videoCid}`;
        a.textContent = `Clip ${i + 1} (.ts) `;
        return a;
      }));
    }
  </script>
</body>
</html>
```

#### React / Next.js component

Sessions grow over time, so a real app re-reads the session (for example every few seconds) and appends new clips as they're fulfilled. mpegts.js **can't add a segment to a running player**, so the component below rebuilds the player whenever a new clip shows up. It then seeks back to the viewer's current position, so the rebuild doesn't look like a restart.

```tsx
"use client";

import { useEffect, useRef } from "react";
import type Mpegts from "mpegts.js";

type Player = ReturnType<typeof Mpegts.createPlayer>;

export interface Clip {
  requestId: bigint;
  videoCid: string;
  durationMs: number;
}

const GATEWAY = "https://<your-gateway>"; // IPFS gateway with CORS enabled

export function SessionPlayer({ clips }: { clips: readonly Clip[] }) {
  const videoRef = useRef<HTMLVideoElement>(null);
  const playerRef = useRef<Player | null>(null);
  const builtForRef = useRef<bigint | null>(null); // requestId of the newest clip loaded

  // (Re)build the player whenever a new clip is appended to the session.
  useEffect(() => {
    const video = videoRef.current;
    if (!video || clips.length === 0) return;

    const newest = clips[clips.length - 1].requestId;
    if (builtForRef.current === newest) return; // nothing new (e.g. a poll re-render)

    const isFirstBuild = builtForRef.current === null;
    const resumeAt = isFirstBuild ? 0 : video.currentTime;
    const shouldPlay = isFirstBuild || video.ended || !video.paused;
    let cancelled = false;

    (async () => {
      // Dynamic import: mpegts.js touches `window`, so never load it during SSR.
      const mpegts = (await import("mpegts.js")).default;
      if (cancelled || !mpegts.isSupported()) return; // no MSE: render a fallback instead

      // mpegts.js cannot append a segment to a running player, so replace it.
      playerRef.current?.destroy();
      const player = mpegts.createPlayer({
        type: "mse",
        isLive: false,
        segments: clips.map((c) => ({
          url: `${GATEWAY}/ipfs/${c.videoCid}`,
          duration: c.durationMs / 1000, // SECONDS, not milliseconds
        })),
      });
      playerRef.current = player;
      player.attachMediaElement(video);
      player.load();
      builtForRef.current = newest;

      // Put the viewer back where they were, so the rebuild is invisible.
      if (resumeAt > 0) video.currentTime = resumeAt;
      if (shouldPlay) {
        const p = player.play();
        if (p) p.catch(() => {}); // autoplay may be blocked until the user interacts
      }
    })();

    return () => {
      cancelled = true;
    };
  }, [clips]);

  // Tear down on unmount.
  useEffect(
    () => () => {
      playerRef.current?.destroy();
      playerRef.current = null;
    },
    []
  );

  return <video ref={videoRef} controls playsInline style={{ width: "100%" }} />;
}
```

Usage: keep the clips from `fetchSession(user)` in client state, re-fetch on an interval, and render `<SessionPlayer clips={clips} />`. Only replace the array when the session actually changed (compare length / last `requestId`), so that background polls don't trigger rebuilds.

**Jump to a clip.** Since `startPtsMs` is each clip's position on the timeline, seeking to clip `i` is `video.currentTime = Number(clips[i].startPtsMs) / 1000`. To highlight the clip that's playing, find the last clip whose `startPtsMs <= video.currentTime * 1000`.

Seeking inside a player built from every clip makes mpegts.js download all earlier clips first, which is slow for later clips. A faster option (used by the [demo app](https://video.munch.cash/)) is to rebuild the player from `clips.slice(i)`, so only clip `i` and later are downloaded. mpegts.js then starts its timeline at clip `i`, so add `clips[i].startPtsMs` back when mapping `video.currentTime` to the session timeline.

### Caching clips

Clips are content-addressed, so a `/ipfs/<cid>` URL never changes and can be cached forever. The browser HTTP cache alone is not reliable for them, especially on mobile: clips are 5–6 MB each and mobile disk caches evict them quickly. The demo app adds a small service worker ([`clip-cache-sw.js`](https://github.com/flap-sh/video-generator-demo-dapp/blob/main/public/clip-cache-sw.js)) that keeps every played clip in Cache Storage, keyed by URL, and answers later requests (including `Range` requests) from there. Browsers may still clear site storage under pressure, and iOS Safari clears it after about 7 days without a visit.

### Browser support and fallbacks

* mpegts.js needs **Media Source Extensions**. Always check `mpegts.isSupported()` before creating a player.
* Desktop Chrome, Edge, Firefox and Safari are supported. On **iPhone**, MSE isn't available before iOS 17.1. Newer iOS versions go through Apple's Managed Media Source, which mpegts.js supports, but test on your target devices.
* If MSE is unavailable, a simple fallback is to link or download the individual clips (`https://<your-gateway>/ipfs/<videoCid>`) or show the `lastFrameCid` images as a storyboard.
* **Downloading a whole session:** because the clips are MPEG-TS with continuous timestamps, concatenating their bytes in order gives one valid `.ts` file (for example `cat clip0.ts clip1.ts … > session.ts`). It plays in desktop players such as VLC and mpv, or can be converted to MP4 without re-encoding, e.g. `ffmpeg -i session.ts -c copy session.mp4`.

***

## 6. Interface Reference

### Requesting

```solidity
/// Text-to-video for the first clip, then automatically PREV_LAST_FRAME (legacy).
function generateVideo(string calldata prompt) external payable returns (uint256 requestId);

/// Explicit reference type + caller-supplied reference CIDs (see §3).
function generateVideo(string calldata prompt, uint8 referenceType, string[] calldata referenceCids)
    external payable returns (uint256 requestId);

function videoGenerateFee() external view returns (uint256);
function isVideoReferenceTypeEnabled(uint8 referenceType) external view returns (bool);
```

### Reading

```solidity
function getVideoRequest(uint256 requestId) external view returns (VideoRequestView memory);
function getVideoReferenceCids(uint256 requestId) external view returns (uint8 referenceType, string[] memory referenceCids);
function getVideoSessionLength(address user) external view returns (uint256);
function getVideoClip(address user, uint256 index) external view returns (VideoClip memory);
function getVideoSessionSlice(address user, uint256 start, uint256 count) external view returns (VideoClip[] memory);
function getLastVideoFrameCid(address user) external view returns (string memory);
function getLastVideoRequestId(address user) external view returns (uint256);
function videoCallbackGasLimit() external view returns (uint256);
```

### Data types

```solidity
enum VideoRequestStatus { NONE, PENDING, FULFILLED, FAILED }

enum VideoFailureReason { NONE, CONTENT_POLICY, PROVIDER_ERROR, PIPELINE_ERROR, TIMEOUT, OTHER }

struct VideoClip {
    uint256 requestId;
    string videoCid;      // MPEG-TS clip
    string lastFrameCid;  // JPEG of the clip's last frame
    uint32 durationMs;
    uint64 startPtsMs;    // start on the session's continuous timeline
    uint64 createdAt;
    uint8 referenceType;  // VIDEO_REF_* used for this clip
}

struct VideoRequestView {
    uint256 requestId;
    address requester;
    VideoRequestStatus status;
    uint8 referenceType;
    uint64 timestamp;
    uint128 feePaid;
    VideoFailureReason failureReason;
}
```

### Callback (optional)

```solidity
interface IFlapVideoReceiver {
    struct VideoRequestResult {
        uint256 requestId;
        IFlapAIProvider.VideoRequestStatus status; // FULFILLED or FAILED
        string videoCid;                            // FULFILLED only
        string lastFrameCid;                        // FULFILLED only
        uint32 durationMs;                          // FULFILLED only
        IFlapAIProvider.VideoFailureReason failureReason; // FAILED only
        string failureDetail;                       // FAILED only
    }

    function onVideoRequestSettled(VideoRequestResult calldata result) external;
}
```

The callback runs **after** the clip has been appended to the session. It gets at most `videoCallbackGasLimit()` gas. A revert, out-of-gas, or missing implementation is tolerated and recorded in `FlapAIProviderVideoCallbackAttempted(requestId, requester, success)`. EOA requesters are never called back.

`CONTENT_POLICY` means the same prompt would be refused again, so change it before retrying. `PROVIDER_ERROR` / `TIMEOUT` mean a fresh request may succeed. A `FAILED` request never adds a clip, so the next type-1 request still continues from the last **fulfilled** clip.

### Events

```solidity
event FlapAIProviderVideoRequested(
    uint256 indexed requestId, address indexed requester, string prompt,
    uint8 referenceType, string[] referenceCids, uint256 feePaid
);
event FlapAIProviderVideoFulfilled(
    uint256 indexed requestId, address indexed requester, string videoCid,
    string lastFrameCid, uint32 durationMs, uint64 startPtsMs, uint256 sessionIndex
);
event FlapAIProviderVideoFailed(
    uint256 indexed requestId, address indexed requester, VideoFailureReason reason, string failureDetail
);
event FlapAIProviderVideoCallbackAttempted(uint256 indexed requestId, address indexed requester, bool success);
```

In `FlapAIProviderVideoRequested`, `referenceCids` is already resolved: for type `1` it holds the previous clip's last-frame CID.

### Errors

| Error                                                                   | When                                        |
| ----------------------------------------------------------------------- | ------------------------------------------- |
| `FlapAIProviderWrongVideoFee(sent, required)`                           | `msg.value != videoGenerateFee()`           |
| `FlapAIProviderEmptyVideoPrompt()`                                      | Empty prompt                                |
| `FlapAIProviderVideoRequestAlreadyPending(requester, pendingRequestId)` | The requester already has a pending request |
| `FlapAIProviderUnsupportedReferenceType(referenceType)`                 | Type not enabled                            |
| `FlapAIProviderBadReferenceCidCount(referenceType, count)`              | Wrong number of CIDs for the type           |
| `FlapAIProviderInvalidReferenceCid(index)`                              | Empty CID or CID > 128 bytes                |
| `FlapAIProviderNoPreviousClip(requester)`                               | Type `1` with an empty session              |

***

## 7. Checklist

* [ ] Read `videoGenerateFee()` at call time and send exactly that amount
* [ ] Handle the one-pending-request rule (check `getVideoRequest(lastRequestId).status` before submitting)
* [ ] Start every session with type `0`, `2` or `4`. Type `1` needs a previous clip
* [ ] Pin your character image to IPFS and keep its CID fixed in your contract
* [ ] Use type `4` `[character, lastFrame]` to keep the character consistent, and type `1` for seamless joins. Alternate them
* [ ] In type-4 prompts, say which reference image is which, and restate the character's key traits
* [ ] Read the last frame from `getLastVideoFrameCid(address(this))`, not from callback-only storage
* [ ] Keep `onVideoRequestSettled` light and check `msg.sender == provider`