Agent-Based Immersive Theater

In-progress sketch — applying agent-based design to immersive and experimental performance.

This is an open notebook entry. I’ve been thinking about whether the same kind of systems-design work I do for robotics — specifying intent, defining the spatial and temporal constraints, monitoring the live state of a chaotic environment, and feeding what happens back into the design — could be useful inside immersive and experimental theater.

The diagrams below are a first cut. Two views: a process view of how a piece would be designed and run, and a system-architecture view of the agents and data that would back it. Both will change. I’m sharing them here because the act of putting them up is part of how I think.

Process view — design, performance, feedback

Pre-performance is where intent gets translated into the constraints a piece will run on: spatial boundaries, risk parameters, the moments where learning is supposed to happen for participants, and the physical and flow design that carries all of it. During performance, the piece is observed from three viewpoints simultaneously, with continuous monitoring of data, immersion, and risk. What gets learned flows back into the next iteration of intent analysis.

graph TD
    subgraph PrePerformance[Pre-Performance Phase]
        style PrePerformance fill:#e6f3ff
        A[Raw Intent] --> B[Intent Analysis]
        B --> C[Scene Design]
        B --> D[Success Metrics]

        C --> E[Spatiotemporal Boundaries]
        C --> F[Risk Parameters]
        C --> G[Learning Moments]

        C --> PHY[Physical Layout Design]
        PHY --> SPACE[Space Configuration]
        PHY --> PROPS[Props & Resources]
        PHY --> TECH[Technical Requirements]

        C --> FLOW[Participant Flow Design]
        FLOW --> PATH[Movement Pathways]
        FLOW --> CHOKE[Bottleneck Analysis]
        FLOW --> TIME[Timing Coordination]

        C --> ROLES[Role Definition]
        ROLES --> MAIN[Core Performers]
        ROLES --> SUP[Support Roles]
        ROLES --> PLANT[Strategic Plants]

        SPACE --> INTEGRATE[Integration Planning]
        PATH --> INTEGRATE
        MAIN --> INTEGRATE

        INTEGRATE --> H[Execution Framework]
        E --> H
        F --> H
        G --> H
        D --> H
    end

    subgraph Performance[Performance Phase]
        style Performance fill:#fff0e6
        H --> I[First Person View]
        H --> J[Second Person View]
        H --> K[Third Person View]

        I --> L[Integration Layer]
        J --> L
        K --> L

        L --> M[Feedback Loop]
        M --> N[Refinement]
        N --> B

        subgraph Continuous_Monitoring[Continuous Monitoring]
            O[Data Collection]
            P[Immersion Check]
            Q[Risk Assessment]
        end

        H --> O
        O --> P
        P --> Q
        Q --> M
    end

    classDef default fill:#f9f9f9,stroke:#333,stroke-width:1px
    classDef emphasis fill:#fff,stroke:#333,stroke-width:2px
    class A,H emphasis

System architecture — agents and data

A second view, looking at the underlying machinery: three databases (history, real-time state, configuration), a small set of specialized agents (spatial, script, role, flow, plus a system integration manager), and an integration layer that processes events, makes decisions, and runs analytics. The point isn’t the specific decomposition — it’s that immersive theater has enough moving parts to benefit from being modeled this way at all.

graph TD
    subgraph DataLayer[Data Layer]
        DB1[Performance History DB]
        DB2[Real-time State DB]
        DB3[Configuration DB]

        subgraph Historical[Historical Collections]
            H1[Past Performances] --> DB1
            H2[Agent Interactions] --> DB1
            H3[Intervention Logs] --> DB1
            H4[Outcome Metrics] --> DB1
        end

        subgraph Realtime[Real-time Collections]
            R1[Current State] --> DB2
            R2[Active Communications] --> DB2
            R3[Sensor Data] --> DB2
            R4[Alerts] --> DB2
        end

        subgraph Configuration[Configuration Collections]
            C1[Venue Layouts] --> DB3
            C2[Script Templates] --> DB3
            C3[Role Definitions] --> DB3
            C4[Success Metrics] --> DB3
        end
    end

    subgraph AgentLayer[Agent Layer]
        A1[System Integration Manager]
        A2[Spatial Planning Agent]
        A3[Script Generation Agent]
        A4[Role Management Agent]
        A5[Flow Control Agent]

        subgraph Protocol[Agent Communication Protocol]
            P1[State Updates]
            P2[Decision Requests]
            P3[Intervention Commands]
        end
    end

    subgraph IntegrationLayer[Integration Layer]
        I1[Event Stream Processor]
        I2[Decision Engine]
        I3[Analytics Engine]
    end

    DB1 --> I3
    DB2 --> I1
    DB3 --> A2
    DB3 --> A3
    DB3 --> A4

    A2 --> P1
    A3 --> P1
    A4 --> P1
    A5 --> P1
    P1 --> A1

    A2 --> P2
    A3 --> P2
    A4 --> P2
    A5 --> P2
    P2 --> I2

    A1 --> P3
    P3 --> A2
    P3 --> A3
    P3 --> A4
    P3 --> A5

    I1 --> I2
    I2 --> A1
    I3 --> I2

    R1 --> I1
    R2 --> I1
    R3 --> I1
    R4 --> I1

    I1 --> H2
    I2 --> H3
    I3 --> H4

    A2 --> C1
    A3 --> C2
    A4 --> C3
    A1 --> C4

    classDef database fill:#f9f9f9,stroke:#333,stroke-width:2px
    classDef agent fill:#e6f3ff,stroke:#333,stroke-width:2px
    classDef integration fill:#fff0e6,stroke:#333,stroke-width:2px

    class DB1,DB2,DB3 database
    class A1,A2,A3,A4,A5 agent
    class I1,I2,I3 integration

Where this might go

Open questions I’m sitting with: how much of the live agent layer should be automated vs. operated by humans in the room; what the right minimum viable version of this looks like for a single piece; and how the feedback loop from performance back into design gets captured without flattening the parts of the work that resist measurement.