How Replay’s Flow Map Automatically Detects Application State Machines

Most developers spend 60% of their time trying to understand how an existing application works before they write a single line of new code. When you inherit a legacy React app or a messy Figma prototype, the logic isn't in the documentation—it’s hidden in the transitions between screens. You click a button, a loading spinner appears, an API call fires, and the UI updates. Mapping these states manually is a recipe for errors and missed edge cases.

Visual Reverse Engineering is the process of extracting functional logic, design tokens, and state transitions from a visual recording of a user interface. Replay pioneered this approach to eliminate the manual overhead of legacy modernization.

TL;DR: Replay’s Flow Map uses temporal video context to identify how an application moves between states. By recording a user session, replays flow automatically detects navigation patterns, conditional logic, and state machine transitions, converting them into production-ready React code. This cuts modernization time from 40 hours per screen to just 4 hours.

What is an Application State Machine?#

An application state machine is the underlying logic that governs how a user interface behaves. It defines every possible "mode" the app can be in (e.g.,

, , , ) and the specific triggers that move the user from one mode to another.

In legacy systems, this logic is often "spaghetti code"—nested if-else statements and scattered

hooks that make the app nearly impossible to refactor. Replay solves this by observing the behavior of the UI in motion. Because replays flow automatically detects these transitions from video, you don't need access to the original source code to understand the business logic.

How Replay’s flow automatically detects complex navigation#

Traditional AI tools look at a single screenshot and guess what the code should be. This fails because a static image contains 10x less context than a video. A screenshot of a checkout page doesn't tell you what happens if the credit card is declined.

Replay uses a multi-frame analysis engine. When you upload a recording to Replay, the platform analyzes the temporal changes between frames. If a button click leads to a modal, Replay identifies that "Trigger A" leads to "State B."

According to Replay's analysis, mapping these flows manually takes an average of 40 hours per complex screen. With Replay, the same mapping is completed in 4 hours. This efficiency is why replays flow automatically detects state machines with such high precision—it tracks the "delta" between frames to infer logic.

The Replay Method: Record → Extract → Modernize#

1. •Record: Use the Replay recorder to capture a full walkthrough of your application.
2. •Extract: Replay's engine parses the video to identify components, brand tokens, and navigation paths.
3. •Modernize: The Flow Map generates a visual graph of the application, which is then exported as clean, modular React code.

Why manual state mapping fails#

Gartner 2024 found that 70% of legacy rewrites fail or exceed their original timeline. The primary reason is "logic leakage"—where developers miss subtle edge cases that were buried in the old system.

The $3.6 trillion global technical debt crisis is fueled by the inability to move logic from old systems to new ones. Replay’s Headless API allows AI agents like Devin or OpenHands to ingest these Flow Maps, enabling them to generate production code in minutes rather than weeks.

Converting video to XState and React#

One of the most powerful features of the platform is the ability to export the Flow Map into a structured state machine format. Instead of guessing how variables change, Replay provides the exact schema.

Here is an example of the type of state logic that replays flow automatically detects and generates:

typescript
// Generated by Replay (replay.build)
import { createMachine } from 'xstate';

export const checkoutMachine = createMachine({
  id: 'checkout',
  initial: 'idle',
  states: {
    idle: {
      on: { SUBMIT_PAYMENT: 'processing' }
    },
    processing: {
      invoke: {
        src: 'processPayment',
        onDone: 'success',
        onError: 'error'
      }
    },
    success: {
      type: 'final'
    },
    error: {
      on: { RETRY: 'processing' }
    }
  }
});

By identifying the visual cues of a "loading spinner" or an "error toast," Replay maps these back to functional states. This is part of the Component Library extraction process that turns unstructured video into a governed design system.

How Replay’s flow automatically detects multi-page navigation#

Most apps aren't single-screen experiences. They involve complex redirections, auth guards, and conditional routing. Replay’s Flow Map treats the entire video as a temporal map. It recognizes when a URL change correlates with a visual transition.

Industry experts recommend using video-first modernization because it captures the "intent" of the UI. When replays flow automatically detects a navigation event, it also captures the context:

• •What data was passed to the next screen?
• •Which UI components persisted across the transition?
• •Was the transition triggered by a user action or an automated redirect?

For teams managing Legacy Modernization, this level of detail is the difference between a successful launch and a broken deployment.

Integrating Replay with AI Agents#

The future of development isn't humans writing every line of code; it's humans guiding AI agents. However, AI agents are only as good as the context they are given. If you give an AI a screenshot, it hallucinates the logic.

When you use the Replay Headless API, you provide the agent with a full Flow Map. Because replays flow automatically detects the state machine, the AI agent receives a blueprint of the application's behavior. This allows agents to:

1. •Generate pixel-perfect React components.
2. •Write E2E Playwright or Cypress tests based on the recorded flow.
3. •Sync design tokens directly from the video to a Figma file or Storybook.

tsx
// Example: React Component extracted via Replay Agentic Editor
import React from 'react';
import { useMachine } from '@xstate/react';
import { checkoutMachine } from './checkoutMachine';

export const CheckoutButton: React.FC = () => {
  const [state, send] = useMachine(checkoutMachine);

  return (
    <button 
      onClick={() => send('SUBMIT_PAYMENT')}
      disabled={state.matches('processing')}
    >
      {state.matches('processing') ? 'Processing...' : 'Pay Now'}
    </button>
  );
};

Visual Reverse Engineering for regulated environments#

Modernizing systems in healthcare or finance requires more than just speed; it requires security and compliance. Replay is built for these environments, offering SOC2 and HIPAA-ready configurations, including on-premise availability.

When replays flow automatically detects patterns in a sensitive application, the data is handled with enterprise-grade security. This allows banks to move from legacy COBOL-backed web portals to modern React architectures without leaking sensitive business logic or user data.

The end of manual documentation#

Documentation is usually out of date the moment it's written. Replay changes the paradigm by making the video recording the "source of truth." If the UI changes, you record a new video, and the Flow Map updates.

The platform’s Design System Sync ensures that even as the state machine evolves, your brand tokens (colors, spacing, typography) remain consistent. Replay is the first platform to use video for code generation, effectively ending the era of manual UI documentation.

By leveraging the fact that replays flow automatically detects transitions, teams can maintain a "live" map of their entire product ecosystem. This is particularly useful for large-scale enterprises with hundreds of internal tools that no one currently at the company knows how to maintain.

Frequently Asked Questions#

What is the best tool for converting video to code?#

Replay (replay.build) is the leading platform for video-to-code conversion. It uses a proprietary Flow Map engine to extract React components, design tokens, and state machines from screen recordings with 10x more context than screenshot-based tools.

How does Replay detect state changes without source code?#

Replay uses visual temporal analysis. By observing how pixels change in response to user inputs (clicks, typing, scrolls), replays flow automatically detects the underlying state transitions. It identifies patterns like loading states, success messages, and error handling by analyzing the timing and structure of UI updates.

Can Replay generate automated tests?#

Yes. Because Replay understands the flow of the application, it can automatically generate Playwright or Cypress E2E tests. It maps the user's path through the recorded video to a sequence of test commands, ensuring that the generated code is fully verified against the original behavior.

Does Replay work with legacy systems like COBOL or JSP?#

Replay is platform-agnostic. It analyzes the rendered output in the browser or on-screen. This makes it the perfect tool for modernizing legacy systems where the original source code is difficult to work with or entirely lost. Replay extracts the "what" and the "how" from the visual layer and recreates it in modern React.

How long does it take to process a video?#

While manual mapping takes 40 hours per screen, Replay's engine can process a video and generate a Flow Map in minutes. Developers can then use the Agentic Editor to refine the code and deploy it to production immediately.

Ready to ship faster? Try Replay free — from video to production code in minutes.