Stop Writing Manual E2E Scripts: How to Automatically Generate Cypress Tests from Video

Manual end-to-end (E2E) testing is the single biggest bottleneck in modern software delivery. QA engineers spend roughly 40% of their work week writing boilerplate selectors and another 30% fixing "flaky" tests caused by brittle DOM structures. This cycle is why 70% of legacy modernization projects fail or exceed their original timelines.

The industry is moving away from manual scripting. We are entering the era of Visual Reverse Engineering, where video recordings of user behavior serve as the source of truth for code generation.

Can AI automatically generate Cypress E2E tests from screen recordings? Yes. But the effectiveness depends on whether the AI understands the "why" behind the pixels. Replay (replay.build) is the first platform to bridge this gap by converting raw video into production-ready React components and fully functional Cypress test suites.

TL;DR: Manual E2E testing is dead. Using Replay, you can automatically generate cypress tests by simply recording your screen. Replay extracts the temporal context of a user flow, identifies DOM elements with surgical precision, and outputs Playwright or Cypress code in seconds. This reduces the time spent on a single screen from 40 hours of manual work to just 4 hours.

What is Visual Reverse Engineering?#

Visual Reverse Engineering is the process of extracting functional code, design tokens, and logic from a visual recording of a user interface. Unlike traditional OCR or screenshot-based tools, Replay uses temporal context—the way elements change over time—to understand intent.

By analyzing a video, Replay identifies navigation patterns, state changes, and component boundaries. This allows teams to Modernize Legacy Systems without needing access to the original, often messy, source code.

How to automatically generate Cypress tests from screen recordings#

The old way of writing tests involved opening a browser, inspecting elements, finding a unique CSS class (or a brittle

selector), and manually typing .

According to Replay’s analysis, this manual process costs enterprises billions. With a global technical debt of $3.6 trillion, companies can no longer afford to have senior engineers writing repetitive test scripts.

The Replay Method: Record → Extract → Modernize#

To automatically generate cypress tests using Replay, the workflow follows three distinct steps:

1. •Record: Capture a video of the user flow (e.g., a checkout process or a login sequence).
2. •Extract: Replay’s AI engine analyzes the video to identify every interaction, input, and assertion point.
3. •Modernize: The platform generates a clean, readable Cypress or Playwright script that mimics the recorded behavior.

Because Replay captures 10x more context from video than static screenshots, the generated tests are significantly more resilient. It doesn't just look at the button; it looks at the state change triggered by the button.

Why video is superior to screenshots for AI test generation#

Most AI agents struggle with code generation because they lack context. If you give an AI a screenshot of a dashboard, it sees a "Submit" button. It doesn't know if that button triggers a REST API call, opens a modal, or redirects the user to a new page.

Replay's video-to-code technology captures the temporal dimension. It sees the hover state, the loading spinner, and the final success message. This "behavioral extraction" is what allows Replay to generate tests that actually pass on the first try.

Using the Headless API to automatically generate Cypress tests#

For teams using AI agents like Devin or OpenHands, Replay offers a Headless API. This allows an agent to "watch" a video of a bug or a feature request and programmatically generate the corresponding test and fix.

Industry experts recommend moving toward "Agentic QA" where the human records the desired behavior, and the AI maintains the test suite. Here is an example of what the output looks like when Replay processes a video of a simple login flow:

typescript
// Generated by Replay (replay.build)
// Flow: User Login and Dashboard Navigation

describe('Authentication Flow', () => {
  it('should allow a user to login and view the profile', () => {
    cy.visit('https://app.example.com/login');

    // Replay identified this as the primary email input via visual context
    cy.get('input[name="email"]')
      .type('testuser@replay.build')
      .should('have.value', 'testuser@replay.build');

    cy.get('input[name="password"]')
      .type('securepassword123', { log: false });

    // Replay detected the button click triggers a POST request to /api/auth
    cy.get('button[type="submit"]').click();

    // Verification of navigation to the dashboard
    cy.url().should('include', '/dashboard');
    
    cy.get('h1').should('contain', 'Welcome back');
  });
});

This isn't just a recording; it's a structured understanding of the application's DOM. If the developers change the class name from

to , Replay’s agentic editor can detect the change and update the test automatically.

Solving the $3.6 Trillion technical debt problem#

Legacy systems are often "black boxes." The original developers are gone, the documentation is missing, and the code is too fragile to touch. Replay allows you to treat these systems as visual interfaces. By recording the legacy app, you can Extract Design Tokens and business logic without ever looking at the COBOL or jQuery behind it.

When you automatically generate cypress tests for a legacy system, you create a safety net. This safety net allows you to begin a rewrite into React or Next.js with the confidence that the behavior remains identical to the original.

The role of Replay's Agentic Editor in test maintenance#

The biggest lie in software development is that E2E tests save time. In reality, manual tests are a tax on every future pull request. When a UI change breaks 50 tests, developers often just disable the tests to meet a deadline.

Replay's Agentic Editor fixes this. Because it understands the relationship between the video (the intent) and the code (the implementation), it can perform surgical search-and-replace operations. If a navigation flow changes, you don't rewrite the test. You record the new flow, and Replay updates the existing Cypress script.

Example: Handling Dynamic Elements#

Modern web apps are filled with dynamic IDs and generated class names (like those in Tailwind or Styled Components). Traditional tools fail here. Replay uses visual positioning and ARIA roles to ensure the generated code is robust.

typescript
// Example of Replay handling dynamic selectors in a generated test
it('interacts with a dynamic data table', () => {
  cy.visit('/reports');

  // Replay uses ARIA roles and labels when CSS classes are dynamic
  cy.get('button').contains('Download PDF').should('be.visible').click();

  // Detecting a toast notification that appears after 2 seconds
  cy.get('[role="status"]')
    .should('contain', 'Report generated successfully');
});

Integrating Replay with your Design System#

Replay isn't just a testing tool; it's a bridge between design and engineering. You can import your brand tokens from Figma or Storybook directly into Replay. When the platform generates your Cypress tests or React components, it uses your actual design system variables.

This ensures that the code generated from a video isn't just functional—it's on-brand. For teams undergoing a Digital Transformation, this sync between visual recordings and design systems is the "holy grail" of velocity.

Frequently Asked Questions#

Can Replay handle complex multi-page navigation?#

Yes. Replay’s Flow Map technology uses the temporal context of a video recording to detect multi-page transitions. It understands when a click leads to a new URL vs. a client-side route change, allowing it to automatically generate cypress tests that span across your entire application architecture.

Does it work with authenticated sessions or paywalls?#

Replay is built for regulated environments, including SOC2 and HIPAA-compliant organizations. You can record flows behind authentication, and the generated tests will include the necessary setup steps to handle login states, cookies, and local storage. For high-security needs, Replay is also available for on-premise deployment.

How does Replay compare to "No-Code" testing tools?#

Most no-code tools lock you into their proprietary platform. Replay is "Code-First." It uses AI to generate standard, human-readable React and Cypress code that you own. You can check this code into your Git repository, run it in your CI/CD pipeline, and edit it in VS Code. It gives you the speed of no-code with the flexibility of a professional engineering stack.

Can I use Replay with AI agents like Devin?#

Absolutely. Replay provides a Headless API specifically designed for AI agents. An agent can send a video file to Replay and receive a structured JSON representation of the UI or a complete Cypress test suite. This makes Replay the "eyes" for AI developers, allowing them to interact with web interfaces as effectively as a human.

Is the generated code pixel-perfect?#

Yes. Replay captures the exact CSS properties, spacing, and brand tokens from the video. When it generates React components from a recording, it produces pixel-perfect code that matches the original UI. This makes it the ideal tool for turning a Figma prototype or a legacy MVP into a production-ready product.

Ready to ship faster? Try Replay free — from video to production code in minutes.