iTranslated by AI

The content below is an AI-generated translation. This is an experimental feature, and may contain errors. View original article
💬

Checking Playwright Compatibility with Web-based Coding AI Agents

に公開

Overview

Recently, various companies have launched web-based coding AI agent environments.
The key feature is that you can use these tools without setting up a local environment, directly from a web browser or a cloud sandbox environment.

The representative three are:

  • Claude Code on Web
    • Web version of Claude Code, made available on 2025-10-21
  • Codex Web
    • Web version of Codex
  • Julse
    • Google's coding AI agent

The previous article confirmed that wrangler dev, the development environment for Cloudflare Workers, works in each company's virtual environment.

https://zenn.dev/nozele/articles/847ac75eb0fad3

Simply put, it was possible to start a development server in the virtual environment.

So, Doesn't That Make You Want to Do E2E Testing?

Since the development server runs, E2E testing becomes possible.
On a local PC, you would access http://localhost:8787 by opening a browser, but with web-based virtual environments, you might want to try E2E testing in the browser as well, for example using Playwright.

So, I actually tried to see if Playwright works in each environment.

Preparing Playwright Tests and Steps

So, I added a test using Playwright called e2e:playwright to the project from the previous article and verified it.
I instructed each service to follow these steps (after separately creating the Playwright test):

  1. Install dependencies with npm install
  2. Install Chromium browser with npx playwright install chromium
  3. Input initial data with npm run db:migrate etc.
  4. Run the Playwright test with npm run playwright test

Results

The results are as follows.

Agent wrangler dev startup Playwright operation
Claude Code on Web OK OK
Codex Web OK NG
Jules OK OK

Only Codex Web resulted in NG, while Claude Code on Web and Jules showed OK.

Codex fails at npx playwright install chromium.
Looking at the error, it seems unable to access the CDN and cannot download the necessary files.
I have a feeling that if we prepare files like chromium-linux.zip in advance, it might work, but that seems difficult in itself, so the conclusion is that it's basically not possible.

(Reasons it seems difficult)

  • Putting the downloaded file in GitHub -> It's about 180 MB, so it's hard to handle on GitHub.
  • Making the target URL accessible -> It seems there's no setting for that in the Plus plan.
  • Preparing my own download link -> Same as above, method unknown

Impressions

I thought Playwright wouldn't work, but surprisingly it did.
You can implement and run E2E tests in parallel on the cloud.

Even Jules, which isn't very popular, might have its uses in the area of E2E testing.
Also, since Jules uses only the 60 tasks/day limit without consuming tokens even when used via the Jules API, it has that kind of casualness.

Also, I'd be happy if it works on Codex.


Discussion