Testing Playwright Compatibility with Various Web-based Coding AI Agents

Overview

Recently, various companies have released Web-based coding AI agent environments.
The key feature is that you can use these tools in a web browser or a cloud-based sandbox environment without having to set up a local environment.

Representative examples include the following three:

• Claude Code on Web
  • The web version of Claude Code, which became available on 2025-10-21
• Codex Web
  • The web version of Codex
• Julse
  • Google's coding AI agent

The previous article confirmed that wrangler dev, the Cloudflare Workers development environment, works within the virtual environments of these services.

https://zenn.dev/nozele/articles/847ac75eb0fad3

Simply put, it was possible to start a development server on a virtual environment.

In that case, wouldn't you want to do E2E testing as well?

Since the development server is operational, E2E testing becomes a possibility.
On a local PC, you would typically open a browser and access something like http://localhost:8787. In a web-based virtual environment, you naturally want to try E2E testing in a similar way using a browser—for example, with Playwright. Therefore, I actually tried to see if Playwright works in each environment.

Preparing Playwright Tests and Procedures

So, I added a Playwright test e2e:playwright to the project from the previous article and tried to verify it.
I gave the following instructions to the various services (after creating the tests for Playwright separately):

1. Install dependencies with npm install
2. Install the Chromium browser with npx playwright install chromium
3. Input initialization data with npm run db:migrate, etc.
4. Run the Playwright test with npm run playwright test

Results

Here are the results.

Agent	Starting wrangler dev	Playwright Operation
Claude Code on Web	OK	OK
Codex Web	OK	NG
Jules	OK	OK

Only Codex Web resulted in NG, while Claude Code on Web and Jules were OK.

Codex fails at npx playwright install chromium.
Looking at the error, it seems it cannot access the CDN and failed to download the necessary files.
I have a feeling it might work if chromium-linux.zip or similar were prepared in advance, but that seems difficult in its own way, so the conclusion is basically that it's not possible.

(Reasons why it seems difficult)

• Keeping the downloaded file on GitHub -> It's about 180 MB, so it's hard to handle on GitHub.
• Allowing access to the target URL -> There doesn't seem to be a setting for this in the Plus plan.
• Providing your own download link -> Same as above, the method is unclear.

Final Thoughts

I thought Playwright might not work, but it unexpectedly did.
You can implement and execute E2E tests in parallel on the cloud.

Even Jules, which isn't widely used yet, might have a use case in the field of E2E testing.
Since Jules doesn't consume tokens even when used via the Jules API—instead just using the 60 tasks/day quota—it offers a certain ease of use.

Also, it would be great if it worked on Codex.

---