👀

Storybook test runner でビジュアルリグレッションテストをする

2024/09/09に公開

Storybook

Test

ビジュアルリグレッションテスト

tech

設定

Storybook test runner は内部で Jest と Playwright を使っている。 Playwright 動作させる OS によってレンダリング結果が異なるので、 Dokcer を使って手元でも CI でも同様のレンダリング結果となるようにする。

Playwright Docker コンテナーを Storybook test runner で使うための設定は次の記事が詳しく書いてくれている。

これを参考に設定していく。

/.storybook/test-runner.ts

import {
  getStoryContext,
  TestRunnerConfig,
  waitForPageReady,
} from '@storybook/test-runner';
import { toMatchImageSnapshot } from 'jest-image-snapshot';

const customSnapshotsDir = `${process.cwd()}/__snapshots__`;

const config: TestRunnerConfig = {
  setup() {
    expect.extend({ toMatchImageSnapshot });
  },


  async prepare({ page, browserContext, testRunnerConfig }) {
    const targetURL = 'http://host.docker.internal:6006';
    const iframeURL = new URL('iframe.html', targetURL).toString();

    if (testRunnerConfig?.getHttpHeaders) {
      const headers = await testRunnerConfig.getHttpHeaders(iframeURL);
      await browserContext.setExtraHTTPHeaders(headers);
    }

    await page.goto(iframeURL, { waitUntil: 'load' }).catch((err) => {
      if (err.message?.includes('ERR_CONNECTION_REFUSED')) {
        const errorMessage = `Could not access the Storybook instance at ${targetURL}. Are you sure it's running?\n\n${err.message}`;
        throw new Error(errorMessage);
      }

      throw err;
    });
  },

  async postVisit(page, context) {
    await waitForPageReady(page);

    const story = await getStoryContext(page, context);
    if (story.parameters.skipVisualRegressionTesting) {
      return;
    }

    const image = await page.screenshot({
      animations: 'disabled',
    });

    expect(image).toMatchImageSnapshot({
      customSnapshotsDir,
      customSnapshotIdentifier: context.id,
      diffDirection: 'vertical',
    });
  },
};
export default config;

/test-runner-jest.config.js

import { getJestConfig } from "@storybook/test-runner";

/**
 * @type {import('@jest/types').Config.InitialOptions}
 */
export default {
  ...getJestConfig(),
  testEnvironmentOptions: {
    "jest-playwright": {
      connectOptions: {
        chromium: {
          wsEndpoint: "ws://127.0.0.1:3000",
        },
      },
    },
  },
};

テストスクリプト

コマンドひとつでは無理なので、次のようなシェルスクリプトを書く。

/bin/vrt.sh

#!/bin/sh

set -ex

PORT=6006

if [ ! -d storybook-static ]; then
  npm run build:storybook
fi

# refer: https://storybook.js.org/docs/writing-tests/test-runner
TEST_COMMAND="npx --yes wait-on tcp:${PORT} tcp:3000 && npm run test-storybook --url http://localhost:${PORT}"
# スナップショットをアップデートしたいとき
if [ "$1" = "update" ]; then
  TEST_COMMAND="${TEST_COMMAND} --updateSnapshot"
fi

PLAYWRIGHT_VERSION=$(jq -r .devDependencies.playwright < package.json)

# refer: https://github.com/open-cli-tools/concurrently
npx concurrently \
    --kill-others \
    --success first \
    --names "SB,PW,TEST" \
    --prefix-colors "magenta,green,blue" \
    "npx --yes http-server storybook-static --port ${PORT} --silent" \
    "PLAYWRIGHT_VERSION=${PLAYWRIGHT_VERSION} docker compose up" \
    "${TEST_COMMAND}"

で、 npm scripts として次を定義する。

package.json

  // snip
  "scripts": {
    "build:storybook": "storybook build",
    "test:storybook": "bin/vrt.sh",
    "test:storybook:update": "bin/vrt.sh update"
  },
  // snip

これで npm run test:storybook と打つと VRT が走るようになる。 /__snapshots__ に画像が生成されるので git commit する。

CI

GitHub Actions の設定ファイルは次。

/.github/workflows/vrt.yaml

name: Visual Regression Testing

on:
  push:
    branches:
      - main
  pull_request:
    paths:
      - "/src/**"
      - "package.json"
      - "package-lock.json"
      - ".github/workflows/vrt.yml"

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

env:
  DOCKER_IAMGE_CACHE_PATH: /tmp/docker-image-cache
  PLAYWRIGHT_VERSION_PREFIX: v
  PLAYWRIGHT_VERSION_POSTFIX: -noble

jobs:
  visual-regression-testing:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Cache .turbo
        uses: actions/cache@v4
        with:
          path: .turbo
          key: ${{ runner.os }}-turbo-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-turbo-

      - name: Setup Node
        uses: actions/setup-node@v4
        with:
          node-version-file: .node-version
          cache: "npm"
          cache-dependency-path: package-lock.lock

      - name: Get Playwright version
        id: get_playwright_version
        run: |
          echo "PLAYWRIGHT_VERSION=${PLAYWRIGHT_VERSION_PREFIX}$(cat /package.json | jq --raw-output .devDependencies.playwright)${PLAYWRIGHT_VERSION_POSTFIX}" >> $GITHUB_ENV

      - name: Cache Docker images
        id: cache-docker-images
        uses: actions/cache@v4
        with:
          path: ${{ env.DOCKER_IAMGE_CACHE_PATH }}
          key: ${{ runner.os }}-playwright-${{ env.PLAYWRIGHT_VERSION }}

      - name: Pull and save Playwright Docker image
        if: steps.cache-docker-images.outputs.cache-hit != 'true'
        run: |
          docker pull mcr.microsoft.com/playwright:${PLAYWRIGHT_VERSION}
          docker save mcr.microsoft.com/playwright:${PLAYWRIGHT_VERSION} -o ${DOCKER_IAMGE_CACHE_PATH}

      - name: Load Docker image from cache
        run: docker load -i ${DOCKER_IAMGE_CACHE_PATH}

      - name: Install dependencies
        run: npm ci

      - name: Cache Storybook build
        id: cache-storybook-build
        uses: actions/cache@v4
        with:
          path: storybook-static
          key: ${{ runner.os }}-storybook-static-${{ github.sha }}

      - name: Build Storybook
        if: steps.cache-storybook-build.outputs.cache-hit != 'true'
        run: npm run build:storybook

      - name: Serve Storybook and run tests
        run: npm run test:vrt

      - name: Upload test results
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          path: /__snapshots__

これで意図しない変更があった場合、 CI が検出してくれるようになる。

GitHubで編集を提案

設定

テストスクリプト

CI

Discussion