Zenn
Closed4

[runn] 複数API/リクエストを組み合わせた良い感じのバックエンドテスト設計方法

Test
DevOps
harrythecodeharrythecode

今までの課題

この手の設計で有名なのが「Postman(Newman)」ですが、GUIで全てをやろうとさせすぎて「自由度がなさすぎ」問題に直面しまくってました。

例えば以下のような場合

  • 複数API/リクエストを組み合わせて、例えば「作成→更新→削除」の流れをレスポンス値を使って上手に設計したい
  • シナリオ自体を「コンパクト」にしたい

単一APIを手軽に検証する場合は良いのですが、複数APIを連携させてテストシナリオを組む場合、非常に大きな労力が必要とすることが問題でした。

そこで以下の解決策。

解決策

  • 結論: runn がめちゃ良い

https://zenn.dev/katzumi/articles/api-scenario-testing-with-runn

せっかちな方向け

Github Actionsのテンプレート を用意したのでそちらを見ると何となく使い方が分かるかと思います。

harrythecodeharrythecode

使ってみる

  • インストール: brew install k1LoW/tap/runn
  • 適当にシナリオ作成
example.yml
desc: Echo HelloWorld
runners:
  req: https://postman-echo.com/
vars:
  key: value
steps:
  -
    req:
      /get?foo1={{ vars.key }}:
        get:
          body:
            application/json:
    test: steps[0].res.status == 200
  • デバッグモードで実行:
$ runn run example.yml --debug
Run 'req' on 'Echo HelloWorld'.steps[0]
-----START HTTP REQUEST-----
GET /get?foo1=value HTTP/1.1
Host: postman-echo.com
Content-Type: application/json


-----END HTTP REQUEST-----
-----START HTTP RESPONSE-----
HTTP/2.0 200 OK
Content-Length: 318
...

{"args":{"foo1":"value"},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-632aa21a-2f66acd6050acb4443d4537f","content-type":"application/json","accept-encoding":"gzip","user-agent":"Go-http-client/2.0"},"url":"https://postman-echo.com/get?foo1=value"}
-----END HTTP RESPONSE-----
Run 'test' on 'Echo HelloWorld'.steps[0]
-----START TEST CONDITION-----
steps[0].res.status == 200
├── steps[0].res.status => 200
└── 200 => 200
-----END TEST CONDITION-----
Echo HelloWorld ... ok

1 scenario, 0 skipped, 0 failures

どんな値が返ってきてるのかはtest部分を以下のように変更。

  • test: steps != null
$ runn run example.yml --debug
...
Run 'test' on 'Echo HelloWorld'.steps[0]
-----START TEST CONDITION-----
steps != null
├── steps => [{"res":{"body":{"args":{"foo1":"value"},"headers":{"accept-encoding":"gzip","content-type":"application/json","host":"postman-echo.com","user-agent":"Go-http-client/2.0","x-amzn-trace-id":"Root=1-632aa502-09b8f95026b92efd3b5e1935","x-forwarded-port":"443","x-forwarded-proto":"https"},"url":"https://postman-echo.com/get?foo1=value"},"headers":{"Content-Length":["318"],"Content-Type":["application/json; charset=utf-8"],"Date":["Wed, 21 Sep 2022 05:45:38 GMT"],"Etag":["W/\"13e-E6dyeWR7YLqi6CQIuVbReAD9RgQ\""],"Set-Cookie":["sails.sid=s%3A9Wy39EBaKj3bkjAVDM9-kSQDHYb2p2z_.p0H31WUEQXW5liToVfUfli5b%2BTVAD3lGxAtoPSTX3jg; Path=/; HttpOnly"],"Vary":["Accept-Encoding"]},"rawBody":"{\"args\":{\"foo1\":\"value\"},\"headers\":{\"x-forwarded-proto\":\"https\",\"x-forwarded-port\":\"443\",\"host\":\"postman-echo.com\",\"x-amzn-trace-id\":\"Root=1-632aa502-09b8f95026b92efd3b5e1935\",\"content-type\":\"application/json\",\"accept-encoding\":\"gzip\",\"user-agent\":\"Go-http-client/2.0\"},\"url\":\"https://postman-echo.com/get?foo1=value\"}","status":200}}]
└── null => null
-----END TEST CONDITION-----
harrythecodeharrythecode

使い方テクニック集

crudPostman.yml
desc: Http Request Examples on Postman
runners:
  req: https://postman-echo.com/
vars:
  foo1: bar1
  foo2: bar2
  base64Auth: "cG9zdG1hbjpwYXNzd29yZA==" # echo -n "postman:password"|base64
steps:
  read:
    req:
      /get?foo1={{ vars.foo1 }}:
        get:
          body: null
    test: |
      steps.read.res.status == 200 &&
      steps.read.res.body.args.foo1 == vars.foo1

  update:
    req:
      /put:
        put:
          body:
            application/json:
              {
                "foo1": "{{ vars.foo1 }}",
                "foo2": "{{ vars.foo2 }}",
              }
    test: steps.update.res.status == 200
    test: steps.update.res.body.data.foo1 == vars.foo1

  patch:
    req:
      /patch:
        patch:
          body:
            application/json:
              {
                "foo1": "{{ steps.update.res.body.url }}",
                "foo2": "{{ vars.foo2 }}",
              }
    test: |
      steps.patch.res.status == 200 && 
      steps.patch.res.body.data.foo1 == steps.update.res.body.url

  basicAuth:
    req:
      /basic-auth:
        get:
          headers:
            "Authorization": "Basic {{vars.base64Auth}}"
          body: null
    test: |
      steps.basicAuth.res.status == 200 && 
      steps.basicAuth.res.body.authenticated == true
  • 実行結果
$ runn run crudPostman.yml --debug
Run 'req' on 'Http Request Examples on Postman'.steps.read
-----START HTTP REQUEST-----
GET /get?foo1=bar1 HTTP/1.1
Host: postman-echo.com


-----END HTTP REQUEST-----
-----START HTTP RESPONSE-----
HTTP/2.0 200 OK
Content-Length: 282
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Sep 2022 08:35:23 GMT
Etag: W/"11a-OAq82ELVnk04SdTiLtXGa9gqsg0"
Set-Cookie: sails.sid=s%3AoJOwBncXUT3DyJyka35COv0U9Sq3VO9I.6UAOxnALzzeOmvAbrMna2ufl3ymaZLobDtL5x5k7PxU; Path=/; HttpOnly
Vary: Accept-Encoding

{"args":{"foo1":"bar1"},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-632acccb-0e8c7f254eaa4e4628242c84","accept-encoding":"gzip","user-agent":"Go-http-client/2.0"},"url":"https://postman-echo.com/get?foo1=bar1"}
-----END HTTP RESPONSE-----
Run 'test' on 'Http Request Examples on Postman'.steps.read
-----START TEST CONDITION-----
steps.read.res.status == 200 &&
steps.read.res.body.args.foo1 == vars.foo1


├── steps.read.res.status => 200
├── 200 => 200
├── steps.read.res.body.args.foo1 => "bar1"
└── vars.foo1 => "bar1"
-----END TEST CONDITION-----

Run 'req' on 'Http Request Examples on Postman'.steps.update
-----START HTTP REQUEST-----
PUT /put HTTP/1.1
Host: postman-echo.com
Content-Type: application/json

{"foo1":"bar1","foo2":"bar2"}
-----END HTTP REQUEST-----
-----START HTTP RESPONSE-----
HTTP/2.0 200 OK
Content-Length: 410
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Sep 2022 08:35:23 GMT
Etag: W/"19a-DQ3n1VofQehnWNrvUAwpEGlLjXo"
Set-Cookie: sails.sid=s%3AR727vJfge831BfzW6MRmkulHYrHKK7u5.O2dp%2FuEinuq9OG2rH9JuOFUFNJHcfHkXQHrOrxGswqI; Path=/; HttpOnly
Vary: Accept-Encoding

{"args":{},"data":{"foo1":"bar1","foo2":"bar2"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-632acccb-124c21924b21a0144e5a7b9d","content-length":"29","content-type":"application/json","accept-encoding":"gzip","user-agent":"Go-http-client/2.0"},"json":{"foo1":"bar1","foo2":"bar2"},"url":"https://postman-echo.com/put"}
-----END HTTP RESPONSE-----
Run 'test' on 'Http Request Examples on Postman'.steps.update
-----START TEST CONDITION-----
steps.update.res.body.data.foo1 == vars.foo1
├── steps.update.res.body.data.foo1 => "bar1"
└── vars.foo1 => "bar1"
-----END TEST CONDITION-----

Run 'req' on 'Http Request Examples on Postman'.steps.patch
-----START HTTP REQUEST-----
PATCH /patch HTTP/1.1
Host: postman-echo.com
Content-Type: application/json

{"foo1":"https://postman-echo.com/put","foo2":"bar2"}
-----END HTTP REQUEST-----
-----START HTTP RESPONSE-----
HTTP/2.0 200 OK
Content-Length: 460
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Sep 2022 08:35:24 GMT
Etag: W/"1cc-76AMb1ReR2/pqZAdLjohK30/k+Y"
Set-Cookie: sails.sid=s%3ASLhMykF9MkTy7GVbaxSkIeAmVuhv0ZYq.bBbgpZ4m%2BMpgYzMcKI3sc9M7OLZOJ5%2BGKMDhiROvC0w; Path=/; HttpOnly
Vary: Accept-Encoding

{"args":{},"data":{"foo1":"https://postman-echo.com/put","foo2":"bar2"},"files":{},"form":{},"headers":{"x-forwarded-proto":"https","x-forwarded-port":"443","host":"postman-echo.com","x-amzn-trace-id":"Root=1-632acccc-4bccc7000da4ab7360c3321f","content-length":"53","content-type":"application/json","accept-encoding":"gzip","user-agent":"Go-http-client/2.0"},"json":{"foo1":"https://postman-echo.com/put","foo2":"bar2"},"url":"https://postman-echo.com/patch"}
-----END HTTP RESPONSE-----
Run 'test' on 'Http Request Examples on Postman'.steps.patch
-----START TEST CONDITION-----
steps.patch.res.status == 200 &&
steps.patch.res.body.data.foo1 == steps.update.res.body.url


├── steps.patch.res.status => 200
├── 200 => 200
├── steps.patch.res.body.data.foo1 => "https://postman-echo.com/put"
└── steps.update.res.body.url => "https://postman-echo.com/put"
-----END TEST CONDITION-----

Run 'req' on 'Http Request Examples on Postman'.steps.basicAuth
-----START HTTP REQUEST-----
GET /basic-auth HTTP/1.1
Host: postman-echo.com
Authorization: Basic cG9zdG1hbjpwYXNzd29yZA==


-----END HTTP REQUEST-----
-----START HTTP RESPONSE-----
HTTP/2.0 200 OK
Content-Length: 22
Content-Type: application/json; charset=utf-8
Date: Wed, 21 Sep 2022 08:35:24 GMT
Etag: W/"16-sJz8uwjdDv0wvm7//BYdNw8vMbU"
Set-Cookie: sails.sid=s%3A1nM53zEvDvlb-vX-Y0GfkO70UhdRLugT.NoccZPXGqFEi46QkLGP%2F1tEiUAHWBdkTAwLb1CPAqwE; Path=/; HttpOnly
Vary: Accept-Encoding

{"authenticated":true}
-----END HTTP RESPONSE-----
Run 'test' on 'Http Request Examples on Postman'.steps.basicAuth
-----START TEST CONDITION-----
steps.basicAuth.res.status == 200 &&
steps.basicAuth.res.body.authenticated == true


├── steps.basicAuth.res.status => 200
├── 200 => 200
├── steps.basicAuth.res.body.authenticated => true
└── true => true
-----END TEST CONDITION-----
Http Request Examples on Postman ... ok

1 scenario, 0 skipped, 0 failures
このスクラップは2022/09/21にクローズされました