iTranslated by AI
How to Use WebMock and VCR
Very few tests were written for code that communicates with external services, and the existing tests were written in a way that accessed the actual URLs every time.
As discussed in detail in the following article, parts of the code that communicate externally are exactly where tests are most needed.
On the other hand, switching values via environment variables or using allow in RSpec to return stubs in order to prevent actual external communication within the production code can lead to decreased code readability.
I used WebMock and VCR to solve this problem, so I'm summarizing my findings here for future reference.
Sample Code
I have created sample code using the ruby-openai gem, which allows you to call the OpenAI API in Ruby.
Since this is just a sample, the code implementation is quite simple.
Note that I am using Ruby version 3.2.2.
Also, assume that a Rails application is running within the same repository.
class Openai
OPENAI_MODEL = "gpt-3.5-turbo"
RESPONSE_TEMPERATURE = 0.1
MAX_TOKEN = 1024
def self.call
access_token = ENV.fetch("OPENAI_ACCESS_TOKEN", nil)
new(access_token).call
end
def initialize(access_token)
@client = OpenAI::Client.new(access_token:, log_errors: true)
end
def call
@client.chat(parameters: {
model: OPENAI_MODEL,
temperature: RESPONSE_TEMPERATURE,
max_tokens: MAX_TOKEN,
messages:[{ role: "user", content: "こんにちは！"}],
})
rescue
"エラー"
end
end
WebMock
WebMock is a gem for stubbing and mocking external HTTP requests instead of actually sending them.
By stubbing responses, you can improve the stability and speed of your tests.
Implementation Example
The actual test code looks like this.
By mocking the request with stub_request, you can prevent communication with the external API during testing.
The content of the response is specified using to_return.
RSpec.describe Openai, type: :services do
describe '.call' do
it "returns valid content" do
stub_request(:post,
"https://api.openai.com/v1/chat/completions"
).to_return(status: 200, body: {"id"=>"chatcmpl-hoge",
"object"=>"chat.completion",
"created"=>1715411192,
"model"=>"gpt-3.5-turbo-0125",
"choices"=>
[{"index"=>0,
"message"=>{"role"=>"assistant", "content"=>"こんにちは！元気ですか？何かお手伝いできることがありますか？"},
"logprobs"=>nil,
"finish_reason"=>"stop"}],
"usage"=>{"prompt_tokens"=>9, "completion_tokens"=>24, "total_tokens"=>33},
"system_fingerprint"=>nil}.to_json , headers: {})
response = Openai.call
expect(JSON.parse(response).dig("choices", 0, "message", "content")).to eq "こんにちは！元気ですか？何かお手伝いできることがありますか？"
end
it "access error" do
stub_request(:post,
"https://api.openai.com/v1/chat/completions"
).to_raise(StandardError)
response = Openai.call
expect(response).to eq "エラー"
end
end
end
You can also use with to specify the content sent as part of the request.
In this case, you can specify not only the parameters to be sent but also things like headers.
Additionally, if the to_return content is extensive, you can write it to a file and load it as shown below:
File.open('/tmp/response_body.txt', 'w') { |f| f.puts 'abc' }
stub_request(:any, "www.example.com").
to_return(body: File.new('/tmp/response_body.txt'), status: 200)
Net::HTTP.get('www.example.com', '/') # ===> "abc\n"
Also, if you need to access localhost in system_spec or similar, add the following setting to spec/rails_helper.rb.
require 'webmock/rspec'
config.before(:suite) do
WebMock.disable_net_connect!(
allow_localhost: true
)
end
If you want to allow access to specific hosts other than localhost, you can add settings individually.
In this way, using WebMock allows you to run tests without communicating with external APIs. However, since you need to describe all the responses corresponding to the test items, the test content can become complex.
VCR
VCR is a gem that saves the results of HTTP requests in test cases in YAML or JSON format. Once recorded, the response is loaded from the saved result during subsequent test executions, allowing tests to run in an accurate state without external communication.
Implementation Example
After installing the gem, add the following configuration to spec_helper.rb.
If you use sensitive information such as an OpenAI API key in your requests, please make sure to configure filter_sensitive_data. Otherwise, the recorded response results will contain sensitive information.
require 'vcr'
require 'webmock/rspec'
VCR.configure do |c|
c.cassette_library_dir = "spec/vcr" # Directory to save recorded cassettes
c.hook_into :webmock # Mock library used internally by VCR
c.configure_rspec_metadata! # Use VCR cassettes in test cases tagged with :vcr
c.filter_sensitive_data("<OPENAI_ACCESS_TOKEN>") { ENV.fetch("OPENAI_ACCESS_TOKEN", nil) }
end
Masked sensitive information
By adding :vcr to the test case where you want to apply VCR, VCR will record the HTTP request.
describe '.call', :vcr do
it "returns valid content" do
response = Openai.call
expect(response.dig("choices", 0, "message", "content")).to eq "こんにちは！元気ですか？何かお手伝いできることがありますか？"
end
it "access error" do
If you run the test in this state, the response will be saved in the spec/vcr directory.
Once executed, subsequent runs will load the saved response to execute the test, allowing the tests to finish quickly. When you want to update the recorded content, you can simply delete the file and record it again.
Summary
In this article, I covered creating tests for parts that involve communication using WebMock and VCR.
To ensure that the content is accurate, I found it effective to use VCR to record the communication as cassettes. However, since it can be difficult to intentionally trigger and record certain errors that might occur, I think it's a good idea to keep in mind using WebMock in combination to handle such scenarios.
Although I described use cases for communicating with external APIs, these tools can also be used for code involving internal APIs or web scraping, making them applicable in various situations.
Articles that might be helpful to refer to
WebMock
- Specifying content with "with" when you want to see the request values themselves in WebMock
- Separating mocks for successful requests and failed responses based on parameters in WebMock
VCR
- Official Documentation
-
VCR Configuration Tips
- Formatting requests/responses for better readability
-
Using VCR to avoid sending an excessive number of requests during development
- Changing cassette update conditions
-
Writing tests using WebMock and VCR with RSpec
- Excluding communications like localhost from VCR
-
Masking sensitive information when dumping requests to external APIs with VCR
- Masking sensitive information
Discussion