A Rack middleware to remove null bytes

This post is a part of YAMAP Engineers Advent Calendar 2023.
https://qiita.com/advent-calendar/2023/yamap-engineers

Introduction

When a Rails controller in your application receives parameters containing null bytes (\u0000 character) and these data are utilized in an SQL statement, it may result in raising an ArgumentError.

ArgumentError: string contains null byte (ArgumentError)
              @connection.exec_params(sql, type_casted_binds)

Creating a Rack middleware

One effective solution is to eliminate all null bytes from the received parameters. This can be achieved by incorporating a Rack middleware to handle the cleansing process.

The middleware functions as a pure Ruby class that takes in the request, cleans up the parameters, and then forwards the request to the next middleware for further processing.

# app/middleware/remove_null_bytes.rb

class RemoveNullBytes
  # NOTE: Limit depth level to prevent performance issues
  DEPTH_LIMIT = 3

  def initialize(app)
    @app = app
  end

  def call(env)
    request = Rack::Request.new(env)
    request.params.each_value do |value|
      remove_null_bytes_recursively(value)
    end
    @app.call(request.env)
  end

  private

  def remove_null_bytes_recursively(value, depth = 0)
    return if depth > DEPTH_LIMIT

    depth += 1
    case value
    when Hash
      value.each_value do |v|
        remove_null_bytes_recursively(v, depth)
      end
    when Array
      value.each do |v|
        remove_null_bytes_recursively(v, depth)
      end
    when String
      value.delete!("\u0000")
    end
  end
end

Afterward, we integrate the middleware into the stack. Placing it before Rack::Head is usually sufficient unless you have additional custom middleware. In such cases, careful consideration of the optimal order is necessary. Utilize the rake middleware command to obtain the current ordered list of middleware in use.

# config/application.rb
...
module MyApp
  class Application < Rails::Application
    ...
    require './app/middleware/remove_null_bytes'
    config.middleware.insert_before Rack::Head, RemoveNullBytes
  end
end

Testing

To verify the effectiveness of the middleware, we can do a request test. Essentially, we create a dummy controller that echoes back every parameter it receives in the response. Subsequently, we run tests to ensure that the response data does not contain any null bytes.

# spec/middleware/remove_null_bytes_spec.rb
require 'rails_helper'

RSpec.describe RemoveNullBytes, type: :request do
  before do
    mock_controller = Class.new(ApplicationController) do
      def create
        render json: params.except(:controller, :action)
      end
    end
    stub_const('MocksController', mock_controller)

    # NOTE: This is needed to prevent Rails from clearing routes
    #       when calling Rails.application.routes.draw
    Rails.application.routes.disable_clear_and_finalize = true
    Rails.application.routes.draw do
      resource :mock, only: :create
    end
  end

  after do
    # NOTE: reset to the original setting
    Rails.application.reload_routes!
  end

  it 'removes null bytes from String params' do
    post '/mock', params: { name: "test\u0000" }
    expect(response.body).to eq({ name: 'test' }.to_json)
  end

  it 'removes null bytes from Array params' do
    post '/mock', params: { names: ["test\u0000"] }
    expect(response.body).to eq({ names: ['test'] }.to_json)
  end

  it 'removes null bytes from nested params' do
    post '/mock', params: {
      user: {
        name: "test\u0000",
        phones: ["080\u000077778888"],
        followers: [
          { name: "followers\u0000" },
        ],
      },
    }
    expect(response.body)
      .to eq({
        user: {
          name: 'test',
          phones: ['08077778888'],
          followers: [
            { name: 'followers' },
          ],
        },
      }.to_json)
  end

  it 'does not modify params deeply nested more than 3 levels' do
    post '/mock', params: {
      level1: {
        name: "test\u0000",
        level2: {
          name: "test\u0000",
          level3: {
            name: "test\u0000",
            level4: {
              name: "test\u0000",
            },
          },
        },
      },
    }

    expect(response.body)
      .to eq({
        level1: {
          name: 'test',
          level2: {
            name: 'test',
            level3: {
              name: 'test',
              level4: {
                name: "test\u0000",
              },
            },
          },
        },
      }.to_json)
  end
end

Conclusion

This approach has proven effective in my experience. If you encounter any issues or have alternative solutions, please feel free to provide feedback in the comments.