Implement UI A/B Testing Effortlessly with the react-ab-test Library

We're going to do an A/B test!
While you might start with a lot of enthusiasm, have you ever found yourself stuck searching for specialized tools, struggling to get consensus for introduction, waiting for payment approval, and then finally getting it—only to realize the tool is incredibly difficult to use?

Or perhaps you are wasting time with long verification periods, testing Pattern A first, then Pattern B later, before finally getting to the actual effect verification?

Actually, there is a library that makes A/B testing a reality!

A/B Testing Library @marvelapp/react-ab-test

The library I'm introducing this time is this one!
https://github.com/marvelapp/react-ab-test

It was created as a fork of the following:
https://github.com/pushtell/react-ab-test

The original only supports older versions of React, so using "@marvelapp/react-ab-test" is recommended!

How to Use

The usage is documented thoroughly on GitHub, so it might not be necessary to explain it in detail here, but I will outline the flow for reference.

Installation

yarn add @marvelapp/react-ab-test

Defining UIs to be Switched

You simply wrap the UI patterns you want to test with the <Experiment> component! Set the name of the test in the name prop.

Wrap each UI pattern in a <Variant> component. Set the name of the UI pattern in its name prop.

index.jsx

import React from 'react'
import { Experiment, Variant } from '@marvelapp/react-ab-test'

export default function Ab(){
  return (
    <div>
      <Experiment name="My Example">
        <Variant name='A'>
          <h1>Type A</h1>
        </Variant>
        <Variant name='B'>
          <h1>Type B</h1>
        </Variant>
      </Experiment>
    </div>
  )
}

That's all it takes to switch between UIs!

Notes on Using with Next.js

I tried testing this with Next.js and encountered the following error.

Next content does not match server-rendered HTML

Because I was using SSR, it seems this occurred because of a discrepancy between what was called on the server-side and what was actually rendered on the client-side.

As a workaround, since the cause was the UI switching triggered by the A/B test library, I resolved it by adding suppressHydrationWarning={true} to the DOM elements being switched.
https://weev.media/article/717

index.jsx

// ...omitted

export default function Ab(){
  return (
    <div>
      <Experiment name="My Example">
        <Variant name='A'>
          <h1 suppressHydrationWarning={true}>Type A</h1>
        </Variant>
        <Variant name='B'>
          <h1 suppressHydrationWarning={true}>Type B</h1>
        </Variant>
      </Experiment>
    </div>
  )
}

Displayed UI Information is Saved in Local Storage

Once either Pattern A or Pattern B is called, that information is saved in local storage. From then on, only the saved UI pattern will be displayed.

Wrap components in <Variant /> and nest in <Experiment />. A variant is chosen randomly and saved to local storage.

Adjusting the Display Ratio

If you want to adjust the display ratio of the UIs, use emitter.defineVariants.

As shown below, set the test name as the first argument, the UI pattern names as the second argument, and the display ratios as the third argument.
(I haven't strictly verified if it follows the ratio exactly, but when I set it to 8:2, it definitely felt like the display probability increased. lol)

index.jsx

import { Experiment, Variant, emitter } from '@marvelapp/react-ab-test'

emitter.defineVariants("My Example", ["A", "B"], [50, 50]);

// ...omitted

Accumulating Measurement Logs

Understandably, this library does not include measurement features. You will need to develop a separate log API or use GA event settings to perform measurements.

That said, it makes switching UIs incredibly easy, which is quite helpful.

It's surprising how many different libraries are out there.