[React] Getting Connected Devices

Overview

When implementing WebRTC, device manipulation is unavoidable.
While libraries and components such as Amazon Chime and the Zoom API wrap device operations for you, when building it yourself, you use APIs that aren't typically used in everyday development.

In this article, I would like to summarize these points, partly as my own study notes.

Getting Devices

Getting Devices with enumerateDevices

In React, most of this can be achieved by accessing the MediaDevices API, which starts with navigator.mediaDevices.
To obtain a list of devices, use enumerateDevices().
https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/enumerateDevices

The response is Promise<MediaDeviceInfo[]>, and MediaDeviceInfo has the following structure:

{
  deviceId: 'xxxxx',
  groupId: 'yyyyyy',
  kind: 'audiooutput',
  label: 'HogeHoge'
}

The value displayed on the UI is the label, but deviceId is used internally when switching devices.
Also, because of the kind property, microphones, speakers, and cameras are returned together as an array.
Therefore, depending on the use case, it is common to apply a filter based on the kind when displaying them.

Example: useSpeakerDeviceList

For example, I create and use a custom hook like the one below in my logic.

import {useState, useEffect, useCallback} from 'react';

// Custom hook to get a list of speaker devices
const useSpeakerDeviceList = () => {
  const [devices, setDevices] = useState<MediaDeviceInfo[]>([]);
  // Device update process
  const refreshDevices = useCallback(async () => {
    const latestDevices = (
      await navigator.mediaDevices.enumerateDevices()
    ).filter((d) => d.kind === "audiooutput");
    setDevices(latestDevices);
  }, []);

  useEffect(() => {
    // Get the latest device list on initial render (=componentDidMount)
    refreshDevices();
    // Perform update process whenever devices are attached or detached thereafter
    navigator.mediaDevices.addEventListener("devicechange", refreshDevices);
    return () => {
      navigator.mediaDevices.removeEventListener(
        "devicechange",
        refreshDevices
      );
    };
  }, []);

  return devices;
};

The key point is that an event listener is set on MediaDevices to detect the attachment or detachment of devices, ensuring that the latest array is always returned.
For other devices (cameras/microphones), you just need to change the condition when applying the filter.

Using Devices

Now that we have obtained the list of devices, I will describe how to use them.
From this point, the implementation methods start to differ depending on the type of device.

Camera (Microphone)

Since the camera output is a <video> tag, the basic approach is to manipulate its ref.
The MediaDevices API has a method called getUserMedia, which is used to configure cameras and microphones.

https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia

Note: Regarding microphones, since this specific implementation is only necessary when you want to display audio levels in the UI, I will omit it here.

SampleVideo.tsx

import { useEffect, useRef } from "react";

const SampleVideo = (selectedVideo: MediaDeviceInfo) => {
  // Obtain the ref for <video>
  const videoRef = useRef<HTMLVideoElement>(null);
  useEffect(() => {
    if (videoRef) {
      navigator.mediaDevices
        .getUserMedia({
	  // Pass the selected device information directly to the video property
          video: selectedDevice,
        })
        .then((stream) => {
          if (videoRef?.current) {
            videoRef.current.srcObject = stream;
            videoRef.current.play();
          }
        })
        .catch((e) => {
          // Error handling
        });
    }
  }, [videoRef, selectedDevice]);

  return (
    <div>
      <Video ref={videoRef} id="video"></Video>
    </div>
  );
}

Speakers

For speakers, I think there are many use cases where you would want to play arbitrary audio rather than displaying something in the UI, so I will describe how to do that.
Pass the path to the audio file to the constructor of the Audio instance (HTMLAudioElement), and then execute setSinkId.

https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/setSinkId

Note: The reason for temporarily casting to any is that while HTMLAudioElement inherits from HTMLMediaElement and should be able to use the setSinkId provided by HTMLMediaElement, it resulted in a type error in my environment's type definitions (@types/react-dom@* version 17.0.11). Since the operation itself works, I am considering it acceptable for now.

const play = (selectedDevice: MediaDeviceInfo) => {
  let audio = new Audio(`https://example.com/sample.mp3`);
  await (audio as any)?.setSinkId(selectedDevice?.deviceId);
  audio.play();

  // If you want to do something after the audio has played, set an event listener
  audio.addEventListener("ended", () => {
    // Some processing...
  });
};

Summary

In this article, I introduced some tips for handling cameras, microphones, and speakers in a React + TypeScript environment without relying on specific libraries.
Although some parts are written in a slightly rough manner and might need more refinement, I think this is generally how you would implement these features using these APIs.

I hope this content was helpful.