Recording cross browser compatible media

The simplest way to record media in the browser is by using the MediaRecorder. It is available in all browsers since 2021. However it was challenging to use it to actually record cross browser compatible audio and videos. Luckily this has changed now.

A lack of mandatory formats

The MediaRecorder as defined by the MediaStream Recording spec is a high level API to record media in the browser. It is deliberately easy to use. The problem with media though is that it can be recorded in many different formats. The spec however doesn't specify which formats should be supported. It only says that a browser should be able to play the files that it has recorded. It's written down in the section which describes the mimeType attribute.

The User Agent SHOULD be able to play back any of the MIME types it supports for recording. For example, it should be able to display a video recording in the HTML <video> tag.

As a result each browser vendor implemented the formats they liked best. Firefox has a strong reluctance to patent protected codecs. Apple, on the other hand, seemed to not like royalty free codecs. Chrome sits somewhere in between. Google bought some patents in the past just to enable royalty free usage of some codecs. But Chrome also ships with a few patent protected codecs. In the end there was not a single format which could be recorded in every browser.

If you wanted to build a truly interoperable solution which could record and play media in every browser you had to transcode the media that was already encoded in a lossy codec. Or if you didn't mind the large amounts of data you could record in an uncompressed format in the browser at first. Then this recording could be used to encode it to a variety of different formats. I built extendable-media-recorder to do exactly that.

The first format supported in all browsers

Luckily things have changed with the release of Safari 18.4. Since then the following container checks should return true in Chrome, Firefox and Safari.

MediaRecorder.isTypeSupported(
    'audio/webm; codecs=opus'
);
MediaRecorder.isTypeSupported(
    'video/webm; codecs=opus, vp8'
);

All browsers do now support recording audio as Opus packaged in a WebM container. And likewise recording a video in VP8 with Opus wrapped in a WebM container is supported, too.

VP8 is not the most modern codec for encoding video. It got super-seeded by VP9 (already supported in Chrome and Safari) and AV1 (already supported in Chrome). But it's a start. I really like that the first format to be supported across all browsers is from the royalty free category.

A flashy demo

Just to be sure that isTypeSupported() is not lying to us I built a little demo which you can try right here. The following box shows the userAgent string of your browser. Pressing the "generate and record" button will turn that string into a sequence of DTMF tones and generate some basic visuals. All of it will be recorded with the MediaRecorder.

Caution it's going to be noisy and flashy.

Node.js/22

Unfortunately the recording has to happen in real time. It's one of the limitations of the MediaRecorder. But once done you can download the resulting file. Save it on your disk and select it with the form below after opening this page in a different browser.

After adding the file to the form above its audio track will get analyzed to read the DTMF tones to ultimately reconstruct the userAgent string of the browser used to do the recording. If all goes well you should see the userAgent of the browser that was used to record the file in a pink box above the video.

Limitations

It's not all perfect yet. The resulting files can be played in every browser but they can for example not be played with the built-in media player on macOS. VLC will happily play them though. Having support for WebM files in all browsers is great but it doesn't necessarily mean that the format is widely adopted and supported anywhere else.

Another thing to be aware of is that no browser produces files that support seeking. In order to do that the file needs to contain a SeekHead and a Cues element as stated in the Muxer Guidelines. It's impossible to write these elements when producing the recorded file in chunks. The MediaRecorder needs to support this mode because it's possible to request parts of the file before it's finished when using the timeSlice argument or when calling requestData(). However the example above doesn't do that. The recorded file could very well contain the necessary elements.

More nerdy details

There are not only a lot of different file formats and codecs to choose from. There are also a lot of ways to encode media in the very same format. There are plenty settings to tweak. And only of few of them can be controlled when using the MediaRecorder.

The bitrate is one of the few things that can be controlled. However the example above doesn't specify the audioBitsPerSecond, videoBitsPerSecond, or bitsPerSecond respectively. These values default to different values in Chrome and Firefox when compared to Safari. But changing those values might actually break cross browser compatibility. Setting the value for audioBitsPerSecond to 510 kbps will for example record a file in Chrome which cannot be played in Safari.

The Opus audio codec encodes the audio in frames of a certain duration. By default a frame contains 20ms of audio. This is also the default used by WebRTC. Chrome and Firefox use it for recording, too. Webkit chose to record frames of the smallest possible duration of 2.5ms each. There is no way to control this value when using the MediaRecorder.

A WebM container can specify a MuxingApp and a WritingApp. Chrome and Safari set these both to Chrome and WebKit respectively. Firefox sets it to QTmuxingAppLibWebM-0.0.1 and QTwritingAppLibWebM-0.0.1.

Last but not least the files produced by Chrome and Firefox are technically invalid according to mkvalidator which is the official validation tool.

Just a start

The WebCodecs API gets currently implemented across all browsers. As a by-product of that work more codecs reach cross browser compatibility. At some point these codecs may also become available for use with the MediaRecorder.

And until then it is possibly to combine libraries like extendable-media-recorder with WebCodecs to record many more cross browser compatible formats in an efficient way. WebCodecs is not free of compatibility issues either but yours truly is working on a library which aims to address that. But I'll leave that topic for a future article.