Add stereo support to usb_audio

[relic-se]

Updates USB Audio device descriptors to assume stereo operation. Mono operation (channel_count=1) is still supported although the device will enumerate as stereo.

Example Code

boot.py

import usb_audio
import usb_hid
import usb_midi

usb_hid.disable()
usb_midi.disable()

# Alter settings here to test different configurations
usb_audio.enable(
    sample_rate=48000,
    channel_count=2,
    microphone=True,
    speaker=True,
)

code.py

from synthio import Synthesizer, Note
from usb_audio import usb_microphone, usb_speaker

SAMPLE_RATE = 48000
CHANNEL_COUNT = 2

# This will need to be replaced with your hardware of choice
from relic_synthiota import Synthiota
synthiota = Synthiota(
    sample_rate=SAMPLE_RATE,
    channel_count=CHANNEL_COUNT,
)
audio = synthiota.audio

if usb_speaker is not None:
    audio.play(usb_speaker)

if usb_microphone is not None:
    synth = Synthesizer(
        sample_rate=SAMPLE_RATE,
        channel_count=CHANNEL_COUNT,
    )
    usb_microphone.play(synth)
    synth.press(Note(440))

if usb_microphone is not None or usb_speaker is not None:
    try:
        while True:
            pass
    except KeyboardInterrupt:
        pass

Notes

• The bits_per_sample argument is removed from usb_audio.enable(...) since it is always assumed to be 16. (open to input here)
• For operation on most devices, usb_hid (and potentially usb_midi) will need to be disabled to avoid reaching the limit of USB endpoint pairs when operating in bidirectional mode. Only uses 2 endpoints when in mono or stereo bidirectional mode which is consistent with previous operation.

In Progress

• Mono output through usb_microphone results in random noise.
• Mono input through usb_speaker results in distorted audio.
• Bidirectional operation is somewhat functional, but an issue with the device descriptor is causing distortion with usb_speaker. Stereo usb_microphone operates correctly.

For interest: @FoamyGuy

[FoamyGuy]

Thanks for working on this! I will test out mono speaker, and try to resolve the issues with microphone.

[mikeysklar]

@relic-se

#11093 (tinyusb bump) just merged, which migrates usb_audio to the TUD_AUDIO20_/AUDIO20_ names and touches tusb_config.h + usb_audio_descriptors.h, so this branch will need a rebase onto main.

[relic-se]

@mikeysklar Thanks for your work here. I'll make sure to follow your guidelines closely.

[relic-se]

@mikeysklar I've updated my code accordingly, and it's looking good. There are still some bugs within this PR which prevents it from being ready to merge at this time.

[relic-se]

As an aside, Linus Torvalds appears to have some success with stereo headset operation on his RP2350 guitar pedal: https://github.com/torvalds/GuitarPedal/blob/main/Software/usb-device.c

[relic-se]

Thanks a million to @FoamyGuy who worked out the bugs with this update! My tests so far indicate optimal performance and stability.

@tannewt Any interest in a final review?

[mikeysklar]

I've been writing a guide this week for building a HIL farm. This PR looked like a good way to test the farm's ability. Built 50a1195 for all 8 boards, all build clean.

Stereo looks great on RP2040 and RP2350. Sent 1 kHz into left and 3 kHz into right, recorded back on the host:

channel	1 kHz	3 kHz
left	8999.7	0.1
right	0.0	8999.7

18000 in, 18000 out. Mono arrives identical on both channels, L==R on 100% of frames. Same through synthio with notes alternating hard left and right.

One finding on nRF52840. The combined microphone=True, speaker=True case enumerates but never transfers, arecord gives Input/output error and zero frames. Single direction is fine:

config	endpoints	result
mic only	0x88 (8 IN)	works
speaker only	0x08 (8 OUT)	works
mic + speaker	0x87 (7 IN), 0x06 (6 OUT)	no data

The code comment already calls this unsupported on nRF52, but it might be closer than it looks. dcd_edpt_open() rejects ISO on any number except 8, but that is one endpoint number, not one direction. The same driver runs ISOIN and ISOOUT together on EP8 via ISOSPLIT (OUT, IN). So ep_out = USB_AUDIO_ISO_EP_NUM and ep_in = USB_AUDIO_ISO_EP_NUM | 0x80 might just work. I have not tried the patch. Disabling usb_midi and usb_hid does not help, the endpoints just move to 0x04/0x85.

Tiny separate thing, the docstring example passes bits_per_sample, which is not in the signature:

>>> usb_audio.enable(sample_rate=16000, bits_per_sample=16)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unexpected keyword argument 'bits_per_sample'

---

The farm, for reference. Eight boards on switchable USB hubs, built and flashed one at a time from the same tree. Versions below are read back off each board's boot_out.txt after flashing:

arch	board	usb_audio	running
rp2040	Metro RP2040	enabled	10.3.0-alpha.3-25-g50a1195c55
rp2350	Metro RP2350	enabled	10.3.0-alpha.3-25-g50a1195c55
nrf52840	Feather nRF52840 Express	enabled	10.3.0-alpha.3-25-g50a1195c55
samd21	Metro M0 Express	off	10.3.0-alpha.3-25-g50a1195c55
samd51	Metro M4 Express	off	10.3.0-alpha.3-25-g50a1195c55
stm32f405	Feather STM32F405	off *	10.3.0-alpha.3-25-g50a1195c55
esp32s2	Metro ESP32-S2	off	10.3.0-alpha.3-25-g50a1195c55
esp32s3	Metro ESP32-S3	off *	10.3.0-alpha.3-25-g50a1195c55

* Out of curiosity I forced CIRCUITPY_USB_AUDIO=1 on these two, which is not a supported config. Both build, but enable() returns success and no audio interfaces show up, so nothing to report there yet. Not what I set out to test.

[mikeysklar]

Had a look at that nRF52840 headset case and it turns out to be a small fix, so I opened relic-se#2 against your branch.

Both directions can share endpoint 8. The constraint there is the endpoint number, not the direction, and TinyUSB splits the ISO buffer when ISOIN and ISOOUT are both open. microphone=True, speaker=True now enumerates on 0x08/0x88 and streams both ways at once on nRF52840. RP2040 is unchanged since it is all behind forced_iso_ep.

[relic-se]

Thank you for your extensive testing and subsequent fix, @mikeysklar . I don't have an nRF52840 board to test this myself, but I trust your review. It seems that we've limited this functionality to RP2xxx and nRF52840, but it would be interesting to see it extended to other platforms in the future, namely ESP32-S3 and M4/M7 as you've suggested.

Tiny separate thing, the docstring example passes bits_per_sample, which is not in the signature:

As listed in the notes of the original description, this PR removes that argument. Good catch with those docstrings. I've updated them to remove that argument. But tbh, I'm still on the fence as to whether or not to include this argument. 😆

[mikeysklar]

Retested at bf828d8 on the farm. All 8 boards build and flash clean, and stereo still checks out on RP2040, RP2350 and nRF52840. The headset fix works from the merged tip too, nRF52840 comes up on 0x08/0x88 and streams both directions at the same time.

I'd leave out the bit setting argument. It implies a bit setting other than 16-bit can be set (which it cannot be today). So no changes needed on that.

I'll look into getting ESP32-S3 going.

[mikeysklar]

Spent some time on ESP32-S3 today and it looks promising, but I would not hold this PR for it.

It enumerates as a proper UAC2 device with a valid stereo descriptor and the isochronous endpoint opens, but getting audio actually flowing looks like it will need both tinyusb and CircuitPython patches, none of it in shared-module/usb_audio.

This one looks done.

• 8 of 8 boards build and flash
• The stereo path is audible on RP2040, RP2350 and nRF52840