Multimedia

GStreamer

GStreamer is a powerful pipeline-based multimedia framework. It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on NeuralPlex. This release uses the gstreamer-imx set of GStreamer 1.0 plugins for NXP’s i.MX platform, which make use of the i.MX multimedia capabilities.

There are two major commands: gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element. gst-launch - allows you to create and execute GStreamer pipelines.

root@neuralplex:~# gst-inspect-1.0 | grep sink
pulseaudio:  pulsesink: PulseAudio Audio Sink
hls:  hlssink2: HTTP Live Streaming sink
hls:  hlssink: HTTP Live Streaming sink
bluez:  avdtpsink: Bluetooth AVDTP sink
bluez:  a2dpsink: Bluetooth A2DP sink
coreelements:  filesink: File Sink
coreelements:  fdsink: Filedescriptor Sink
coreelements:  fakesink: Fake Sink
alsa:  alsasink: Audio sink (ALSA)
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
...

Detailed hlep on specific element:

root@neuralplex:~# gst-inspect-1.0 <name of the element>

Here are a couple examples of playing video and audio files:

# Play Video
gst-launch-1.0 filesrc location=big_buck_bunny_1080p_h264.mov ! decodebin \
  ! imxvideoconvert_g2d ! waylandsink

# Play Audio
gst-launch-1.0 filesrc location=Sultans_Of_Swing.mp3 ! decodebin ! alsasink

# Play Video and Audio
gst-launch-1.0 filesrc location=big_buck_bunny_1080p_h264.mov ! decodebin name=dec \
  ! imxvideoconvert_g2d ! waylandsink dec. ! queue ! audioconvert ! alsasink device=hdmi

ALSA

# Speaker Test (Play 440Hz sin wave)
speaker-test -c8 -t sin -f440 -P9 -s7

# Speaker Test (Play wav file on specific speaker output)
speaker-test -c8 -t wav -w ~/Welcome_to_the_Jungle.wav -P9 -s7

# Speaker Mapping:
s1 = OUT1
s2 = OUT5
s3 = OUT2
s4 = AMP_L
s5 = OUT4
s6 = OUT3
s7 = AMP_R
s8 = OUT6

# Record Audio
arecord -D plughw:0,0 -r 48000 -f S32_LE sample.wav -c4

# Play Audio
aplay Sultans_Of_Swing.wav

Audio Routing

The NeuralPlex has two amplified speaker outputs for connecting directly to two speakers for stereo audio. These two channels are connected to outputs 7 and 8 on the audio codec. When trying to play an audio file in Qt, this typically defaults to channels 1 and 2. Audio routing enables Qt to play audio on channels 1 and 2, and ALSA knows to re-route the audio to channels 7 and 8 in the backend. To adjust audio routing, edit the /etc/asound.conf and add entries for ttable. As shown below, (index zero-based) channel 0 is re-routed to channel 6 and volume is set to 25%, and channel 1 is re-routed to channel 7 and volume is set to 25%.

pcm.!default {
        type asym
        playback.pcm {
                type plug
                slave {
                        pcm "hw:PCM3168A,1"
                        rate 48000
                        format S24_LE
                        channels 8
                }
                # Route audio amp channels 6 and 7 to the
                # front left and right speakers and attenuate to 25%
                ttable.0.6 = 0.25
                ttable.1.7 = 0.25
        }
        capture.pcm {
                type plug
                slave {
                        pcm "hw:PCM3168A,0"
                        rate 48000
                        format S32_LE
                        channels 6
                }
        }
}

# Bluetooth A2DP sink
pcm.bt-sink {
        type plug
        slave {
                pcm "hw:PCM3168A,1"
                rate 48000
                format S24_LE
                channels 8
        }
        # Route audio amp channels 6 and 7 to the
        # front left and right speakers and attenuate to 25%
        ttable.0.6 = 0.25
        ttable.1.7 = 0.25
}

# HDMI Audio Output
pcm.!hdmi {
        type plug
        slave {
                pcm "hw:imxaudiohdmitx,0"
                rate 48000
                format S32_LE
                channels 2
        }
}

ctl.!default {
        type hw
        card PCM3168A
}

Bluetooth Audio

For Bluetooth audio transmission, NeuralPlex uses the BlueALSA backend, enabling audio data exchange via Bluetooth with audio playback/capture devices, such as smartphones, laptops, headsets, and/or speakers.

The following sections have examples of how to use the NeuralPlex as a Bluetooth speaker for your host device

Connect Phone to NeuralPlex

Make the Board Discoverable and Pairable:

Enter bluetoothctl:

root@neuralplex:~# bluetoothctl

Then run:

[bluetooth]# power on
[bluetooth]# discoverable on
[bluetooth]# pairable on

Pair and Connect from Your Phone:

On your phone, enable Bluetooth, search for devices, and pair with the NeuralPlex (it should appear as “neuralplex”). While bluetoothctl is open, you will need to type yes to authorize multiple services (audio and phone calls) during the pairing process. Once paired, connect the phone to the NeuralPlex as an audio device (A2DP profile). Your phone should show it as connected for media audio.
Trust device (optional)

If you wish to add your phone to the list of devices to trust, which avoids needing to re-pair each time you connect, make a note of your phone’s MAC address and execute the following command:
bluetoothctl
```
# Use your phone's MAC address (e.g. 00:11:22:33:44:55)
[bluetooth]# trust 00:11:22:33:44:55
```
Exit bluetoothctl with exit or Ctrl+D.

Play Audio

Before you can play audio from your phone, you will need to ensure that the bluealsa-aplay service is running:

# Check the status of the blueasla-aplay service
root@neuralplex:~# systemctl status bluealsa-aplay

# If it's not running, enable and start it:
root@neuralplex:~# systemctl enable bluealsa-aplay
root@neuralplex:~# systemctl start bluealsa-aplay

Play an audio file on your phone, and will be streamed to the bt-sink profile in ALSA.

Connect NeuralPlex to phone

Open bluetoothctl
Terminal window
```
root@neuralplex:~# bluetoothctl
```
Inside the prompt, power on Bluetooth
bluetoothctl
```
[bluetooth]# power on
```
Start scanning for devices
bluetoothctl
```
[bluetooth]# scan on
```
Find your device’s MAC address (e.g., 00:11:22:33:44:55)

Pair, Trust, and Connect

[bluetooth]# pair 00:11:22:33:44:55
[bluetooth]# trust 00:11:22:33:44:55
[bluetooth]# connect 00:11:22:33:44:55

Exit
bluetoothctl
```
[bluetooth]# quit
```