Adventures in pipewire

Baggypants · August 14, 2024, 11:23pm

So, spurred on by the meeting at ZynthianClub I did a bit of poking about with pipewire in Zynthian. Which is roughly documented here

In short, I got pipewire working using the supplied packages, ran some synths, connected midi inputs and audio outputs, had a go with switching between the hifiberry and a usb audio device, ran with multiple outputs, and it all worked.

Next I want to wrangle automating connections with wireplumber, so when I unplug the usb device it automatically switches back to the hifiberry, and back when you plug it back in.

Then I guess I want to rip the zynmixer out from the rest of the code and run it on top.

I didn’t do anything with setting the latency, it seemed as good as Jack for me.

Wapata · August 15, 2024, 8:17am

By the way, how do you measure latency ? A loop from audio out to audio in and a engine that give the result ??

Baggypants · August 15, 2024, 9:23am

You can do that but most of the time it’s calculated internally from how many periods and samples you set the jack config to.

The way I tend to do it is record the output triggered from a midi keyboard against the keyboard audio.

riban · August 15, 2024, 10:55am

Jack provides a tool to do this (within jack and I suppose jack emulation in pipewire) called jack_delay that appears as a source and destination within the routing graph and measures the time between the audio it produces and it receives. You loop the audio at the physical i/o to get full round-trip latency measurment.

Baggypants · August 23, 2024, 6:34pm

@riban @jofemodo How do I get the zynmidirouter to load without complaining jack isn’t running. The synths (setbfree & jalv) just link to the jacklib, so they run with the pipewire compatibility lib. But the router complains about jack and dies.

riban · August 23, 2024, 9:31pm

You have to run it in an environment that knows about Pipewire’s jack emulation. I can’t remember exactly but I think there is an env var you should set. I had to do that each time and couldn’t figure out how to make it work automatically.

Baggypants · August 24, 2024, 8:29am

You shouldn’t need the environment variable if you re-order the library load order.

This is how I setup the environment.

apt update
sudo apt install pipewire-audio pipewire-jack
cp /usr/lib/systemd/user/pipewire.* /etc/systemd/user
sed -i 's/^ConditionUser=!root/# ConditionUser=!root/' /etc/systemd/user/pipewire.{service,socket}
touch /etc/pipewire/media-session.d/with-jack
cp /usr/share/doc/pipewire/examples/ld.so.conf.d/pipewire-jack-aarch64-linux-gnu.conf /etc/ld.so.conf.d/0-pipewire-jack-aarch64-linux-gnu.conf
ldconfig
systemctl --user daemon-reload
systemctl daemon-reload
systemctl disable --now zynthian jack2
systemctl --user enable --now wireplumber
pw-play /usr/share/sounds/alsa/Front_Left.wav

I also have to mask the jack2 service because it loves to jump in and interfere.

The midi router is doing “some test” that the softsynths don’t bother with.

Unless it has some other environment variable I haven’t spotted.

riban · August 24, 2024, 9:07am

I got it working but had to start Zynthian with the pipewire launch tool. I didn’t resolve that issue.

Baggypants · August 24, 2024, 9:41am

The library reorder should solve that.

cp /usr/share/doc/pipewire/examples/ld.so.conf.d/pipewire-jack-aarch64-linux-gnu.conf /etc/ld.so.conf.d/0-pipewire-jack-aarch64-linux-gnu.conf
ldconfig

Baggypants · August 24, 2024, 2:26pm

Well, I’ve got it to move on a tiny bit further…

But now something is segfaulting.

ooming image by 150%...done
  Merging...  Clipping image...done
done
didn't find evidence of prior run.
  Building XImage...done
ZynCore: Configured 4 x Logical Zynswitches...
ZynCore: Setting-up 4 x Physical Zynswitches...
ZynCore: Setting-up 4 x Zynpots (zyncoders)...
ZynCore->gpiod_start_callbacks: Callback thread created successfully
Started libzynaudioplayer using libsndfile-1.2.0
Started libzynmixer
./zynthian.sh: line 270: 3096269 Segmentation fault      ./zynthian_main.py
/zynthian/config/img/fb_zynthian_message.png is 480x320 PNG image, color type RGB_ALPHA, 8 bit
  Zooming image by 150%...done
  Merging...  Clipping image...done
done

Baggypants · August 24, 2024, 2:48pm

Ah, this is the error I was seeing before. “Error connecting with Jack server”

24 15:43:23 zynthian startx[3096970]: ZynCore: Configured 4 x Logical Zynswitches...
Aug 24 15:43:23 zynthian startx[3096970]: ZynCore: Setting-up 4 x Physical Zynswitches...
Aug 24 15:43:23 zynthian startx[3096970]: ZynCore: Setting-up 4 x Zynpots (zyncoders)...
Aug 24 15:43:23 zynthian startx[3096970]: ZynCore->gpiod_start_callbacks: Callback thread created successfully
Aug 24 15:43:23 zynthian startx[3096970]: ZynMidiRouter: Error connecting with jack server.
Aug 24 15:43:23 zynthian startx[3096970]: ERROR:zynthian_gui_config.<module>: lib_zyncore: Failed to initialise zynmidirouter (2)
Aug 24 15:43:23 zynthian startx[3096989]: /zynthian/config/img/fb_zynthian_message.png is 480x320 PNG image, color type RGB_ALPHA, 8 bit
Aug 24 15:43:23 zynthian startx[3096989]:   Zooming image by 100%...done
Aug 24 15:43:23 zynthian startx[3096989]:   Merging...done
Aug 24 15:43:23 zynthian startx[3096989]:   Building XImage...done
Aug 24 15:43:34 zynthian startx[3097000]: ZynCore: Configured 4 x Logical Zyn

Baggypants · September 2, 2024, 3:34pm

The exact point it is segfaulting is when it instantiates the zynthian mixer lib

github.com

zynthian/zynthian-ui/blob/5601f7e30fb6ccd49ea63f14c8c398769704a95d/zyngine/zynthian_engine_audio_mixer.py#L47


      
          
          class zynmixer(zynthian_engine):
          
          	# Subsignals are defined inside each module. Here we define audio_mixer subsignals:
          	SS_ZCTRL_SET_VALUE = 1
          
          	#	Function to initialize library
          	def __init__(self):
          		super().__init__()
          		self.lib_zynmixer = ctypes.cdll.LoadLibrary("/zynthian/zynthian-ui/zynlibs/zynmixer/build/libzynmixer.so")
          		self.lib_zynmixer.init()
          
          		self.lib_zynmixer.setLevel.argtypes = [ctypes.c_uint8, ctypes.c_float]
          		self.lib_zynmixer.getLevel.argtypes = [ctypes.c_uint8]
          		self.lib_zynmixer.getLevel.restype = ctypes.c_float
          
          		self.lib_zynmixer.setBalance.argtypes = [ctypes.c_uint8, ctypes.c_float]
          		self.lib_zynmixer.getBalance.argtypes = [ctypes.c_uint8]
          		self.lib_zynmixer.getBalance.restype = ctypes.c_float
          
          		self.lib_zynmixer.setMute.argtypes = [ctypes.c_uint8, ctypes.c_uint8]

using gdb shows that for some reason its failing to load a different library?

[New Thread 0x7fe53ef180 (LWP 867459)]
Started libzynmixer

Thread 12 "python3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fe5bff180 (LWP 867458)]
__GI___memset_generic () at ../sysdeps/aarch64/memset.S:86
86      ../sysdeps/aarch64/memset.S: No such file or directory.
(gdb) bt
#0  __GI___memset_generic () at ../sysdeps/aarch64/memset.S:86
#1  0x0000007fe6412da4 in onJackProcess () from /zynthian/zynthian-ui/zynlibs/zynmixer/build/libzynmixer.so
#2  0x0000007ff5e64dd8 in ?? () from /usr/lib/aarch64-linux-gnu/pipewire-0.3/jack/libjack.so.0
#3  0x0000007ff5cf8928 in ?? () from /usr/lib/aarch64-linux-gnu/spa-0.2/support/libspa-support.so
#4  0x0000007ff5d78a7c in ?? () from /lib/aarch64-linux-gnu/libpipewire-0.3.so.0
#5  0x0000007ff7d4ee58 in start_thread (arg=0x7fffff7797) at ./nptl/pthread_create.c:442
#6  0x0000007ff7db7f9c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79
(gdb)

that ../sysdeps/aarch64/memset.S looks like the problem maybe? I don’t know if the backtrace helps.

It looks like it is certainly loading the pipewire jack lib though.

Ah, I misunderstood the error. GDB is trying to load the source for the memset function but it’s not installed.

So I guess it’s something about these lines where memset is called:

github.com

zynthian/zynthian-ui/blob/5601f7e30fb6ccd49ea63f14c8c398769704a95d/zynlibs/zynmixer/mixer.c#L159


      
              pthread_exit(NULL);
          }
          
          static int onJackProcess(jack_nframes_t nFrames, void *pArgs) {
              jack_default_audio_sample_t *pInA, *pInB, *pOutA, *pOutB, *pChanOutA, *pChanOutB;
          
              unsigned int frame, chan;
              float curLevelA, curLevelB, reqLevelA, reqLevelB, fDeltaA, fDeltaB, fSampleA, fSampleB, fSampleM;
          
              // Clear the normalisation buffer. This will be populated by each channel then used in final channel iteration
              memset(pNormalisedBufferA, 0.0, nFrames * sizeof(jack_default_audio_sample_t));
              memset(pNormalisedBufferB, 0.0, nFrames * sizeof(jack_default_audio_sample_t));
          
              // Process each channel
              for (chan = 0; chan < MAX_CHANNELS; chan++) {
                  if (isChannelRouted(chan) || (chan == (MAX_CHANNELS - 1))) {
                      //**Calculate processing levels**
          
                      // Calculate current (last set) balance
                      if (g_dynamic[chan].balance > 0.0)
                          curLevelA = g_dynamic[chan].level * (1 - g_dynamic[chan].balance);

riban · September 2, 2024, 3:56pm

If that is the trigger then either pNormalisedBufferA/B is not initialised or sizeof(jack_default_audio_sample_t) is returning an invalid value. I wonder if the buffers are defined with this data type?

[Edit] Buffers are defined as pointers to jack_default_audio_sample_t so that looks okay. Maybe there is an issue with the default audio data size. Maybe pipewire uses a different audio data size. Maybe the lib needs to be recompiled against the pipewire jack compatibility layer. Try recompiling audio mixer. Ensure the dev packages for pipewire (jack layer) are installed.

Baggypants · September 2, 2024, 4:19pm

That didn’t seem to make much of a difference.

I also tried commenting out the two lines, and it segfaulted in a different memory location and then powered off the pi which made getting a backtrace a little more tricky. So I think we’re close

Baggypants · September 3, 2024, 8:49am

ok I compiled with debugging symbols, here is the backtrace. nFrames is set to 1024 pArgs is 0x0

Thread 2 "python3" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1525.1627]
__GI___memset_generic () at ../sysdeps/aarch64/memset.S:86
86	../sysdeps/aarch64/memset.S: No such file or directory.
(gdb) bt
#0  __GI___memset_generic () at ../sysdeps/aarch64/memset.S:86
#1  0x0000007fe6412da4 in onJackProcess (nFrames=1024, pArgs=0x0) at /zynthian/zynthian-ui/zynlibs/zynmixer/mixer.c:159
#2  0x0000007ff5b34dd8 in on_rtsocket_condition (data=0x2937920, fd=<optimized out>, mask=<optimized out>)
    at ../pipewire-jack/src/pipewire-jack.c:1546
#3  0x0000007ff59c8928 in ?? () from target:/usr/lib/aarch64-linux-gnu/spa-0.2/support/libspa-support.so
#4  0x0000007ff5a48a7c in ?? () from target:/lib/aarch64-linux-gnu/libpipewire-0.3.so.0
#5  0x0000007ff7d4ee58 in start_thread (arg=0x7fffffd417) at ./nptl/pthread_create.c:442
#6  0x0000007ff7db7f9c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:79

so I set the buffers to 256 as describe here:
https://linuxmusicians.com/viewtopic.php?t=25768

pw-metadata -n settings 0 clock force-quantum 256

and it doesn’t segfault anymore. It doesn’t load properly either. it’s stopped on the zynthian loading screen and the log hangs at:

Process python3 created; pid = 1540
Listening on port 12345
Remote debugging from host ::1, port 57490
ZynCore: Configured 4 x Logical Zynswitches...
ZynCore: Setting-up 4 x Physical Zynswitches...
ZynCore: Setting-up 4 x Zynpots (zyncoders)...
ZynCore->gpiod_start_callbacks: Callback thread created successfully

riban · February 14, 2025, 2:56pm

lib_zyncore_init() fails unless zynthian_gui_config.py is run with pw-jack wrapper. It looks like initiating a library loaded with cdll fails to use the configured pw-jack lib defined with /etc/ld.so.conf.d/0-pipewire-jack-aarch64-linux-gnu.conf.

Also, even launching zynthian with pw-jack wrapper fails because setting aliases fails (maybe not supported by pipewire?) and soundcard port names are all rather odd, like 7.1 names rather than simple input/output 1,2,3…

Tithrion · February 14, 2025, 8:51pm

That might be due to a (wrong) default profile.
Try to set the profile to “Pro Audio”, that changes the naming to SOMETHING 1,2,3,…, at least at my workstation…

riban · February 14, 2025, 9:52pm

I spent some time today trying to get pipewire working bit failed. I got sound out but there are challenges to make zynthian work. I came to the conclusion that we probably should not try to get zynthian working in top of the pw-jack abstraction / interface layer. We will spend a lot of time and effort to get to a suboptimal compromise. If it were quick and easy then there may be benefit but that seems unlikely. Instead we should get zynthian working natively with pipewire. We would still require the pw-jack plugin for jalv and possibly other engines.

If someone wants to do this then feel free. I am yet to be convinced that there is sufficient benefit. It’s is something worth investigating because jack may become unsupported but we don’t need to move away from jack yet.

Baggypants · February 15, 2025, 1:47am

Hotplug, multidevice support and the handy tools for tracing xruns are the possible arguments for pipewire, some of which we have now. Maybe support for lowlatency bluetooth in the future. And the fact it’s fairly well maintained I suppose.

riban · February 15, 2025, 7:07am

We have hotplug, multidevice support. We have BLE. Jack is currently supported. So, IMO insufficient benefit to dedicate effort (yet) but… feel free to make it work!