I also deal with these strange noises a couple of times. With the Zynthian less, with the MOD-Duo it’s probably because one audio input is mounted too close to the processor board, and with TeensyMIDIAudio/MicroDexed perhaps due to a worse input power filtering.
My experiences are as follows:
- Metal housings are helpful (especially when neon tubes are within reach).
- Pay attention when choosing power supplies!
- Type of sound card in combination with the processor (also the type of installation, whereby with the Raspi these are usually plugged on directly)
I also noticed that if you are using a PCM5102a you should consider to use short and shielded cables for the I2S bus…
I think this is not very helpful - but I am out of ideas 
Regards, Holger