That module doesn’t provide GPIO Pin access, so you might be limited to a high latency USB Audio Codec, anything other than USB MIDI might be tricky too.
A custom board ibcluding the Zyithian essentials would be best. That board appears to have space enough for an added consumer grade audio In/Out Codec and DIN Midi circuitry. Here’s an example board with built in Codec.
One of the smaller compute module hosts is shown here.