I think what you want to write is a Control Device Driver, see:
Which you would write in Python, perhaps using MidiDings for any time critical needs:
The alternative would be a LV2 plugin, but I don’t think it would be as flexible, but we shall see what others think…
3 Likes