MIDI Strumming

Hey ya’ll!

I thought i’d share a project that I’ve been working on for a bit, and I think its in a (fingers crossed) decent place.

In fact, @riban here helped with some knowledge to make this work in Zynthian and gave the great suggestion to not make it an actual plugin, but a background service. And I do much appreciate especially that last bit because it’s working everywhere now!

Anyway - my idea was a drawing tablet that basically has virtual strings on it that you can strum like a guitar and it acts as a MIDI controller.

I’m calling it Sketchatone and you can find it at sketchatone.com

You’ll find more explanation there including an intro video on the front page. I’ll keep this post short, but I’ll directly share a quick video I made today trying out some of the Zynthian synth leads

5 Likes

Far Out! Now where is that drawing tablet? Do you think the drawing tablet resolution is necessary to have a good ‘musical feel’ or would a multitouch display perhaps be OK - I’m thinking the Zynthian V5 5 inch screen and-or a cell phone screen or even an Android or Apple tablet, with the added advantage that you could display the strings.

Is Sketchatone an open source project?

Yeah, its fun to use!
So yes, I think a touchscreen could have plenty of resolution in theory. I would imagine on tinier screens if you were using your finger it might be harder to hit the right string (of course you’re right….you can see them in this case, so that’s better). Either way, maybe a stylus would be better.

That said, there are a number of things a tablet gives you that a screen can’t AFAIK.

So this project takes advantage of a few other things past strumming. All are configurable, but as default:

  • Vertical coordinates allow pitch bending when you get to the top or bottom of the tablet (yes I know a screen can do this)
  • The stylus supports something like 16,000 levels of pressure. I use this to map note velocity (how hard its pressed)
  • The stylus supports reporting tilt in both the horizontal and vertical direction. I use this right now to control how long the note is held. Straight on interaction will give normal long notes. Tilting will offer various degrees of cutting the note short

Past that, the drawing tablet I have in the video has some nice tactile buttons - and I if you’re using them to chord switch like I am, it goes a big way to feel like a real instrument.

Last point on a tablet vs a screen - when I get into it and really have fun playing and strumming, I’m using some force. I’m hitting things. I feel like if this was on my screen, I’d be a lot more careful and delicate and it just wouldn’t be the same!

In terms of it being open source - not quite. I made it Creative Commons. Basically, I’m happy if people want to fork it and make other awesome stuff with it, but I think I’d be bummed if some corporation just stole and started making money off of it.

So Creative Commons isn’t technically open source. Folks can use it, fork it, do whatever, non-commercially with credit to the original. That said, I’m open to feedback on this if it prevents something I didn’t anticipate

Thanks for the info on what makes the drawing tablet a good choice and the physicality of it!

Creative commons sounds fine, and I wasn’t so much concerned with the details as much as your general intentions.

So maybe one of those mil-spec ruggedized tablets? :wink:

And speaking of strumming, are you aware of Artiphon Instrument 1, and Chorda?

They each, in their own way, try to provide strumming - kind of like a more modern take on the Omnichord. Sadly, Artiphon has ceased operations, but the instruments are still available on the usual used markets.

No! I hadn’t heard of Artiphon, but I came across a few diy Arduino projects. All looked cool, but def more electronics/hardware than I am comfortable with

1 Like

Hi @ben4real !

Congrats for your creation, and thanks for sharing! This really looks like a ton of fun and perfectly aligned with the zynthian spirit.

Do you think we could integrate your driver in the zynthianOS and UI? It would simplify the steps for a lot of potential users.

The sketchatone device should be detected and shown in the MIDI-IN device list, like any other MIDI device, so you can decide from the UI what chains you want to connect, etc.

Ideally, this would be automatic if your software could work as a standard USB-MIDI driver. If not possible/easy, we could find another way.

All the best,

Hey! Glad you dig.

So @riban, when I originally was asking about this pointed me in a good direction. He helped me make it so that “Sketchatone” actually appears in the Zynthian UI as a MIDI device. I wanted to make sure this worked because being able to manage it and go into multitimbral mode is important.

In terms of integrating it into Zynthian, that’s perfectly fine with me - but if I were a maintainer, I might hold out and see if folks can use it well enough as is before I do.

The main reason for this is to get a feel for how folks use this and what’s successful and whats not (especially in terms of all the configuration options) before rolling it into something like Zynthian.

The more specific thing I’d be worried about is drawing tablet support. Right now, it only supports 2 tablets - and thats because those are the ones I currently own. I have a separate project Blankslate: Welcome to Blankslate to test and configure tablets. Basically the “driver” that makes drawing tablets work in this project can be generated here. But again, I’ve only tested on the 2!

Bottom line - if folks want to leverage my work to integrate into Zynthian, I’m all for it. Ya’ll def have my permission. But if I was maintaining Zynthian, there are some complexities that might give me pause!

4 Likes

I found an interesting MIDI Strumming Controller:

1 Like

I swear this was sold out before I started this project - or maybe it was another project like it. Either way I’m glad I didn’t try this. I’m TERRIBLE with hardware. I literally just melted a battery case a month ago because I wasn’t careful with my soldering. I also destroyed 2 Raspberry Pi 5s until I realized I was attaching the GPIO pins the wrong way.

Nope. Software for me.

1 Like

@mr_floydst has experimented with something similar GitHub - mrfloydst/midistrumming: Lets you strum chords on a synthesizer using a smartphone or computer and mouse · GitHub

1 Like

Hi

congrats on creating such a great way to spend way too much time just doodling around.

I managed to track down some used tablets on ebay, unfortunately not the ones you used.

The xppens I sort of got working by changing the ids, but they are lacking the buttons and don’t seem to have tilt.

Would it be possible to define virtual buttons on the right side of the tablet which is not fully used for strumming?

My Huion registers events but something is wrong. This is what I get with systemctl status:

? sketchatone.service - Sketchatone MIDI Strummer
Loaded: loaded (/etc/systemd/system/sketchatone.service; disabled; preset: enabled)
Active: active (running) since Sat 2026-06-20 10:02:46 BST; 1min 10s ago
Process: 1824 ExecStartPre=/bin/sleep 3 (code=exited, status=0/SUCCESS)
Main PID: 1825 (python3)
Tasks: 14 (limit: 4763)
CPU: 936ms
CGroup: /system.slice/sketchatone.service
±1825 /usr/bin/python3 -m sketchatone.cli.server -c /opt/sketchatone/configs/config.json

Jun 20 10:03:26 zynthian python3[1825]: - Report ID 10
Jun 20 10:03:26 zynthian python3[1825]: (Button interface: Report ID None)
Jun 20 10:03:26 zynthian python3[1825]:
Jun 20 10:03:26 zynthian python3[1825]: ? Warning: Unknown Report ID 8
Jun 20 10:03:26 zynthian python3[1825]: Available modes:
Jun 20 10:03:26 zynthian python3[1825]: - Report ID 10
Jun 20 10:03:26 zynthian python3[1825]: (Button interface: Report ID None)
Jun 20 10:03:26 zynthian python3[1825]:
Jun 20 10:03:26 zynthian python3[1825]: ? Warning: Unknown Report ID 8
Jun 20 10:03:26 zynthian python3[1825]: Available modes:

Any hint on what needs changing?

I attached my configs for reference.

xp-pen-deco340.json (5.7 KB)

xp-pen-star640.json (5.7 KB)

huion-h640p.json (7.9 KB)

Hi! To answer your first question, I don’t have plans to have virtual buttons as part of the tablet drawing surface. Honestly, I don’t love the virtual buttons that exist on my Deco 640 - it’s hard to know if you pushed them because they’re not tactile. What I am changing over to in the future is just listening to keyboard events natively (the Huion acts like a keyboard with it’s buttons, and then also hook up a separate mini keyboard if you want)

In terms of what you’re seeing on the Huion not working, the unknown report IDs make sense. I’m finding not much seems to change from tablet to tablet for drawing - but the actual “channel” you listen to DOES change.

I have a project that Sketchatone depends on called Blankslate. It has a tablet learning component to it. You can clone the project and run in Python or Node.js, but ideally the web app would work.

Go to https://bengfarrell.github.io/blankslate/, and click on config → generate new config.
This will get you to the config generator. Ideally you’ll connect your tablet, start the walkthrough and have a brand new tablet config generated at the end.

I’d expect not much is different than what you already have, except for a couple of different IDs.

Fingers crossed this works for you! Unfortunately with this, I just won’t know if the tablets you have are slightly different and hit an aspect I didn’t expect, but from what I’ve seen so far you’ll just have to plug in a couple different numbers

1 Like

Hi.

I tried the Web App under debian, Win7 and Win11. Only the Huion got as far as step 9 (the tablet buttons). I cannot continue there even if I put in 0 for number of buttons. For all the other steps it did capture events.

The Xppens I never got to register any events. I believe there is a driver loaded and that might be in the way. I tried disabling the driver in Windows but I could not get it to recognize any events from the xppens.

What is the best OS to try this app?

Those keyboard buttons sound interesting, so you just connect a usb keyboard and can configure the keypresses in the sketchatone config?

My apologies - I might not have understood before. It sounds like where you are you can get the tablet working in terms of drawing/pressure/tilt etc, just not the buttons.

I interpreted your message before that nothing was working, but now I think I’m wrong - it’s only the buttons.

Anyway yes, those tablet side buttons are the worst part of this because everything else seems straightforward.

I’ve found that on the XP Pen, if you have the driver/app installed and running, it intercepts all internal button events and treats them as keyboard input only. If you had the driver running, but turn it off, and leave your tablet plugged in - you’re still in that mode. You can think of it like a switch to use different interfaces in the tablet.

If the driver is never installed, or off - the button events come in through a normal HID (human interface device) API and don’t act like a keyboard.

On the Huion, the buttons are actually wired to act like a keyboard and come through this weird HID/keyboard input which I made actually work with the current version of this project.

The unfortunate bit is that keyboards are a bit locked down permissions-wise if you need to use them. I think it’s that folks don’t want weird apps logging your keystrokes and spying on you without you as a user knowing about it.

So you’re correct - the web app will not identify buttons if they act like a keyboard (driverless mode on the XP Pen should work).

To listen to key input on the Huion, you need to run as root. At least that’s what I found on MacOS and the Raspberry PI OS (which might as well be Debian AFAIK). I haven’t tried this project on Windows yet, so not sure about there.

Anyway, I’ll launch the python script with sudo to use it if I need the tablet buttons on my Huion.

Again, the web app won’t work for this, but the blankslate Python command line configuration builder tool should work for this (again assuming they are minor differences from my tablet).

I’m also doing a bunch of feature work on v0.3. I’ve added an optional config block here that just allow mapping actual key characters to buttons

“keyboard”: {
“mappings”: {
“1”: “button:1”,
“2”: “button:2”,
“3”: “button:3”,
“4”: “button:4”
}

If you’re a dev and can work with github, the main branch on sketchatone should offer this functionality. I think most things are pretty stable - I’m just working on a Python GUI so I can package this thing up as an appliance (Pi4 + screen).

Ideally though - now that you know you need sudo/root access, maybe the existing Huion profile might work. If you don’t have any luck, I’m halfway inclined to buy your tablet because I’m curious. If you DO get it working, I’d love to add your config profile to the project.

Hope that helps!

1 Like

Hi

I got all tablets working with basic strumming. Tilt is not working and the buttons do not work. Do all Tablets support Tilt, or is it only the better ones? On the Huion I had to change “reportId”: 10, to “reportId”: 8, inside the json file.

Since the Web App did not get past the Button Test I tried both npm and python.

I installed npm on Debian trixie but I get this certificate error:

~/blankslate$ npm install
npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm WARN deprecated @humanwhocodes/config-array@0.13.0: Use @eslint/config-array instead
npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm WARN deprecated @humanwhocodes/object-schema@2.0.3: Use @eslint/object-schema instead
npm WARN deprecated eslint@8.57.1: This version is no longer supported. Please see Version Support - ESLint - Pluggable JavaScript Linter for other options.
npm ERR! code UNABLE_TO_GET_ISSUER_CERT_LOCALLY
npm ERR! errno UNABLE_TO_GET_ISSUER_CERT_LOCALLY
npm ERR! request to https://artifactory.corp.adobe.com/artifactory/api/npm/npm-virtual/ws/-/ws-8.18.3.tgz failed, reason: unable to get local issuer certificate

I then tried the python version as normal user and root but it never sees a device. Do I need to put my Device Ids somewhere inside the code?

venv) root@rechenbrett:~/blankslate/python# tablet-config-generator

======================================================================

:artist_palette: TABLET CONFIGURATION WALKTHROUGH

This interactive wizard will help you generate a tablet configuration.
Follow the on-screen instructions for each step.

[?] Select data source:

Real HID Device
Mock Device (for testing)
Exit

✗ Error: No HID devices found

✗ Error: Failed to initialize device reader

(venv) root@rechenbrett:~/blankslate/python# python -c “import hid; print([d for d in hid.enumerate() if d[‘usage_page’] == 13])”

(venv) root@rechenbrett:~/blankslate/python#

I never installed any drivers for the tablets but it just works. So how do I unload any loaded driver?

Hi, I took a deeper look at this, and honestly I was a bit worried that you were using a tablet that didn’t work here for some reason and I got curious. So - I just bought the Huion h640p (one of the ones you’re trying to use according to your JSON file names).

For the other ones, the XP 340, I couldn’t find that anywhere so couldn’t speak to it, and then the other XP you had, you are correct, it doesn’t look like tilt is supported.

On the Huion, assuming we have the same one now, it does look like tilt works (I’ve been using it).
Your earlier log warning you on the report IDs, it makes sense now. The correct report ID is 10. You said you changed it to 8. I’m not sure what led you to think 8 is correct, but I think this led you down the wrong path. Some of the utilities I have will work - like the tablet-events Python script. It will show all events streaming in, but it WILL say that it’s using report ID of 10 even though the config specifies 8.

In any event I pushed the config for this tablet: sketchatone/public/configs/devices/huion_inspiroy-h640p.json at main · bengfarrell/sketchatone · GitHub

Not only does tilt work for me, but the side 6 buttons work as well. I know you said you were running as root, and I might be wrong to suggest this, but maybe just run with sudo anyway if you fail to get the buttons to register.

In terms of the configuration utility. Unfortunately, the Node.js version was my bad. I’m using a work machine and the package-lock.json was forcing packages to go through my company’s internal artifactory package manager. I’ve deleted this in the repo, but it’s as easy as just deleting package-lock.json from the project root.

For the Python utility, I can’t necessarily explain why it couldn’t find any devices. Although, I will say that I’ve accidentally left open tablet sessions in another terminal, and if it’s occupied like that it can’t be read elsewhere.

Lastly - you asked a question about drivers. So yes if it just works and you haven’t installed drivers, don’t worry about it. It actually doesn’t need a driver to function. These tablets seem to usually have a couple different interfaces. One works without any drivers, and I’ve found if you use a driver it just switches to another interface with slightly different values (like the reportId).

You may find that if you installed a driver, run the configuration, it will work. But then with a driver or app running for the device, it will no longer work with the same configuration. I have a “modes” list in the JSON config, but I add the different modes for this by hand to cover my bases on a specific tablet model.

I hope this works for you!

1 Like

Thanks for going through all this trouble.

Looks like there are many different versions out there, or older models. Mine really needs reportid 8 to work.

After deleting this lock file I got the Node.js version running as root.

Turns out none of my tablets support tilt.

The wizard did not recognize the buttons on the huion, but it let me save the file after a while.

Here are my three json files created with the wizard.

xp-pen_g430s_config.json (1.1 KB)

inspiroy_h640p_config.json (1.9 KB)

star_g640_config.json (1.1 KB)

For sure, I was definitely curious - I’d really hope that as I try more tablets they aren’t so different than what I’ve discovered so far.

It sounds like for the most part that the drawing pad is holding up. Its a bummer you don’t have tilt support on yours!

The buttons are the most infuriating part of this - it took me a while to discover what these tablets are doing and how they differ. Makes me think my inclination towards really just giving up direct device support and supporting keyboard input as keyboard input is the way to go. If you’re on sketchatone main (what will be version 0.3), you’ll see support for that and you can just map characters to buttons in the JSON config (the sketchatone config, not the blankslate device one)