My favourite quote so far from the copious learned paper 
“The pipe organ is essentially a performer driven timbral synthesiser, offering a number of different sounds that are acoustically
blended together.”
I have raised a feature request to improve access for visually impaired users. I assigned the ticket to myself but am not sure how quickly I will get something working. Maybe later this week but as usual, no promises.
The Stevie Wonder and Ray Kurzweil collaboration (105 sec. Video)
(Keyboard development followed Kurzweil’s creation of a text reader)
No surprise his keyboard ergo-metrics are the best
I’ve been interested in adaptive technology for some time. I was just searching for references to using IR LEDs for tactile heat source button indicators, before reading about light bulb heat above.
Getting a unique sound when lightly gliding over a control array button may be the most direct replacement for indicator LEDs. A few very short ring tone effects are probably more usable than spoken words as the indicator. Really annoying in a music playing situation, perhaps a private earpiece would help. It might be an interesting research exercise to see if capacities switch technology can be adapted to glide over detection.
The most elaborate tactile feedback scheme: Ultrasonic emitter matrix panels are used for Haptic interfaces allowing one to “feel” simple invisible geometric shapes. Ultrahaptics.com and Ultraleap hand sensor are combined to provide unique 3D interaction . Perhaps more of a $3000 novelty, I have not seen them used in adaptive technology.
(Similar ultrasonic arrays are used in directional Audio Flashlight “Holographic Whisper” sound projectors. Hard to follow 20 min seminar talk)
[Native Instruments’ Komplete Kontrol – a range of keyboards with touch-sensitive rotary encoders and auditory feedback functionality.
In a Red Bull Music Review (I only knew they sponsored racers) where tactile and auditory feedback are mentioned, a couple of users are briefly profiled.
A Native Instruments article from August 2017
Talking keyboards with Andre Louis
“When I go over and touch or press a knob on the keyboard it will actually describe what I’m selecting,”
May 18th Native Instruments Releases Komplete Kontrol 2.0.2 With MIDI And Controller Enhancements
1 Like
Music Accessibility - Products, Groups and Forums List Nov. 2017
This little 4 page PDF may be of some use.
.
Seeing With Your Ears: Making Music Production Accessible for the Visually Impaired.
Follows the technology visually impaired producer Jason Dasent employed through hiss career.
I have been one of Native Instruments customers / endorsers over the years so used to train customers with this hardware and software approach. Though this is a good system, it has some serious drawbacks. they include most concerningly poor accessibility support for screen readers like VoiceOver for the actual Komplete Kontrol app which is the driving force behind running any of the instrument libraries, issues include access to the preferences to configure audio and MIDI and as such require sighted assistance, which defeats the purpose of the need. Further to that, the speech engine takes the system speech UI where parameters such as speech rate, volume, etc cannot be changed, so if a user relies on system speech for other functions not assigned to a screen reader such as voiceover, then there can be some confusion. I’ve owned both the original S88 and S88 MK2, when the MK2 released, support was worse, a large number of spoken functions disappeared making navigation and announcement of functions poor.
I left working with Native Instruments after a number of issues arose and ended my association with their software and hardware. ok, part of me regrets this move, but at the same time, I wasn’t willing to be treated in such a disgusting manner over trying to resolve matters which were necessary. Compared to some users who are easily pleased with what they have, the difference is in workflow and actual functional needs, why be happy with what you have and suffer loss of particular resources, when a developer should do everything possible to find ways around an issue to support disabled musicians, producers and engineers such as myself.
What is not described in the article, and from 2017 is the fact that there are still issues, the article is a Native Instruments biased article and not an independent consultation. Nothing was mentioned about system-wide accessibility to the application such as navigation of the screen reader cursoer, inability to access the settings / prefs dialogue to configure the UI, Audio, MIDI and other prefs, the fact that there is no means of adjusting speech parameters, etc or routing speech to another audio interface or channel, etc. There are issues we still are waiting on fixes on 4 years later.
A positive though with this product is that the UI physical control is nice and tactile, it is ideal within Logic Pro X for navigating tracks, editing and mixing, having access to most controls is ideal, but it is limited. the feel of the komplete kontrol S series is really nice and I mean that, but accessibility is purely reliant upon Komplete Kontrol Software and the system Speech UI without modifications and no means of screen reader navigation to configure anything.
So you understand, I’m not just a product user / customer, I’ve been in the Accessible Music tech scene as a developer, trainer, advisor, etc, I’ve owned a professional studio environment I’ve designed from the ground up to meet my needs, sadly closed in December 2017 after a family tragedy. These days I still keep my hand in but don’t own a studio anymore, just my mac, braille display, a decent master controller, the software and dev tools I need, etc. that’ll do.
lew
2 Likes
I can understand your logic there, however, Aeolus is expandable and voiceable beyond standards, thusly, a stop name or position is changed, assignment of function is changed. so the concept you raise would not work. the simple solution which I use in audio production is an ear piece for VoiceOver while relying on reference monitors for mixing, if I’m doing in-depth work and VoiceOver is on the go constantly, I use a pair of computer speakers for the screen reader output while production work is spread between sets of mix monitors. always have done this. What you’re examining is a complex matter which doesn’t need to be complex, re-inventing the wheel isn’t needed here lol trust me.
The trick of making an organ accessible isn’t always to do with the controls themselves, the actual tactile experience is there from positional memory. it’s to do with provision of information. so using aeolus on a mini pc, sending the audio for aeolus to the outputs of an interface, but using a screen reader’s speech output to the system audio only where then the headphone out of the computer would then provide an ear piece when needed is fine.
There is research showing, as I know personally, that with light, there is given heat radiation as a waste element. it’s like when you turn a light on in your room, if you’re sensitive enough to it, you feel the heat on your skin. the same can be achieved if you’re working with a system where various lights are on, if you run your fingers across, some can be felt with slight warmth, sometimes reliable, others not. the other tool in my adaptive resources kit is a light scanner. sometimes a great help. I also use an OrCam ReadSmart for reading print out to me on the go.
Trust me on this, as someone who’s been a serving organist with ssight loss, blind for over 21 years, I know this sector too well lol
lew
I had mentioned just creating an audible counterpart to the indicator lights under the buttons of some MIDI button matrix boxes, which provide just a few colors as simple state indicators. Using short cell phone like UI tones allow checking a row of buttons in a second or 2, without activating the buttons. It would require memorizing the the meaning of a few tones, but would provide the closest recreation of the sighted interface information provided by lights.
In the simplest case you could simply check if an organ stop assigned a button is currently on.
The actual button mapping may have to be memorized, like for a sighted user, perhaps with some sparse row column braille markings.
The hardware implementation of soft touch combined with momentary pushbutton with tactile feel is the big challenge. Perhaps a capacitive sensor grid molded into the top of a rubber button that passes light, or some more predictable alternative sensing like a reflected IR proximity transceiver for each switch.
1 Like
I understand where you’re coming from in a development sense, but I’d say this. Understand what a client would actually need in detail. this is where I come from. Audible tones are one thing, etc however, there are problems with that. what if you’re working with a console with over 100 stops as a good example, then you have couplers for each division, then pistons, etc. how are you going to process that degree of tonal information for short pulse / mid-pulse tones with pitches, especially while playing. that isn’t viable.
ok, say you have a console surface like a johannus ecclesia d570 as an example, without the motorised stops as an example, the stops are gently pulled but reset back as recoil, they have light (LED’s behind the faces of each stop with the engraved names., a system like that, the LED is not to surface, it is carried through a tube to spread light, therefore any temperature isn’t as obvious as that of placing a finger directly on an LED., so that becomes a useless function. If you were to assign tones / pulse tones to each division to a module, then you’d have to concentrate even more on what tone relates to each stop, especially if the stop changes (dynamic stops).
What you need to know is that Aeolus is a “Dynamic” instrument in the sense that it is a fully programmable instrument, the stop names can change, stop amounts can change, addition or subtraction of functions can change, sequencer modes are updated, so therefore speech output via text to speech is fundamentally necessary. Trust me on this as a long serving church organist who’s worked with pure consoles, hybrid and digitals, different surface designs, etc.
sit at an organ, with a laptop and a braille display connected, the braille display is either sat on the music rest above the top manual or to your side to quick reference, that’s just part of the sensory side of a blind organist.
now put yourself in the position where you have no sight, you’re a church organist, walking in to a church you’ve been called to play for, you don’t know the console. How do you know what stops are available, where the couplers are, what pistons are available, etc? how do you know what toe pistons are available and what they’re assigned to do? if the console is an electrified pure console, there’s no computer method, so you are either relying on feeling the layout, but mostly relying on a sighted person to read stops, pistons, etc for you to memorise.
If you were fortunate enough to use something like an OrCam ReadSmart or OrCam MyEye, which becomes your eyes to print, colour, etc for the MyEye series, then either device will read text providing it is legible and speak it to you. I have a ReadSmart and have tested it on stops, pistons, and it works reasonably well, takes a little time to train it at times. but once you have the initial layout understood and memorised, then you’re good.
You make sure that in certain circumstances way before you’re due to perform, that you’re comfortable and knowledgeable of what works, what doesn’t, listen to each stop just in case the stop might not be the right one for a particular hymn, psalm, grace, salutation, etc. then you’re ready to go.
in the case of a digital organ, as said before, compared to a pure console where you have electro-mechanical or hydraulic action stops and tracker action manuals with physical couplers you can feel assigned to certain manuals, etc, the tactile principle is obvious. Compare it to a modern digital console like a Johannus, Allen, Rodgers, etc where lit touch stops or touch tabs are fitted, they have an auto-return, they don’t raise or tilt lock. so then you have that added headache. a light scan tool which is a small pen that either can vibrate or speak (prototype) provides me with some detail, but then in conjunction with the Orcam Read Smart, allows me to ID a stop.
However, in the process of building a modern digital console based on Aeolus and the lack there-of of traditional draw stops or active tab stops, either of which use an electro-mechanical action over a power rail, then there are two logical ways around this. The first is a custom panel of momentary switches, where to the top corner outer of a momentary switch, a RAISED LED could be felt if on, bear in mind this doesn’t work for all blind people, works for some. The other way is to bypass this and use a touch screen environment.
now, both of these options require speech output to engage, but the tactile custom panel also requires midi programming to assign. A touchview display system helps sighted users directly, but if Aeolus through the Organnery distro, supports the embedding of a screen reader to allow multi-gesture touch, then engaging with each stop, sending audio to the system audio of the host computer to an ear-piece, rather than sending the spoken audio out of the group outs of a pro audio interface driving say, 18 speakers in a church or cathedral setting if a digital console were installed, then it works out better.
The concept of a speaking organ is perfectly viable and in respect of modern technology and what the Organnery project is undertaking, which is the recycling / upgrading of digital and hybrid organs, to feature digital technologies which are expandable and upgradeable to today’s standards, this system could thereby accomodate a blind or severely sight impaired organist. What if the host computer allowed a user like me to boot the same Aeolus setup with a unique config to suit my needs, it doesn’t harm the console at all and restores it back to previous state. something that’s being investigated.
I love this place, sharing ideas, concepts and demonstrating working principles with equally mad people
lew
Sharing with you some design concepts for tactile control of a console, based on Yaeltex, a developer of tactile surfaces. www.yaeltex.com
These are my own designs, thank god their website allows a user like me with a bit of access dev tools to design by description / voiceover feedback.
lew 
IDEAL SPEC Panel 2:
IDEAL SPEC Panel 1
Rationalised spec panel 2 as half size unit
Rationalised spec panel 1 full unit all stops merged to single frame
Regarding tactile feedback (like piezoelectric transducers)
It’s possible that each control could continuously output it’s ID signal in a simpler open loop scheme, it would probably produce an unacceptable racket.
I was reviewing braille output devices, which get unavoidably complicated, a simple alternative is a morse code output through a single transducer, it gets compared to braille in studies, there is an added temporal dimension, so reading tends to be slower.
(Morse code was a requirement for an Amateur radio license, this was eliminated for the lightweight “technician” class license)
Some references:
Feel-The-Code (A Haptic Morse Code Reader Project, 9 year old code link is dead)
Tactile Taps Teach Rhythmic Text Entry: Passive Haptic Learning of Morse Code
Perhaps the most interesting aspect is their using the bone transducer in Google Glass to transmit the signal. Separating this information from you normal hearing channels.
A study of information absorption efficiency: Tactual recognition of embossed Morse code, letters, and braille
Good Vibes enables the deaf-blind to communicate with each other through Morse code (App) Using a cell phone vibrator to bridge communication barriers.
Braille and Morse Code both have 6 positions, but Morse code can have a dot or a dash (pronounced di da) the space between the variable length characters in More Code also takes another 1 or 2 additional time slots. (Deciphering morse code information while composing a rhythm track could be maddening)
The faithful reproduction of an organ stop array seems unnecessarily cumbersome. (although some people memories work better with a fixed place for everything, rather than a compact file system.) Zynthian reduces complicated wired sets of signal generators and processors to a patch number.
Imagine as you move a stylus over an imaginary grid area, you get a display or a more code number for the current cell you are over. This could be transmitting to the moving finger or a fixed transducer for the second hand.
By the way, I did run across some references to one study using those expensive hand sized haptic transducers for transmitting braille. Here . . . and Here.
Can I ask you a perfectly reasonable and decent question here… Are you yourself registered blind? do you have particular specialist qualifications in sight loss rehabilitation? are you a trainer in braille / music braille? If Neither of these apply, then why state the above comment? I am a braille user, I use a braille display, Do you know how much they cost? a brailliant BI40 as an example which is bluetooth and USB supported is nearly £2500 exc vat. that tool is my essential tool for proofing tech documents, music scores, etc because I don’t have a braille embosser her to print with. So you’re also aware, I rely on a braille display to proof read because it’s sharper than embossed paper, Due to a stroke some years ago my touch response isn’t as sharp as it once was, can’t help that but I accept it and get on with it.
I realise you’re coming up with some good responses here and I like where you’re going, but you are missing some of the concepts here. You’re also saying that designing an organ to “faithfully recreate stops” no. If you yourself were a church organist sat at a console, you’d realise the functions present of a console, you’d realise exactly what’s needed and how to interact with the instrument. If you were to examine a draw stop console, you’d notice left and right jambs (english spec), Terraced left and right design (German / French) or Tab design over swell manual (Continental / modern european / american design), under each manual are assignable pistons and couplers, unless couplers are included to the draw stops of particular jamb designs.
I’d encourage you to experience what it is like to see nothing but black and gain a knowledge of the AVAILABLE accessibility technology tools from developers like Apple, Freedom Scientific, GW Micro, Humanware amongst others, rather than trying to create solutions which increase stress factors.
Please don’t take this the wrong way and I apologise in advance to anyone reading this if it sounds that way, But, having owned my own recording studio designed for accessibility in music production, as a trainer, tutor, advisor and former developer, a performing musician and composer who’s been fully blind for over 21 years, I am more than qualified personally and professionally to make assessments of system integration and builds.
I am in the process of a build where initially for my own use, but also a potential “to market” Organ System for the Blind. for organists studying at home or even working at churches / cathedrals. I am aware of the various multi-sensory input / output tools available, but please trust me in my field of expertise when I say that engaging the use of a screen reader to support Aeolus / Organnery Distro for not only stop / piston engagement, etc but also Voiceing, I know where-of I speak. Having spent many years, with others, trying to engage with Milan Digital Audio (makers of Hauptwerk) and losing that battle. I now am working on something much more viable.
Please understand where I am coming from. I respect your comments and statements, however, I made my position clear in my original thread.
All the best.
Lew 
Folks, I do most sincerely apologise for my above response.
Perhaps as someone rightly cooented in the original aeolus thread, that my particular issue did not belong here. I came across this forum, not knowing that zynthian was a hardware project, but had considered this was a porting of and could find particular answers / raise some ideas.
My apologies for taking up time and thread space with the matters of accessibility in hardware / software development for sight loss in Music tech.
lew
1 Like
Hi @lewisalexander2020 !
Welcome to our community and thanks a lot for your contribution. Really!
I’ve been working a little bit in a “proof of concept”, and i’m not the only one 
Currently, I have over my desk a zynthian box that speaks quite decently, telling the menu title and the menu entries you are navigating. And it uses the RBPi audio output, while keeping the true HQ audio output free of annoying voices. It needs some extra adjustments, but it’s not far from something usable.
Perhaps there is a fellow zynthianer living near from you that could bring you the opportunity of tenting it and give us your precious (and exigent!) feedback.
Kind Regards,
Ah ha, there is indeed hope.
I would be happy to take on a unit as a test / development unit (which is where I’d be flying you an email privately shortly) to discuss it further.
One of the concerns raised to me is that this is in kit form, now to me, that isn’t too much of a concern as I have prior technical electronics background, most other blind users don’t have that background in installation / repair, etc and I’m stating it as fact. So as a unit, if I were to recommend it to sight impaired / blind users, the system would have to be assembed before dispatch and an option in the online order process that if purchased for a sight impaired / blind user, that the screen reader service be available.
As a build, if running a screen reader, I’d advise using the pi 4B running at 8gb ram and this is to ensure that there is enough memory address support for the speech engine while other processes are engaged.
One of my observations is the lack of a main volume control, based on the description of specifications. Could one of the encoders when pressed whilst rotated, be assigned as an output gain? The other query being the lack of a power button. I understand that the pi is USB-C powered and as such powers as soon as physically connected. that in itself does put a potential risk factor to the internal sdhc / sdxc card as the storage media as it is volatile in terms of voltage spikes as an example. Yes, a pi can be replaced, but the cost of replacing that pi is a concern to some.
sorry, devil’s advocate lol
I’m happy to provide what support I can, it’s what I do
lew
I’d read the forum from your position , you will find discussion of a very diverse set of issues and many complete solutions and dangling threads of consciousness in various stages of completeness and description.
The status bar
https://wiki.zynthian.org/index.php/Zynthian_UI_Users_Guide#The_Status_Area
contains a lot of stuff worth knowing in our world, like the Xrun and wether or not you are recording, so how these essential low level actions that need flagging might be transmitted in a non visual way? Would that be request a status or could a event of some kind be repeated?
I’ve found myself with the recorder on and not noticed when running without a gui of any kind. . .
1 Like
the status bar would need to be accessed. now I understand it’s a touch screen environment right? how agile is the screen to an average finger, or is this stylus dependent? if it could be navigated by finger, then a screen reader would announce such status elements.
forgive the question, but what is XRun? and I wasn’t aware this could record.
I understand that the latest build has 4 touch buttons. could they be programmed / assigned as part of the sequencer? if so, then surely on a visual note, record would show active as RED to a touch button as a physical button. from a spoken output side then yes, the status should give feedback of a record arm or record state, xRun, etc.
Sorry regarding the multi-access feedback. I can understand that there are other ways of providing feedback, but in today’s mainstream accessibility supply, the common 2 functions are braille output and spoken audio output, both from a screen reader. going in to the realms of haptics, vibra-sense and other resources just means having to train a user in a way that isn’t really beneficial unless a unser has that particular multi-sensory requirement. this becomes the realm of specialist accessibility / assistive technology development which is costly and rather limited in it’s range of supply.
lew
The Zynthian is more like a hardware synth than a computer, even though (like many modern devices) it has a general purpose computer at its heart. The proposals in the thread vary from the practical through the pragmatic to the weird and possibly wonderful. It is always worth thinking outside the box as a better solution is only found by trying something different from the norm. Of course a user with such extensive experience as yourself is invaluble in honing those ideas. The approach that @jofemodo and I are taking to validate some proof of concept ideas are based on the current user interface within Zynthian. A generic screen reader doesn’t necessary fit this device because it is not normally driven by a mouse or keyboard.
The official touchscreen is a 3.5 inch resistive screen which is not particularly responsive to touch and has occasional errors. It is possible to build a bespoke Zynthian with larger capacitive screen which is more responsive to touch. The default screen is not a good fit for dynamic screen reading, i.e. you can’t move a pointer over it to readout its content. More often than not such activity will trigger an action.
Xrun is a failure of the system to write or read audio data within the allocated time. It occurs when modules take longer than they are allowed to read, write or process audio. Xruns manifest as clicks in the audio and are very undesirable. They can indicate that the processor is too busy, e.g. too much audio processing by engines. They can also indicate issues with transfer of audio, e.g. to or from a soundcard.
The Zynthian user guide is at Zynthian UI Users Guide - ZynthianWiki. It does have quite a few images and is unlikely to have been proof read for accessibility but it does describe many of the features and how to use them.
It sounds like you have a good thing going with Organnery which might florish into something well suited to you. You have inspired us to improve the accessibility of Zynthian and I expect we will soon have something that a visually impaired user can operate as well as a sighted user using the main user interface. This work may not be a complete solution and there will always be elements that need improvement but it feels quite feasible that we can add good accessible features to the Zynthian.
1 Like
Nice one.
Here’s a challenge for you. do any of you have an iphone? sorry but it’s the only way I can demonstrate it…I’m referring to your comment about keyboard and mouse, it’s not needed as a screen reader providing the interface supports touch, can actually speak what’s placed under the finger when pressed once, a good example would be this…
with an iphone, VoiceOver runs automatically when set up, so my iphone is instantly ready to use. I touch the screen once and depending on where I place my finger, it will announce what the item is. depending on how the voiceover setup is configured, it will be as verbose as you want, or in my case, to the point. so, it’s told me what the item is, I can double tap to engage it, like a double click, but it replaces the standard layer of a sighted use of the iphone’s basic UI which is just a single touch or gesture. The gesture layer and touch layer changes for VoiceOver so that certain functions are easier to use and differ, you can access a function rotor amongst many other things.
so in the concept of the screen reader, the touch interface would be using a touch layer supported by the screen reader.
I do admit, I like your idea of a larger working display area. regardless of my needs, think of users who want more on their screen realestate, those who own workstations like the korg kronos and other touch view instruments, they tend to use something like a 7” display.
Thank you so much for your kindness. as I say happy to help and sorry if I put my boot in regarding other adaptations. you have to think remits of accessibility and general use requirements of adaptation, I did this as part of my masters dissertation and produced a 116 page research document with various case studies, technical and psychological implications, etc. that paper drove me up the wall. My main dissertation was nothing to do with that, it was to do with interpretations of choral works for organ and going against the grain of traditional composition.
lew
The touch interface on Zynthian is an afterthought or second rate citizen. I can be challenging to perform all actions from the touch screen. There is no window manager and most available screen readers would struggle to work with Zynthian or not work at all. Purpose written accessibility would be faster and more effective to implement than trying to bolt on an off the shelf screen reader. You are absolutely right that blind and visually impaired users have learned how to use the available tools on the devices they use whether that be screen readers on phones, tablets and computers or screen zoom or speech recognition, etc. Zynthian is not an Android or iPhone and is not a Windows or Mac OS computer. It does not use many of the technologies that those system’s accessibility modules connect to. We should definitely not try to reinvent the wheel but we should also not try to fit a square peg into a round hole.The work that Kurzweil did with his keyboards is probably similar to what we can achieve in a fairly short period. Once we have something working I will run a blind test to see how effective I find it.