Neural Amp emulation plugin

This is amazing. This must be included in zynthian,

I have just installed it as per @riban instructions above, downloaded the one amp model from forum:
hbe L 2 esr 0.035.json (53.4 KB), copy to zynthian and set it inside AIDE-X using VNC-Engines link from webconf.

Here is one capture me playing with backing track. I was so exacted that I over bended few times notes - sorry for that. This is out of box amp model above with only added delay in chain. No EQ-ing at all. I did not change a single parameter of AIDE-X. This is strat with bridge single coil pickup. File is directly exported from zynthian.

This is the end of my hx-stomp. I never managed to get this only amp sound from it. There is no effects here (distortion, reverbs, choruses, etc etc). It is usign out of box AIDA-X speaker IR. Truly amazing ! And potential is huge. We can trully advertise zynthian as a proper Neural Amp Modeler.

3 Likes

That was rather reminiscent of Zappa. Nice demo. I will try to find time to investigate better integration. @jofemodo, how best to add to Zynthian official?

1 Like

I think that all controls are there apart from a way of selecting a different amp models or speaker IRs.

We could have a workflow were user would upload .json amp model file and .wav speaker IR file through web/config and expand web-config to classify uploaded .json files as amp model and .wav as speaker IR.

Finnally, it would very helpful if we can get amp input and output v-meter visible when amp is selected so that people can see if input is over clipping or too low. This is also available on aida-x plug-in ui.

Some initial investigation shows:

  • Audio input and output are mono
  • There are 10 controllable parameters presented to naitive GUI
    • On/Off
    • Input level
    • EQ Pre / Post
    • Peak/Bandpass (mid eq type)
    • Bass
    • Middle
    • Treble
    • Depth
    • Presence
    • Output Level
  • There are 21 controllable parameters presented to ZynUI - extra parameters:
    • Antialiasing level
    • Net bypass
    • EQ bypass
    • Bass frequency
    • Mid frequency
    • Mid Q (bandwidth)
    • Treble frequency
    • Cabinet simulation bypass
    • Param 1
    • Param 2
    • DC blocker
  • Adjustment of parameters in ZynUI does not update naitive GUI - reported upstream
  • Adjustment of parameters in naitive GUI does not update ZynUI - reported in issue tracker
  • LV2 has 3 output controls (monitoring data):
    • Model Input Size (Error | Snapshot | With param 1 | With param 2)
    • Input meter
    • Output meter
  • LV2 has input and output event ports (often presented as MIDI but does not seem to be here)
  • I can not see an immediately obvious way to programatically load models or impulse resonse
  • LV2 save preset will store the currently loaded model and IR, so we may be able to use load/save preset to access model & IR
  • LV2 presets are not offered in ZynUI - may need to adjust jalv config

I am still testing the logic but we may be able to implement a method of uploading a model+IR zip package that then appears as one or more presets. We should also be able to save presets - we would want to avoid saving new copies of the model + IR for each saved preset. That may require some plugin-specific code in the jalv engine.

2 Likes

Update:

Without any code changes I can save and load presets:

  • Bold press on chain to show chain menu
  • Select “AIDA-X” from the menu
  • Select “Save Preset”
  • Create a new bank and name it
  • Name the preset

You can then load your saved presets like any other preset, including the model and IR.

You can also upload and download presets in webconf like any others. Some users may have the aptitude to be able to download a preset, change its content and upload the resulting zip as a new preset which would be a way to get new models and IR into Zynthian. Of course it would be nicer if that were automated.

With a model loaded, the RPi5 is now using lots more CPU. (I thought it was too good to be true!). It peaks to 100% on all cores but seems to sit mostly around 75% on all cores without xruns. This probably depends on the model so we may wish to share / catalogue our experience with different models.

The input and output meter values can be polled by jalv using the monitors command so we may be able to build a GUI _widget_to display these levels.

The LV2 works with and without naitive GUI so it can be used on a Zynthian without enabling VNC.

The LV2 exposes more control than the naitive GUI but there are some parameters that I don’t understand, e.g. “Param 1” & “Param 2”.



image

I had to restart webconf service before the names of the presets and banks updated - looks like a webconf bug.

[Edit] Zynthian save process seems to only handle one IR wav but that might be the workflow I am using. We need more investigation here which may be done when figuring out how to create preset packages from existing IR/model files.

Note: I haven’t actually tried playing audio through any of these presets so can’t validate that what is shown on screen is actually happening. The lack of sync between LV2 control and naitive GUI means either nothing is happening or we can’s see what is happening. (The latter isn’t a big issue - just something to be aware of.)

1 Like

That is great @riban.

I did see these additional 11 parameters in many videos reviewing aida-x but i could not get into them using plugin UI. On mod audio UI there is a settings button in top right corner to open additional controls but on plugin UI there is no such button.

What about me downloading all free amp models from their forum and getting them already available in zynthian.

I will be careful for only loading free one. There are some that people just converted from proprietary models such NAM DSP.

That would be very helpful, thanks! The IR wav files look quite small (25KB) so it should be okay. It seems that all files end up in /zynthian/zynthian-my-data/presets/lv2/AIDA-X.presets.lv2/. There is a manifest that defines the presets and banks and then individual ttl files for each preset. The preset ttl files refer to the appropriate json (model) and wav (IR) files. (They could be placed elsewhere but Zynthian’s current implementation puts them all in the same place.)

Webconf download will provide a zip of the whole directory and I suspect the upload is similar - which may mean we can’t upload individual presets. (I would suggest this is a bug - we should be able to download and upload a preset and for webconf to merge the manifet.)

I think that having a set of liberal licenced models and IR would be a really good place to be. It would defer the need for modifications that allow user manipulation of model / IR.

Be careful with licensing. “Free” is an ambiguous word. We need models and IR that are licenced that matches the Zynthian mechanisms of distribution.

  • NETBYPASS is the same as disabling model in GUI.
  • CABSIMBYPASS is the same as disabling cabinet IR in GUI.

This probably explains why I saw such low CPU when I first tested. I would have clicked those subtle buttons in the GUI (to the left of the model / IR buttons).

[Edit] Maybe we could arrange the “factory presets” that we create from the online available models and IR into groups that use the same IR for each group and different models for each preset. This might allow us to provide access to any combination of IR + model.

Be aware that some online presets may just be different settings of the same IR + model. We probably don’t want / need many / any duplicates (unless there is significant benefit). I would expect a user (probably guitarist) to adjust thier settings to suit their rig / environment. (We are guitarits, ready to knob twilddle - not keyboardists, satisified with the DX7 factory defaults!!! :wink: )

Wow that takes me back, you know I instantly recognised it but couldn’t remember the name. I haven’t listened to his stuff for a long time.

I’ve been keen to try guitar related things with Zynthian, this looks like a promising addition to the kit.

3 Likes

Oops! Sorry - I didn’t recognise this rendition of “Always With Me, Always With You” by Joe Satriani despite owning “Surfing With The Alien” (though I rarely play it) and your rendition being spot on (except the one bend that you mentioned). Very nice playing mate!

I have knocked together a PoC widget view with input and output meters.

Yes, i realised the same too.

Therefore, my plan is to first get all good models and IRs from their forum and analyse them and store information about them. Once we fully understand them we can decide on classification. Have in mind that there are models of effects only, models of amps with speaker (in this case it also says what mic is used) or without speaker in which case we need IR. We also need to look at is there a free collection of IRs that we can include.

:grinning: without very recognisable intro it is not easy. I purchased this album when it was released. As a student at that time i could not replicate this sound having just nock off replica of les Paul and home built valve amp with no effects at all. I always admired Satch.

Ah, forgot to say. Aida-x inside zynthian is one of the best father day presents.

3 Likes

Hi @riban & @stojos !

Excellent work with this plugin, mates!!! For a long time I’ve been thinking of pushing for having this on zynthian but i have too many task with higher priority over my desk. Now you almost have everything working! It’s really great!! I think it’s a huge addition to zynthian collection because neural models reach a point where conventional algorithms can’t reach, and now, with the Pi5, we have this huge sound in zynthian! Let’s push for including this in Oram before the release!!

@riban, regarding the custom widget, i think i did some provision/tests for having half-height widgets, what would fit nicely for this case, so you can have the list of control screens in the top(or bottom) half, and the custom widget in the other half. I would take a look and try to remember how i did.

I will test all this ASAP, but first i need to do some PCB modifications and place an order for the V5.1 mainboards. Summertime is too close and in Spain, everything will be stopped until September!!

BTW, it would be nice to include some models & IRs with small preset collection, so users can enjoy the AIDA-X from minute 1.

Cheers!

2 Likes

Totally agree. Simply take care of not copying propietary stuff.

Regards,

1 Like

I was testing with AIDA-X and i also reached very high CPU-consumption, but i didn’t get perceptible clicks and pops. It seems that the algorithm is quite intelligent. Perhaps it adapts the processing to the available CPU resources. This should be tested.

Of course, you absolutely need a good heat dissipation because it’s going to increase the temperature. I saw the over-temperature warning while using the Pi5 with just the thermal block. I will repeat the tests tomorrow with the aluminum case attached to the thermal block. Anyway, i’m pretty sure the case will get quite warm for sure!!

AIDA-X seems to be the new performance killer for the zynthian@Pi5! Yeahhhh!

Cheers!

I forgot to say that the AIDA-X plugin is now included in Oram, so you can update and test!
Probably you will need to restart the zynthian UI to get the plugin to work!

Enjoy!

3 Likes

I use official raspberry pi 5 heatsink with fan. Fan is only on while starting. I can’t confirm if it is on or off while using AIDA-X because I play loud. I do not see any temperature issues.

I don’t have issue playing sequences using drum sound fonts and bass synths or playing backing tracks while playing through AIDA-X.

Aida-x is running on mod dwarf that is similar hardware to pi4 and there it uses around 70% cpu and you can still add demanding effects such as reverb. Therefore we should not have any performance issues on pi5.

I updated zynthian tonight and played with latest AIDA-X screen. Good news - controlling AIDA-X from zynthian screen is working. And yes, as @riban stated status is not update on plugin UI in VNC but that is not a big deal. Also, you are not supposed to play too much with controls - only to slightly adjust to your pickups. If you have high gain model you will not be able to adjust it to act as clean. Instead you should load appropriate model for the amp clean settings. Amp models often come with clean, crunchy and high gain.

In regards of CPU. It looks like whatever you do it is around 70% - I added delay, reverb and flanger and CPU is the same as without it. Sometime it quickly goes to over 90 but then it quickly comes back.

I have noticed if you quickly turn encoders sometime it takes time the whole UI to catch up. Everything get frozen - however it does not affect what is playing . Maybe this is due to high CPU.

This seems to confirm the hypothesis of adaptative performance that matches available CPU power.

Regards

setBfree + AIDA-X = ohhh yeahhhh!

Enjoy

1 Like