Questions about zynthian OS automated build system

Hi there,
Can someone explain me how do the automated build system work ?

  • what are base files that are used for that (github repo from github.com/zynthian ?)
  • is raspbian buster minimal image the startpoint ?
  • is the build done on a cloud instance ?
  • how to automate that ?

Thank you.

My aim is to port zynthian on another platforms, Allwinner H5 (nanopi K1plus), and rockchip rk3399 (nanopi m4v2).
See here: https://forum.armbian.com/topic/9009-info-friendlyarm-pcm5102a-hat-with-nanopi-neo-under-mainline-4xx-and-dev-533/?do=findComment&comment=91059
And even more, but it’s like an “infinity and beyond” dream : https://www.youtube.com/watch?v=yejqDshtyLA

3 Likes

You should take a look to these script:

This is the main setup script that runs on a raw rasbian buster-lite image to create the ZynthianOS image …

Regarding the nightly build system, it uses CustomPiOS, that uses qemu for creating an ARM Virtual Machine, boot with the buster image, run the setup script, etc. It’s something complex, but you can download everything, including some docker image.

Regards!

Thank you.
I’ve already tried last year to adapt the setup script for building zynthian on the nanopiK1plus. But that takes hours, and unfortunately result was Always broken.
That’s why I asked how you are proceeding for building zynthian on a virtual machine.

, but you can download everything, including some docker image.

Where to find that ?
https://github.com/zynthian/ZynthianOS/ ?

The forum has a good search tool:

:wink:

:cold_face: I have forgot that feature hahaha
Thank you @jofemodo, I will investigate that when I’ll be back home (after 15 days of containment at my girlfriend’s home -frankly not that bad :smiley:- but far from my SBC and my musical gear).

2 Likes

Hey @le51 I am the creator and maintainer of CustomPiOS which is the system used to build Zynthian’s OS (we call those scripts ZynthianOS).

CustomPiOS supports changing the base image that the build is based off which was intentionally done so you can support other SBCs.
I haven’t had much cooperation maintaining other devices and would welcome and help if I can getting other machines to support Zynthian and other distro builds.

Also I see you mentioned armbian, there is an armbian build variant. But it needs more love and caring. There is an “ubuntu base image” switch too, which I started adding to the devel branch of CustomPiOS.

  • Zynthian uses ZynthianOS: https://github.com/zynthian/ZynthianOS
  • Yes but you could adapt it to any base image, we use rasbian because it has non-free drivers we want to have available.
  • Yes, we use Jenkins that executes a docker instance of CustomPiOS
  • Yes, there is a nightly build. Best method is to have Jenkins that can run docker commands, from there just follow the instructions. I can throw in our jenkins shell script:
#!/bin/dash

cd /home/guysoft

  sudo ./docker_rebuild.sh
  sudo ./build_zynthianos.sh

docker_rebuild.sh

#!/bin/bash
set -x
pushd /home/guysoft/ZynthianOS/src
  docker pull guysoft/custompios:devel
  docker-compose stop
  docker-compose rm -f
  docker-compose up -d
popd

build_zynthianos

#!/bin/bash -x

# Clean Jenkins log => Sometimes it grows too much ...
rm -rf /var/log/jenkins/*

if [ -z ${BUILDS_DIR} ]; then
        export BUILDS_DIR="/home/guysoft/builds"
fi

if [ -z ${ZYNTHIAN_FORCE_RBPI_VERSION} ]; then
        export ZYNTHIAN_FORCE_RBPI_VERSION="Raspberry Pi 3 Model B Plus"
fi

pushd /home/guysoft/ZynthianOS

    # Store Start Timestamp
    START_TIMESTAMP=$(date --rfc-3339=date)

    # Copy soundfonts to CustomPiOS filesystem
    rsync -rlv --size-only /home/txino/soundfonts ./src/modules/zynthianos/filesystem
    
    # Delete README files from filesystem
    rm -f ./src/modules/zynthianos/filesystem/*/README
    rm -f ./src/modules/zynthianos/filesystem/home/*/README

    # Create build info file with useful info about the build ...
    build_info_fpath=./src/modules/zynthianos/filesystem/home/pi/build_info.txt
    echo -e "ZynthianOS: Built on os.zynthian.org\n" > $build_info_fpath
    echo -e "Timestamp: $START_TIMESTAMP\n" >> $build_info_fpath
    echo -e "Optimized: $ZYNTHIAN_FORCE_RBPI_VERSION\n" >> $build_info_fpath  

    # Build ZynthianOS
    sudo /usr/local/bin/zynthianos
    
    # Post-build proccess
    pushd ./src/workspace

        # Copy Soundfonts to SD image
        #losetup 
        #cp -a /home/txino/soundfonts/sfz/* ./mount/zynthian/zynthian-data/soundfonts/sfz

        # Get ZIP filename    
        ZIPNAME=$(ls *.zip | head -n 1)
    
        if [ ${ZIPNAME} ]; then
            # Calculate new filename
            NEW_ZIPNAME=${START_TIMESTAMP}${ZIPNAME:10:100}
   
            # Rename zip file
            mv ${ZIPNAME} ${NEW_ZIPNAME}
    
            # Rename img file inside zip file
            IMGNAME=${ZIPNAME:0:-3}img
            NEW_IMGNAME=${NEW_ZIPNAME:0:-3}img
            sed -i "s/${IMGNAME}/${NEW_IMGNAME}/g" ${NEW_ZIPNAME}

            # Calculate MD5-sum file
            md5sum ${NEW_ZIPNAME} > ${NEW_ZIPNAME}.md5

            # Move zip & MD5-sum files to "builds" directory
            mv -v ${NEW_ZIPNAME} ${BUILDS_DIR}
            mv -v ${NEW_ZIPNAME}.md5 ${BUILDS_DIR}
        fi

    popd

popd

Hi Guy, thank you for your input.

I’m now back at my home.

Firstly, I will try to build zynthian step by step on the nanopi m4v2 itself (wich is much more powerfull than the nanoPiK1plus, with its Nvme HD and 4 Go ram), starting from the armbian server image and using the script

setup_system_rbpi_raspbian_lite_buster.sh

Then, if I’am successfull with this (<=> having an armbian adapted setup_system script), I’ll go further in automation (as I’m pretty new with these docker/jenkins stuff).

Sorry to revive this thread, but have you got anywhere?
I am also thinking on building zynthian over armbian for H3 (Orange Pi PC plus).
Are there any Raspbian-specific dependencies that would otherwise be a bottleneck in building zynthian for H3?

I think this isn’t a very straightforward process, but are there any hints/tips for porting zynthian to other platforms?

Hi @xanderz ,

it’s not an easy task. A full ZynthianOs is a bunch of Gb of a “working” software stack (raspi base system wich is based on debian, with the help of kxstudio and maybe other deb repo and of course all the apps that are build from source -see here- with everything encapsulated in Zynthian sofware/UI/hardware).
Maybe this could help:

Got it.
But is there anyone around that tried running zynthian on H3 with 1GB of RAM(or at least, similar hardware)?
I honestly am afraid of spending lots of hours in porting zynthian and then finding out that the hardware is crap and has lots of lag.
I would have bought a RPi 3/4 if I can damn find at least one in any store in Romania(chip shortage issue maybe?).
Digikey, Mouser and such have tens of weeks lead time…

@xanderz you will have hours of toil to port Zynthian to any other platform with no guarantee it will work properly, if at all. I would not even attempt this unless I had the device and lots of time. 1 GB RAM will be a limiting factor. It can really slow down compilation of code. Chip shortage is a real pain affecting almost all technology delivery but if you get a chance, try to get a Raspberry Pi 4 (2GB or larger). You can sometimes find vendors selling their own kits with larger RAM units, PSU, etc. for a good deal. Keep looking my friend :slight_smile:.

I agree with @riban
It took me hours (days ?) to build Zynthian on a H5 board equiped with 2Gb ram and 32Gb of emmc drive.
I achieved to make the dac to work (with pcm5102 chip) and the UI to get up. But most of the software instruments where buggy (100% of cpu load, interface freezing, application crash and so on).

The very first thing you should test is the audio ability.
Start with whatever debian based os works for the board. add the kxstudio repos and install something like setBfree or Fluidsynth. Get Jackd running with whatever alsa device is reported, plug it all together with qjackctl, patchage or something and give it a quick run with a usb midi keyboard. I’ve done this sort.of thing on the pinephone or pinebook pro and it will give you a good view of how it will manage with audio output capability, or xruns and stuff.

IF it works really well then try git cloning all the zynthian stuff on top of the build and see what runs. I also did that with Fedora on a laptop before I got a zynthian kit. It’s terrible but.can get you off the ground.

@riban There are very few resellers, and they are selling really overpriced raspberry pi4s for which money I can easily buy a used Thinkpad with an i5/i7 that can easily run any vst.
But this defeats the whole purpose of having a fully-embedded, small form factor synthesizer.

@Baggypants thanks for the tips, will try that today and see how it goes. Maybe also patch the kernel to use the RT patch and see if it helps in any way.

I’m planning to get the most barebone armbian image and work from there.
I think I will scrap the on-board compilation idea and go with docker as I have more than decent computer to handle the build

Realtime kernel is not advantageous and generally gives worse performance. Low latency kernel is important.

@Baggypants so I just installed fluidsynth and got a relatively lightweight soundfont(~12MB).
I am really impressed with the performance. The lag is quite small(I can notice it, but it’s very small indeed).
Owning a Korg PA1X they are very close in terms of responsiveness.
No xruns displayed, but I might want to load it a bit more with a beefier soundfont.

Excellent news! Mess with the jack settings to get lower latency with potentially more xruns. There will be a sweet spot somewhere.

For a beefier synth try ob-xd and one of the massive pads. You will need to launch it using jalv. Beyond that there’s Surge synth.

Ok, so I went headfirst and try building zynthian on my orange pi. Removed some lines from the install scripts which were raspberry-specific(wiring being one of them).
Script ran fine, but did not manage to see any ouput because I left it running and when I came back to it, it already finished and closed the screen made when I launched the setup script. So I guess, if there are no logs saved, I can’t know what went wrong.
Anyways, rebooted the board, and hdmi displayed the armbian boot sequence then an error about /sda/fdb1 or something like that. I guess, for the moment, since I don’t have a SPI screen on-hand, I will try to use HDMI (found out on the forum that it is possible to do so).
I wanted to change the hardware settings using the webconf tool, but it doesn’t work. Tried manually launching it with the script and I get all sorts of errors of missing modules in python. So I’m manually installing them, but this is clear that the setup script failed one way or another.
Nevertheless, here’s a snip over ssh. Looks like some parts of zynthian are actually configured:

1 Like