Buildroot for Zynthian?


#1

Hi Developers,

I currently have to do some “simple” tasks for my work with buildroot. Maybe this is also a way to produce a Zynthian-Base-System instead of using Minibian. I think to create everything with buildroot should not be the option.

What do you think - just for the next years?

Regards, Holger


#2

what about dietpi?

they support a lot of architectures.


#3

Hi @C0d3man!

You know i would love to see an optimized Zynthian image. Currently i can’t put my focus on that because i suspect that it would take too much time, but if you can go forward with it, slowly but firmly, it would be great!! Let’s do it! I will help you as much as i can …

In relation with this subject, i’ve received a nice proposal a few days ago. Take a look to this thread:

https://github.com/zynthian/zynthian-sys/issues/25#issuecomment-362435949

Kind Regards,


#4

Hi Fernando,

that sounds really good! I think I will try a little bit with buildroot - but I think that it will be a hard way to get it suitable for Zynthian. One example for problems: python is normaly build with a disabled tkinter, you have to enable it by yourself (it seems there is no configuration option) and you have to check that all other needed packages are built, too.

We will see, if I will get this running sometimes.

I will checkout CustomPIOS. This sounds really interesting.

Regards, Holger


#5

Yes, building a working image completely from scratch would be a hard work, although it would be the best option.

In the other hand, having an automated environment for building and testing raspbian-based images would be simpler and more affordable. We have a good starting point with the current “setup script”, so perhaps we should concentrate our energy in going forward with that.

I would configure a dedicated server for that and we could generate daily/weekly images. For me, this would be a dream!! :star_struck::star_struck:

Regards,


#6

Hey guys,
Developer of CustomPiOS here.
So I have a ZynthianOS repo on github that builds an .img file
which builds an img file with Zynthian on it using a realtime kernel.

It works and takes ~12 hours to build, I was going to upload a build.log but stupidly deleted it when rebuilding, so I will be able to upload a new one hopefully tomorrow.
But it does build and its predictable. I wrote the build instructions pretty quickly too. I can set up my server to build it, or use one you provide.


#7

Hi @guysoft!

Thanks a lot for your help with this. I’m impatient for testing the new stretch image :star_struck:

What would be the minimum machine you recommend for building the images?
Currently i have a Jessie VM with (1 core, 2 GB RAM, 77 GB of free disk space) that i use as developing environment for the shop, so it’s basically idle. If needed i could increase the num of cores, the memory or add extra disk space.

Anyway, i’m trying to make a build by myself and get my own conclusions :wink:

Kind Regards,


#8

Not a lot. The faster CPU the better. You might want virtualization capabilities so you can use vagrant.


#9

OK! It’s building now. Initially without Vagrant … i never used before and i’ve to study a little bit before using it :wink:

Regards!


#10

@guysoft,

I see these lines in the “start_chroot_script”:

apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y jackd
# Enable realtime permissions (taken from /etc/security/limits.d/audio.conf.disabled )
dpkg-reconfigure -p high jackd

Should be moved to the zynthian setup script itself?

Regards,


#11
  1. There was some interactive dialog there which I wanted to avoid with that.
  2. Note in order to develop it here I am using in config:
    https://github.com/guysoft/ZynthianOS/blob/devel/src/modules/zynthianos/config
[ -n "$ZYNTHIANOS_SYS_REPO_SHIP" ] || ZYNTHIANOS_SYS_REPO_SHIP=https://github.com/guysoft/zynthian-sys.git
[ -n "$ZYNTHIANOS_SYS_REPO_BRANCH" ] || ZYNTHIANOS_SYS_REPO_BRANCH=realtimepi-test

It uses my fork at the moment because the zynthian fork because of rpi-update being used, which makes the os panic, I think because of the relatime kernel:

diff --git a/scripts/setup_system_rbpi_raspbian_lite_stretch.sh b/scripts/setup_system_rbpi_raspbian_lite_stretch.sh
index 47a5082..172eb2a 100755
--- a/scripts/setup_system_rbpi_raspbian_lite_stretch.sh
+++ b/scripts/setup_system_rbpi_raspbian_lite_stretch.sh
@@ -35,14 +35,11 @@ apt-get -y dist-upgrade
 
 # Install required dependencies if needed
 apt-get -y install apt-utils
-apt-get -y install sudo apt-transport-https software-properties-common rpi-update htpdate parted
+apt-get -y install sudo apt-transport-https software-properties-common htpdate parted
 
 # Adjust System Date/Time
 htpdate 0.europe.pool.ntp.org
 
-# Update Firmware
-rpi-update
-
 #------------------------------------------------
 # Add Repositories
 #------------------------------------------------
@@ -122,7 +119,7 @@ git clone -b master https://github.com/zynthian/zynthian-ui.git
 
 # Zynthian System Scripts and Config files
 cd $ZYNTHIAN_DIR
-git clone -b master https://github.com/zynthian/zynthian-sys.git
+git clone -b realtimepi-test https://github.com/guysoft/zynthian-sys.git
 
 # Zynthian Data
 cd $ZYNTHIAN_DIR

You can create a config.local file in the distro folder and put in it:

ZYNTHIANOS_SYS_REPO_SHIP=https://github.com/zynthian/zynthian-sys.git
ZYNTHIANOS_SYS_REPO_BRANCH=master

#12

Hey,
Installed ZynthianOS on my Jenkins server, you can watch the build in action here:
http://gnethomelinux.com:8080/job/ZynthianOS/2/console

I can build this setup on your server. So you can make nightly/weekly builds.


#13

OK! Thanks a lot for your work, @guysoft! This is wonderful. I would love to have time to dive into these amazing tools like Jenkins, Vagrant, Docker, etc. I’m really impressed with the possibilities … but let’s go step by step :wink:

Currently i’m doing my own build for understanding a little bit of CustomPiOS and checking the fixes i’ve done to the setup scripts. When i have a working image (i need to test it in a real ZynthianBox!), we should automate the copying of default data (soundfonts, plugins, presets, etc. => about 10GB of data!) for having an image ready to share.

When i reach this point, i will be ready for the next step and your magic tools … Vagrant, Jenkins and so on … meanwhile i will read some docs trying to understand the basics.

Regards … and thanks a lot!!


#14

The process blocked again in the same point:

Submodule 'DPF' (git://github.com/DISTRHO/DPF) registered for path 'build/DPF'
Submodule 'instruments' (git://git.code.sf.net/p/zynaddsubfx/instruments) registered for path 'build/instruments'
Submodule 'rtosc' (https://github.com/fundamental/rtosc) registered for path 'build/rtosc'
Cloning into '/home/pi/zynthian-sw/zynaddsubfx/DPF'...

Some tip?


#15

If I can help let me know. I can clean the stretch build and wget errors in the meantime. Its easy to spot now that I can create builds and parse the output with a single click. I can test it nightly and incrementally fix stuff.

Yes, I had that problem before, using vagrant build method solved it. For me it manifested in Ubuntu which should have been equivalent to Jessie at the time.
Suggest you use vagrant. It will keep the environment the same. No surprises.

BTW the build finished here, and you can immediately see in the parsed console output the errors. Out of 30,000 lines it spotted 5 errors (I am not parsing the compilation errors yet, it found python and 404 errors):
http://gnethomelinux.com:8080/job/ZynthianOS/2/parsed_console/
I can fix the python stuff in my repo.


#16
Thanks a lot for your help with this. I’m impatient for testing the new stretch image

Ok, I have an image you can test.
There is a known bug with the host name I fixed and I am building a new image with it fixed, however, it will take another 12+ hours to build and upload so suggest you use the workaround for now.

Thinks that I found that dont work are:

  1. Zynthian-webconf password
  2. Mod-ui (see bug).
  3. The gui boots black - is that expected?
  4. Looks like pitft compilation fails.
  5. There are two build errors in the console that you can see

So image (suggest you use a download accelerator, hosted by http://docstech.net/, who hosts my FullPageOS images):
http://unofficialpi.org/Distros/ZynthianOS/nightly/2018-02-09_2017-11-29-realtimepi-stretch-lite-0.2-0.1.zip
md5:
http://unofficialpi.org/Distros/ZynthianOS/nightly/2018-02-09_2017-11-29-realtimepi-stretch-lite-0.2-0.1.zip.md5

Build log:
http://gnethomelinux.com:8080/job/ZynthianOS/8/consoleFull

There is still a few things to fix, but once they are people could actually use this.
Enjoy,
Guy


#17

Could you test this in a console?
sudo getent shadow root

if you could systemctl stop webconf and start /zynthian/zynthian-webconf/zynthian_webconf.sh (in sudo context), we see the error messages when you try to login


#18

Here is what I get
root@zynthianos:/home/pi/zynthian-webconf# sudo getent shadow root
root:*:17499:0:99999:7:::

And output when running zynthian-webconf and using password “raspberry”:
root@zynthianos:/home/pi/zynthian-webconf# ZYNTHIAN_DIR=/zynthian ZYNTHIAN_CONFIG_DIR=/zynthian/config HOME=/root /zynthian/zynthian-webconf/zynthian_webconf.sh
/zynthian/config/zynthian_envars.sh: line 78: warning: command substitution: ignored null byte in input
/zynthian/config/zynthian_envars.sh: line 78: warning: command substitution: ignored null byte in input
DEBUG:root:CONFIG VAR: ZYNTHAIN_SETUP_APT_CLEAN=TRUE
DEBUG:root:CONFIG VAR: SOUNDCARD_CONFIG=dtoverlay=hifiberry-dacplus
DEBUG:root:CONFIG VAR: ZYNTHIAN_PLUGINS_DIR=/zynthian/zynthian-plugins
DEBUG:root:CONFIG VAR: ZYNTHIAN_WIRING_LAYOUT=PROTOTYPE-4
DEBUG:root:CONFIG VAR: ZYNTHIAN_TOUCHOSC=1
DEBUG:root:CONFIG VAR: ZYNTHIAN_MY_DATA_DIR=/zynthian/zynthian-my-data
DEBUG:root:CONFIG VAR: RBPI_VERSION=Raspberry Pi 3 Model B Rev 1.2
DEBUG:root:CONFIG VAR: CFLAGS=-mcpu=cortex-a53 -mfpu=neon-fp-armv8 -mneon-for-64bits -mfloat-abi=hard -mvectorize-with-neon-quad
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_FONT_SIZE=10
DEBUG:root:CONFIG VAR: ZYNTHIAN_DATA_DIR=/zynthian/zynthian-data
DEBUG:root:CONFIG VAR: ZYNTHIAN_SW_DIR=/zynthian/zynthian-sw
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_COLOR_BG=#000000
DEBUG:root:CONFIG VAR: ZYNTHIAN_SYS_DIR=/zynthian/zynthian-sys
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_ENABLE_CURSOR=0
DEBUG:root:CONFIG VAR: CXXFLAGS=-mcpu=cortex-a53 -mfpu=neon-fp-armv8 -mneon-for-64bits -mfloat-abi=hard -mvectorize-with-neon-quad
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_COLOR_PANEL_BG=#3a424d
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_FONT_FAMILY=Audiowide
DEBUG:root:CONFIG VAR: JACKD_OPTIONS=-P 70 -t 2000 -s -d alsa -d hw:0 -r 44100 -p 256 -n 2 -X raw
DEBUG:root:CONFIG VAR: DISPLAY_WIDTH=
DEBUG:root:CONFIG VAR: ZYNTHIAN_MY_PLUGINS_DIR=/zynthian/zynthian-my-plugins
DEBUG:root:CONFIG VAR: ZYNTHIAN_AUBIONOTES_OPTIONS=-O complex -t 0.5 -s -88 -p yinfft -l 0.5
DEBUG:root:CONFIG VAR: FRAMEBUFFER=/dev/fb1
DEBUG:root:CONFIG VAR: DISPLAY_CONFIG=dtoverlay=pitft28-resistive,rotate=90,speed=32000000,fps=20
DEBUG:root:CONFIG VAR: ZYNTHIAN_CONFIG_DIR=/zynthian/config
DEBUG:root:CONFIG VAR: SOUNDCARD_NAME=HifiBerry DAC+
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_COLOR_ON=#ff0000
DEBUG:root:CONFIG VAR: ZYNTHIAN_DIR=/zynthian
DEBUG:root:CONFIG VAR: LV2_PATH=/zynthian/zynthian-plugins/lv2:/zynthian/zynthian-my-plugins/lv2:/zynthian/zynthian-my-data/presets/lv2:/zynthian/zynthian-my-data/presets/lv2
DEBUG:root:CONFIG VAR: ZYNTHIAN_RECIPE_DIR=/zynthian/zynthian-sys/scripts/recipes
DEBUG:root:CONFIG VAR: DISPLAY_HEIGHT=
DEBUG:root:CONFIG VAR: ZYNTHIAN_AUBIONOTES=1
DEBUG:root:CONFIG VAR: ZYNTHIAN_WIRING_SWITCHES=
DEBUG:root:CONFIG VAR: ZYNTHIAN_SCRIPT_MIDI_PROFILE=/zynthian/zynthian-data/midi-profiles/default.sh
DEBUG:root:CONFIG VAR: MACHINE_HW_NAME=armv7l
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_DIR=/zynthian/zynthian-ui
DEBUG:root:CONFIG VAR: ZYNTHIAN_PLUGINS_SRC_DIR=/zynthian/zynthian-sw/plugins
DEBUG:root:CONFIG VAR: ZYNTHIAN_UI_COLOR_TX=#ffffff
DEBUG:root:CONFIG VAR: DISPLAY_NAME=PiTFT 2.8 Resistive
DEBUG:root:CONFIG VAR: ZYNTHIAN_WIRING_ENCODER_B=
DEBUG:root:CONFIG VAR: ZYNTHIAN_WIRING_ENCODER_A=
INFO:tornado.access:302 GET / (192.168.1.42) 58.10ms
INFO:tornado.access:200 GET /login?next=%2F (192.168.1.42) 34.37ms
INFO:tornado.access:200 POST /login?next=%2F (192.168.1.42) 16.67ms

BTW line 78 is

model=cat /sys/firmware/devicetree/base/model 2>/dev/null


#19

your root account doesn’t have a password?
Please set one and try again.

The default root password in our system is raspberry.

root@zynthian:~# cat /sys/firmware/devicetree/base/model 2>/dev/null
Raspberry Pi 3 Model B Rev 1.2root@zynthian:~#

what is the output in your system and if null, do you know the command to get the same output?


#20
  1. Ok, when I set a password with passwd it works and I can login. In Raspbian there is no password by default. I can make “zynthianos” (how I am calling the CustomPiOS build for now) set one. Guess that means another build soon.
  2. I also get the same output. it seems to give that error only when saving it in to a variable.