Building zynthian image now works with docker

guysoft · December 8, 2018, 2:57pm

Hey all,
Thought you might like to know building the zynthian image has just become much more simpler, and can be done on windows, mac and linux, using docker.

The steps to build using docker are:

This is as easy now as:

Install Docker
Install docker-compose
git clone https://github.com/guysoft/ZynthianOS.git
cd ZynthianOS/src
wget https://raw.githubusercontent.com/guysoft/CustomPiOS/devel/src/docker/docker-compose.yml
wget --directory-prefix=./image -c --trust-server-names 'https://downloads.raspberrypi.org/raspbian_lite_latest'
echo "DISTRO_NAME=zynthianos" > .env
sudo docker-compose up -d
sudo docker exec -it zynthianos-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

Thats it! a few hours later you have a zynthian image ready.

Official nightly build also uses this method.

guysoft · December 10, 2018, 3:26pm

There was a bug in the inscrutions.
Step 7 should have been:
7. echo "DISTRO_NAME=zynthianos" > .env
Fixed

wyleu · December 11, 2018, 2:32am

Failed on build . . . .

umount: /distro/workspace/mount: target is busy
(In some cases useful info about processes that
use the device is found by lsof(8) or fuser(1).)
++++ true
++++ echo_red -e ‘\nBUILD FAILED!\n’
++++ echo -e -n ‘\e[91m’
++++ echo -e ‘\nBUILD’ ‘FAILED!\n’

BUILD FAILED!

++++ echo -e -n ‘\e[0m’

docker_build_fail.txt (6.4 KB)

guysoft · December 11, 2018, 10:30am

umount: /distro/workspace/mount: target is busy suggests something is wrong with the container.
Run:

docker rm -f zynthianos-build

And try again, if you still use zynthianos-build-build delete that too.,

wyleu · December 11, 2018, 10:47am

New beast launched 10:34 (Tuesday )

wyleu · December 12, 2018, 6:01pm

Ok i finally stopped it after over a day …

git submodule init
Submodule ‘robtk’ (git://github.com/x42/robtk) registered for path ‘robtk’
git submodule update
Cloning into ‘/home/pi/zynthian-sw/plugins/fat1.lv2/robtk’…
strip -s build/fat1.so

zynthian_build2.txt (3.2 KB)

guysoft · December 12, 2018, 9:48pm

Looks ok in the nightly build:

03:39:29 make[3]: Entering directory '/home/pi/zynthian-sw/plugins/stepseq.lv2'
03:39:29 git submodule init
03:39:30 Submodule 'robtk' (git://github.com/x42/robtk) registered for path 'robtk'
03:39:30 git submodule update
03:39:30 echo "]; ." >> build/stepseq.ttl
03:39:30 cp -r modgui/* build/modgui/
03:39:31 Cloning into '/home/pi/zynthian-sw/plugins/stepseq.lv2/robtk'...
03:39:32 strip -s build/stepseq.so
03:39:33 Submodule path 'robtk': checked out '3abd3e14306d9eb01e95775a321b7fab4af78f57'
03:39:33 make[3]: Leaving directory '/home/pi/zynthian-sw/plugins/stepseq.lv2'
03:39:33 make[2]: Leaving directory '/home/pi/zynthian-sw/plugins/stepseq.lv2'
03:39:34 make[1]: Leaving directory '/home/pi/zynthian-sw/plugins/stepseq.lv2'
03:39:34 sudo: unable to resolve host f39a8248424b

The line is a little different, I’d try again, the code might have changed. Its not showing strip -s build/fat1.so there.

In general it looks like your build is working at this point, give or take its success.

wyleu · December 13, 2018, 8:37am

I burn it to something …

I hunted thew img file… The only one I found under the ZynthianOS folder.

But it does at least boot a pi!!

And away it goes for another build (11:00 13/12/2018)

wyleu · December 13, 2018, 11:20am

sudo docker exec -it zynthianos-build /CustomPiOS/nightly_build_scripts/custompios_nightly_build

+++ dirname /CustomPiOS/build_custom_os
++ cd /CustomPiOS
++ pwd

DIR=/CustomPiOS
/CustomPiOS/build
‘[’ -n ‘’ ‘]’
LOG=build.log
define SCRIPT
IFS=’\n’
read -r -d ‘’ SCRIPT
true
‘[’ build.log ‘!=’ no ‘]’
touch build.log
touch: cannot touch ‘build.log’: No such file or directory

guysoft · December 13, 2018, 9:25pm

Can you show the output of df from the container,

its saying it can’t create build.log which suggest your mount pints for /distro are wrong.

guysoft · December 13, 2018, 9:32pm

It looks from the image installed something
The build has been broken the past few days, see: https://jenkins.zynthian.org/job/ZynthianOS/
Take it up with @jofemodo

I think at the moment its good others can build. perhaps in the future it might be worth to shorten he build process so its easy to build at home. But for not I’d recommend making debuging of it easy first because every other commit @jofemodo does, the build breaks. That should be a rare event. Shortening the build might help it, but Zythian is HUGE, and the only real way to do that is to maintain a Debian software repository.

It might be worth making a docker container that boots the nightly image and lets everyone try out the nightly and debug it.

wyleu · December 14, 2018, 8:24am

zynthian_docker_df.txt (2.4 KB)

guysoft · December 16, 2018, 12:29am

That is df from your host system and not from the container. The issue you are having is because the mount points inside your container are most likely wrong.

Try this:

sudo docker inspect -f '{{ .Mounts }}' zynthianos-build

Should list a mount point.

wyleu · December 16, 2018, 11:09am

wyleu@wyleu-HP-G72-Notebook-PC:~$ sudo docker inspect -f ‘{{.Mounts }}’ zynthianos-build

Error: No such object: zynthianos-build
wyleu@wyleu-HP-G72-Notebook-PC:~$ cd ZynthianOS/
wyleu@wyleu-HP-G72-Notebook-PC:~/ZynthianOS$ sudo docker inspect -f ‘{{.Mounts }}’ zynthianos-build

Error: No such object: zynthianos-build
wyleu@wyleu-HP-G72-Notebook-PC:~/ZynthianOS$

guysoft · December 16, 2018, 11:00am

It says the container os not there dude

sudo docker ps -a?

wyleu · December 16, 2018, 11:15am

~/ZynthianOS/src$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fec5648853d0 redis:2.8 “docker-entrypoint.s…” 2 months ago Exited (255) 2 months ago 0.0.0.0:6379->6379/tcp distracted_wright
f21f214e4f0b redis:2.8 “docker-entrypoint.s…” 3 months ago Created confident_lalande
e51cc8cb1725 redis:2.8 “docker-entrypoint.s…” 3 months ago Exited (0) 2 months ago 0.0.0.0:6379->6379/tcp friendly_babbage
320ab0510c8a ubuntu “bash” 3 months ago Exited (130) 3 months ago agitated_raman
22c849d1da06 ubuntu “bash” 3 months ago Exited (0) 3 months ago festive_archimedes
baa461250855 ubuntu “bash” 3 months ago Exited (0) 3 months ago practical_feynman
8636120ff0e2 hello-world “/hello” 3 months ago Exited (0) 3 months ago zen_jackson

cleared down, started again . . .

This run looks like it’s doing something

wyleu · December 16, 2018, 1:15pm

++++ grep 2018-11-13-raspbian-stretch-lite.img2
++++ awk ‘{print $4-0}’
+++ root_offset=50331648
+++ echo ‘Mounting image 2018-11-13-raspbian-stretch-lite.img on /distro/workspace/mount, offset for boot partition is 4194304, offset for root partition is 50331648’
Mounting image 2018-11-13-raspbian-stretch-lite.img on /distro/workspace/mount, offset for boot partition is 4194304, offset for root partition is 50331648
+++ sudo mount -o loop,offset=50331648 2018-11-13-raspbian-stretch-lite.img /distro/workspace/mount/
+++ [[ 1 != \2 ]]
++++ expr 50331648 - 4194304
+++ sudo mount -o loop,offset=4194304,sizelimit=46137344 2018-11-13-raspbian-stretch-lite.img /distro/workspace/mount/boot
mount: /distro/workspace/2018-11-13-raspbian-stretch-lite.img: failed to setup loop device: No such file or directory

exit 1

Sorry that this is little more than an error msg dump . . .

Perhaps I should dispose of my docker installation and start again . . .

Werewolf · December 16, 2018, 2:39pm

sudo modprobe loop ?

wyleu · December 16, 2018, 2:51pm

No direct reply.

Tried that but I don’t see a loop module in lsmod after doing it . . .

and now I’m back to the instant error about the build.log…

I’m sorry this has become just plain bewildering . . .
Time to read a docker book. . . .

Again . . .

Werewolf · December 16, 2018, 5:37pm

No such file or directory error gives missing loopback kernel module too. On device tree loopback nodes created?

rmmod loop && ls /dev/loop*

ls: /dev/loop* ei saa kasutada: No such file or directory

modprobe loop && ls /dev/loop*

/dev/loop0 /dev/loop1 /dev/loop2 /dev/loop3 /dev/loop4 /dev/loop5 /dev/loop6 /dev/loop7 /dev/loop-control