Rclone looks like a great solution but… getting this to work on most user’s computers (which are likely to be Windows or Macs) without low-level geekiness is a challenge. I tried but failed. Most of the service providers require OAuth which cannot be forwarded, etc. I think I will concentrate on the Borgbackup option which I think can be done without complex user interaction, using the ssh public key of the zynthian. I have already managed to do some backups from the zynthians but there would need a significant amount of time to code up a UI, most likely extending the backup page in webconf. Maybe we might add features like:
Add / remove / modify cloud backup serivce.
Button to backup to cloud service.
List of backups, maybe in a tree view to allow selection of individual files.
Button to restore (selected files) from cloud service.
Maybe the existing restore from file option could offer a similar selection list/tree.
It is a little bit strange to have only one specific solution, but it is a start. I do believe that features like WebDav are worth considering, since they make it possible to connect to the Zynthian from a diversity of services.
For me personally, there is a need for automated backup for sure, but even more a need for syncing between two or more Zynthian machines.
But you know, as usual i’m both very satisfied and happy with everything in this community!
Perhaps you’re right that Rclone is not the best fit, I hadn’t tested it at all.
Before you decide on Borgbackup, take a moment to consider Restic, I’ve been using it for many years for backing up my Laptops. I’ve also used as an addon for backup of Kubenetes Clusters.
Also consider Kopia which I’m seeing a lot in the linux space, it has a nice Windows GUI Interface for accessing your backups.
I am considering each being able to backup to their own repo (or a common one) and for each to be able to restore from any of the repos (or folders within each repo). That way you can have individual backups with the abiltiy to restore from a different device’s backup.
This is very similar to borgbackup. I think it may be a derivative or inpsired by it. It looks good and BorgBase support restic backups. I don’t know if rsync.net do but I suspect not. Kopia also has similar features. The challenge is to find service providers that support these and that we can integrate with simply. It would be good to have a long list of supported providers and using common standards helps with that, but my aspiration is for something that is very simple to configure within zynthian / webconf and just works. I certainly don’t want to add a support burden with individual codebase for each service provider.
I never got on with WebDav (and most other NAS) because it tends to randomly disappear without warning and it can be challenging to detect and recover… but anyone with better experience is welcome to help with a robust solution.
[Edit] It looks like rsync.net supports restic too. I have just bought a year of storage from rsync.net and have the free quota on BorgBase so I will start by implementing something that works with these two platforms (or at least one of them) for a show-and-tell that may allow further / wider discussion.
I am working on integrating this because it does what the other solutions do and has the GUI which some users might like to access their repository… although that does require some geeky config. It also supports other services like webdav so could scale well. I have made a manual backup to rsync.net using kopia and recorded the required steps. I am not looking at integrating the configuration and manual backup/restore into zynthian as a PoC. Something I am not so keen on is that each file or directory has its own snapshot so although you can perform a full backup of multiple folders and files in one operation, they appear as separate snapshots, making it more awkward to present as a single snapshot for restore. I think Restic is similar. I really want to present a the files that are available to restore at each point in time which may be from the last backup or a previous one if they weren’t included or changed in the last backup.
@riban This was my approach for backing up with Kopia, I used the “Kopia WebUI” running locally on the Zynthian to set things up and for later restore operations.
Here is my notes …
# Zynthian Version: ORAM Staging
# zyncoder: oram (ea17c87)
# zynthian-ui: oram (11d4e85)
# zynthian-sys: oram (fae4c87)
# zynthian-data: oram (781d76c)
# zynthian-webconf: oram (0e2c255)
# Note: reboot made after update before continuing
# Install
curl -s https://kopia.io/signing-key | sudo gpg --dearmor -o /etc/apt/keyrings/kopia-keyring.gpg
echo "deb [signed-by=/etc/apt/keyrings/kopia-keyring.gpg] http://packages.kopia.io/apt/ stable main" | sudo tee /etc/apt/sources.list.d/kopia.list
apt update
apt install kopia
# Start Kopia Server on port 51515, really insecure
kopia server start --address 0.0.0.0:51515 --insecure --server-username=root --server-password=opensynth
# Access Kopia WebUI
# http://zynthian.local:51515
# User: root
# Password: opensynth
# Configure Repository via Kopia WebUI
# Default repository config location after creating a repository is...
# /root/.config/kopia/repository.config
# Command to Start Kopia Server using configured Repository
kopia server start --address 0.0.0.0:51515 --insecure --server-username=root --server-password=opensynth --config-file=/root/.config/kopia/repository.config
# Show my configured Polices
/usr/bin/kopia --config-file=/root/.config/kopia/repository.config policy list
# fa14fdd2bf79b36c8d5f889ccedf1610 (global)
# 39846582739e8f872e5aca4e3ad45f92 root@zynthian:/root
# 7d5cd56bcafdcdb0df10d2c4ab38501b root@zynthian:/zynthian/config
# 4a020ef45afc67f64992149b90428e9b root@zynthian:/zynthian/zynthian-my-data
I’ve approached it differently. I installed kopia on the zynthian, added the zynthian’s ssh key to the remote server then created the repo and backed up. I’ve got a PoC working but realised when I read your report that I haven’t installed kopia in the update scripts. I will do that when I get home.
