yunohost upgrade v11 → v12

Knowing migrations between Debian Major Releases I tested and prepared my yunohost Upgrade. These are my notes that I might use for the next upgrade as well.

preparations upgrade bookworm (v12) ✅

update of all components in v11 ✅

on the way

  • conduit missing ✅
  • docker image of whatsapp bridge ✅
  • (hedgedoc only available for yunohost 12)

clean up config-regen ✅

config-regen shows some stuff that should be looked at and possibly migrated to the yunohost way.

plan ✅

Test this on a testing board:

  • board prepared ✅
    • differences: root is not a software-raid (being on an emmc, because on my test setup the sata-controller doesn't work for being booted from)
  • convert root fs to btrfs to have snapshots for falling back ❌
    • generally good idea ✅
    • → live migration not possible, booting into an sdcard armbian using u-boot seems a problem on the testing setup (which contains an emmc which the production system doesn't)
  • make snapshot / backup → doesn't work, see above ❌

upgrade bookworm (v12)

Time needed with services stopped (see below): Start 12:33:05, End 13:49:05 → migration runtime ca. 1:20h (this is on a RockPro64, the system running on a software raid level 1 on two cheap SSDs and a 100Mbit/s downstream internet connection, time for downloading upgrade packages included).

This can be improved by first downloading the upgrade packages in production and then updating during downtime.

preparations

download

Just download all the upgrade packages.


yunohost tools update
yunohost tools upgrade system
cp -a /etc/apt ./apt-bookworm
cd apt-bookworm/trusted.gpg.d/
wget https://forge.yunohost.org/yunohost_bookworm.asc
gpg --dearmor yunohost_bookworm.asc
rm yunohost_bookworm.asc

Inside the copied directory edit sources.list and sources.list.d/* to contain 'bookworm' everywhere in place of 'bullseye' to prepare the download of the upgrade packages.

From the apt-bookworm/sources.list.d/yunohost.list remove the [signed-by=/usr/share/keyrings/yunohost-archive-keyring.gpg] part.


mount --bind apt-bookworm /etc/apt
apt-get update
apt-get --download-only full-upgrade
umount /etc/apt

inform users

  • install webserver replying with correct error
    • → docker nginx ✅
    • change IP of yunohost in firewall
    • too much trouble to put all certificates for https on that server. Will only reply on http.

connectivity

  • connect console (pikvm) and test access to console to debug boot issues
  • set root password to something simple - just in case

downtime

  • set firewall to dnat to the 503 maintenance webserver
  • stop WhatsApp Bridge docker container

backup with services stopped

To get a binary backup with databases intact once

  • stop all services systemctl isolate rescue-ssh.target
  • run backup-script on backup server
  • ln backup as last yunohost 11 backup
  • restart

upgrade yunohost

  • stop app services: systemctl stop fittrackee fittrackee_workers.service conduit gotosocial.service gitea.service anacron.service anacron.timer microblogpub.service flohmarkt_flohmarkt.ween.de.service rustdeskrelay.service rustdesksignal.service couchdb.service dovecot.service rspamd.service hedgedoc.service ntfy.service postsrsd.service atd.service
  • dump mysql database: mysqldump --all-databases > /data/mysqldump.sql
  • run yunohost-migrations
  • before reboot: Interface is renamed to end0 → change /etc/network/interfaces
  • check update of repositories for armbian and rspamd in /etc/apt/ → no problems here during testing
  • disable raspamd.list in /etc/apt/sources.d/rspamd.list (to keep compatible as much as possible I'll go with the rspamd_ynh package and use a forked version to be more up-to-date and upstream that if no problems can be found)
  • systemctl start nginx fail2ban.service (without this installation of rspamd_yunh didn't work)

rspamd & mail

  • rspamd is removed by migrations
    • rspam yunohost app installs version 3.9.0 instead of my actual 3.11.0.
    • install from fork yunohost app install https://github.com/chri2/rspamd_ynh/tree/3.11
    • issue
    • fork

check configuration files

snc is a package not publicitly available checking all files edited by vi into an svc. It allows to easily go back to a former version of a file and detect changes that were not checked in by vi or manually.

  • tls cipherlist got extended by DHE-RSA-CHACHA20-POLY130 (dovecot, postfix)

postfix

  • extension for extra aliases missing: ldap:/etc/postfix/ldap-ou_alias.cf
  • recommendations from https://www.postfix.org/smtp-smuggling.html missing
  • different smtpd_sender_restrictions - comment: 2025-01-20 cv uhh oh! undocumented changes - needs to be reviewed sometimes!

Restore original files and take care of integrating them later, but migrate (TODO: understand changes):


# Rmilter
-milter_mail_macros =  i {mail_addr} {client_addr} {client_name} {auth_authen}
+milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen} {auth_type}
 milter_protocol = 6
-smtpd_milters = inet:localhost:11332
+smtpd_milters = inet:localhost:8891 inet:localhost:11332
+non_smtpd_milters = inet:localhost:8891
  • /etc/postfix/app_senders_login_maps just changed order → accept

flohmarkt


cd /var/www/flohmarkt-dir
rm -rf ./venv
python3 -m venv --without-pip venv
. venv/bin/activate
python3 -m ensurepip
pip3 install app/
pip3 list # check list of packages installed
deactivate

microblogpub

TODO: Brings its own python version - maybe I'll need to recompile?

reboot

apt autoremove --purge reboot

check for failed services


systemctl list-units --failed
  UNIT                       LOAD   ACTIVE SUB    DESCRIPTION                                         >
● ✅ console-setup.service      loaded failed failed Set console font and keymap
● ❓ fittrackee.service         loaded failed failed fittrackee service
● ❓ fittrackee_workers.service loaded failed failed fittrackee task queue service
● ✅ mariadb.service            loaded failed failed MariaDB 10.11.6 database server
● ✅ networking.service         loaded failed failed Raise network interfaces
● ✅ nginx.service              loaded failed failed A high performance web server and a reverse proxy se>

networking can be ignored - I setup the old and the new interface name for the ethernet interface. Added info to remove entry for old interface name.

fittrackee is not critical, I'll take care of it later.

console-setup

setupcon complains that it can't access /tmp.

Maybe it is too early during boot process or the tmpfs entry in fstab is wrong or tmpfs should be mounted by a different unit.

fixed, Solution: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=846256#44

nginx

tries to resolve hosts from /etc/nginx/conf.d/*/reverseproxy.conf which doesn't work in my test setup.

Should be fine in my production system.

  • networking: change of interface name (see above)
  • nginx: worked after reboot (probably a follow-up to networking - no networking, no nginx?)

mariadb/mysql

used by:

  • gitea
  • snappymail

innodb broken, probably because the rsync is from an opened database copied to the test machine

repair in case it happens after upgrade:

  • stop mariadb if it is running
  • deleted /var/lib/mysql/*
  • installed new default database mysql_install_db
    • files owned by root - better use sudo to call this
    • otherwise chown -R mysql: /var/lib/mysql/*
  • start database
  • import dump mysql < mysqldump.sql

Other case with same error messages I had: https://jira.mariadb.org/browse/MDEV-28880

uptime — swtich to productive

  • set a new root password
  • reboot
  • change firewall to nat to yunohost instead of docker nginx

clean up

  • remove old interface name from /etc/network/interfaces
  • upstream or document change found by
    • run regen-conf
    • run snc → none anymore
  • repair fittrackee → upgrade rebuild the python environment, done
  • look at files find /etc -iname '*.dpkg*', migrate and delete
  • update system and apps