/xs-update-manjaro

Update script for Manjaro

Primary LanguageShellOtherNOASSERTION

xs-update-manjaro

ReadMe for Beta/Dev (Switch to Stable)

Table of Contents

↕

Summary

This is a highly configurable, non-interactive script for automating updates for Manjaro Linux, with basic support for other distributions based on Arch Linux. It supports updating the following:

  • Main system packages (via pacman)
  • AUR packages (via an AUR helper)
  • Flatpak packages

Status Notifications are currently supported on the following Desktop Environments:

Suggested Usage and Disclaimer:

Please do not use this script blindly. You should have a firm understanding of how to manually update your computer before using this. You can learn about updating your computer at the following:

This is not a replacement for manually updating/maintaining your own computer, but a supplement. This script automates what it can, but updates needing manual steps (for example, merging .pacnew files) will still need those. Some of the manual steps have been incorporated into this script, but your system(s) may require additional manual steps depending on what packages you have installed.

Always have external bootable media (like a flash drive with manjaro on it) available in case the system becomes unbootable.

Support of other distributions based on Arch Linux

Functions in this script are designed to be distro-agnostic and should work with any distro that uses pacman. Manjaro Linux continues to be the primary testing environment, but feel free to submit issues/pull requests concerning other distributions.

↕VMs tested before each Release (updated from old/original snapshots)↕
DistroDesktopArchSnapshot Date/Version
Manjaro LinuxXfcex86_6417.1.7 18.0 18.1.0 2021/06/08
Manjaro LinuxKDEx86_6420.0-rc3 2020/10/11
Manjaro LinuxGnomex86_642021/03/21
Arch LinuxXfcex86_642022/02/04
EndeavourOSXfcex86_642021/08/30
Garuda LinuxXfcex86_642021/08/09
↕Hardware tested before each Release (continuously updated)↕
HardwareDistroDesktopArchFresh install version/date
AMD Ryzen 3500u (thinkpad laptop)Manjaro LinuxXfcex86_642021/11/26
Intel core i7-3770 + Nvidia gtx 970 (dell tower)Manjaro LinuxXfcex86_642018/10/28
Pinephone v1.2bManjaro LinuxPhosharmbeta 23

Legal stuff

↕

This is licensed under Apache 2.0

  • TL/DR (as I understand it): You can modify, redistribute, or include in sold products as long as you include the license. You lose this right if you start throwing around litigation. No warranty or guarantee is included or implied. Use at your own risk.
= Expand for License details =

Copyright 2016-2023 Steven Hoff (aka "lectrode")

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Execution Overview

↕

Overview of what the script does from start to finish. Some steps may be slightly out of order for readability.

Initialization

↕
  • Define main functions
  • Load Config
  • Determine notification function (config: enable, manual selection)
  • Initialize logging (config: location)
  • Load Persistent data (config: location)
  • Export Config and Persistent data files
  • Perform checks:
    • Ensure only 1 instance is running
    • Wait up to 5 minutes for network connection
    • Check for script updates (config: enable, branch)
    • Wait up to 5 minutes for any already running instances of pacman/pikaur/apacman
    • Check for and remove db.lck
    • Check for and remove /tmp/pikaur_build_deps.lock
  • Start background notification process
  • Package cache cleanup (see Cleanup Tasks for details)

Update Official Repos

↕

  • Update mirrorlist (config: frequency)

    • pacman-mirrors [--geoip || -c $main_country_str]
    • Upon failure, falls back to pacman-mirrors -g

  • Update package signature keys (config: frequency)

    • pacman-key --refresh-keys

  • Pre-update system checks

    • Check if too old (does not support installs with xproto<=7.0.31-1)
    • Partial cache cleanup
    • Check Free space

  • Update repo databases

    • pacman -Syy
      • Repair upon detection of "error: GPGME error: No data" (config: enable):
        • delete package database files (normally stored in /var/lib/pacman/sync)
        • re-attempt pacman -Syy

  • Update keyring packages

    • manual update if packages are older than 1.5 years (config: enable)

  • Download package updates

    • pacman -Su[u]w --needed --noconfirm [--ignore $main_ignorepkgs_str]
      • Upon dependency resolution issues, this will be re-attempted, but with 'd' and 'dd' parameters to skip dependency checks
      • This ensures that as many packages are downloaded as possible before making any major changes

  • Apply manual package changes (config: enable)(see this section for details)

    • If pacman<5.2, switch to pacman-static
    • Required removal and/or replacement of known conflicting packages

  • Update System packages

    • Installed repo packages that end with "-keyring"
    • Installed repo packages that end with "-system"
    • Packages specified in $main_systempkgs_str

  • Check/fix package database errors (config: enable)

    • For every package with errors:
      • search local caches and arch linux archive for package of installed version
        • if installed version not found, fall back to installing the latest version
      • create missing files/desc
      • reinstall with pacman -[S|Udd] --noconfirm --overwrite=* packagename
        • package dependencies are ignored if re-installing same version

  • Update packages from Official Repos

    • pacman -Syyu[u] --needed --noconfirm [--ignore $main_ignorepkgs_str]
    • If this fails, AUR updates are skipped

Update AUR packages

↕

  • AUR updates are skipped after critical system package updates if aur_aftercritical_bool is false

  • Determine available AUR helpers (config: frequency, manual selection)

    • Check if pikaur is functional (config: enable)

  • If AUR helper available/enabled, detect and rebuild AUR packages that need it (config: enable)

    • If packages are still detected as needing a rebuild afterward, these packages are excluded from future attempts (config: number of days to exclude)

  • If selected, update AUR packages with pikaur

  • If selected, update AUR packages with apacman

Cleanup Tasks

↕

  • All cleanup operations (config: enable)

    • Remove orphan packages (config: enable)

      • mark keyring packages as explicitly installed if they would otherwise be removed
      • pacman -Rnsc $(pacman -Qtdq) --noconfirm

    • Package cache cleanup

      • Clean AUR package cache
        • ``paccache -rfqk[$cln_aurpkg_num`](#cln_aurpkg_num "") -c /var/cache/apacman/pkg`
        • ``paccache -rfqk[$cln_aurpkg_num`](#cln_aurpkg_num "") -c /var/cache/pikaur/pkg`
      • Clean AUR build cache (config: enable)
        • rm -rf /var/cache/pikaur/aur_repos/*
        • rm -rf /var/cache/pikaur/build/*
      • Clean pacman package cache

Update Flatpak

↕
  • Update flatpak packages (config: frequency)
    • flatpak update -y
  • Remove flatpak orphan packages (config: enable)
    • flatpak uninstall --unused -y

Final Actions

↕
  • Stop background notification process
  • Determine final message
  • Perform System Power Action (i.e. reboot) if required (config: enable)
    • Delay system power action if users are logged in (config: enable, ignore these users)
    • sync; [reboot|halt|poweroff] || systemctl --force [reboot|halt|poweroff] || systemctl --force --force [reboot|halt|poweroff] (config: action)
  • Final message if system power action not performed (config: duration)
  • Stop auto-update service and quit

Supported Automatic Repair and Manual Changes

↕

Note: All current and future automatic repair and manual package changes can be disabled in one setting

Automatic Repair

This script supports detecting and repairing the following potential issues:

Manual Package Changes

Every once in a while, updating Manjaro requires manual package changes to allow updates to succeed. This script supports automatically performing the following:

  • Setup and use pacman-static if pacman<5.2

  • Mark packages as explicitely installed:

adapta-black-breath-theme
adapta-black-maia-theme
adapta-breath-theme
adapta-gtk-theme
adapta-maia-theme
arc-themes-maia
arc-themes-breath
matcha-gtk-theme		left as orphans with kvantum-manjaro>=0.13.5+1+g333aa00-1
breath-wallpaper
vertex-maia-icon-theme		left as orphans with manjaro-xfce-settings>=20200109-1
  • Mark packages as dependency:
phonon-qt4-gstreamer
phonon-qt4-vlc
phonon-qt4-mplayer-git		extras for phonon-qt4<4.11.0
(qt4 support dropped, package removed from repos 2019/05)
  • Package removal:
jre-openjdk
jre-openjdk-headless		<=21.u35-32023/11/02: java 21 packages now conflict. of the ones installed, the most functional package is kept
amtk<=5.6.1-22023/09/28: replaced with libgedit-amtk (revisit later)
gnome-shell-extension-desktop-icons-ng<=47-12022/12/16: replaced with gnome-shell-extension-gtk4-desktop-icons-ng
libxfce4ui-nocsd<=4.17.0-12022/12/23: removed from repos
lib32-db<=5.3.28-52022/12/21: moved to aur
kjsembed<=5.100.0-12022/12/20: removed from repos
glib2-static<=2.72.3-12022/09/07: merged into glib2
wxgtk2<=3.0.5.1-32022/07/14: removed from arch repos
manjaro-gdm-theme<=20210528-12022/04/23: removed from repos
libkipi<=22.04.0-12022/04/22: moved to AUR
kvantum-qt5<=0.20.2-22022/01/02: removed from repos
user-manager<=5.19.5-12020/11/04: removed from repos
kvantum-theme-matchama<=20191118-12022/02/14: removed from repos, 2023/10/11: re-added (renamed)
[lib32-]libcanberra-gstreamer<=0.30+2+gc0620e4-32021/06: merged into lib32-/libcanberra-pulse
python2-dbus<=1.2.16-32021/03: removed from dbus-python
knetattach<=5.20.5-12021/01/09: merged into plasma-desktop
ms-office-online<=20.1.0-12020/06: former default pkg moved to AUR
libxxf86dga
libdmx
libxxf86misc		 <=1.1.5-1
<=1.1.4-1
<=1.0.4-1		2019/12/20: moved to AUR
pyqt5-common<=5.13.2-12019/12: removed from repos
ilmbase<=2.3.0-12019/10: merged into openexr
breeze-kde4
oxygen-kde4
sni-qt		 <=5.13.4-1
<=5.13.4-1
<=0.2.6-5		2019/05: removed from repos
libmagick<=7.0.8.41-12019/04/22: merged into imagemagick
colord<=1.4.4-12019/??: conflicts with libcolord
kuiserver<=5.12.5-32018/06/12: removed from repos
[lib32-]gtk3-classic<=3.24.24-1Xfce 18.0.4: replaced with gtk3
engrampa-thunar-plugin<=1.0-2Xfce 17.1.10: removed from repos
  • Package replacement:
dbus-x11<=1.14.4-12022/12/16: dropped in favor of dbus
qpdfview<=0.4.18-12022/04/01: former default pkg moved to AUR, replaced with evince
galculator-gtk2<=2.1.4-52021/11/13: replaced with galculator
manjaro-kde-settings-19.0,breath2-icon-themes,plasma5-themes-breath2<=20200426-12021/11: replaced with manjaro-kde-settings,plasma5-themes-breath
[lib32-]jack<=0.125.0-102021/07/26: replaced with [lib32-]jack2
gksu-polkit<=0.0.3-22020/10: replaced with zensu
  • Package overwrite:
glibc-locales<=2.38-52023/10/01: split package (glibc) conflicts with files generated by old

  • Transition packages that depend on electron to electronXX where required

  • Install new base-devel metapackage if system had the (now defunct) group installed

Installation and Requirements

↕

Dependencies:

Required:

  • coreutils, pacman, grep, iputils

Optional:

pacman-contribfor package cache cleanup support (if packaged separately, i.e. Arch Linux)
pacman-mirrorsfor mirror update support
AUR Helperfor AUR package support
flatpakfor flatpak package support
notification daemonusually a part of the desktop environment; for notification support
lsoffor more thorough detection of reboot needed on login
notify-desktoprequired for KDE notifications, optional alternative for Xfce, Gnome
wgetif available, will use instead of curl

Installation

  1. Move script files to these locations:
ElectrodeXS.png         -> /usr/share/pixmaps/
auto-update.sh          -> /usr/share/xs/
xs-autoupdate.service   -> /etc/systemd/system/
xs-updatehelper.desktop -> /etc/xdg/autostart/

  1. Make sure auto-update.sh is allowed to execute as a program

  2. Enable running the auto-update script at startup (optional):

  • sudo systemctl enable xs-autoupdate
  1. You can manually run the script with the following:
  • sudo systemctl start xs-autoupdate (run silently as service)
  • sudo /usr/share/xs/auto-update.sh (watch logs)
  • sudo /usr/share/xs/auto-update.sh nofork (watch logs, do not fork to background)

Supported AUR Helpers:

↕

If you want the script to automatically update packages from the AUR, it will need one of the following:

pikaur (recommended)

You can install pikaur with another AUR helper, or install it directly with the following:

sudo pacman -S --needed base-devel git
git clone https://aur.archlinux.org/pikaur.git
cd pikaur
makepkg -fsri

Features:

Drawbacks:

  • Does not support automatically importing PGP keys
  • (workaround: pass --skippgpcheck custom flag to packages that need it)
apacman (deprecated)

You can install apacman (deprecated) with the following:

git clone https://aur.archlinux.org/apacman.git
pushd apacman
makepkg -si --noconfirm
popd
rm -rf apacman
#Replace old apacman with my fork with some fixes (not currently maintained)
sudo wget "https://raw.githubusercontent.com/lectrode/apacman/master/apacman" -O "/usr/bin/apacman"
sudo chmod +x "/usr/bin/apacman"

Features:

  • Automatically imports PGP keys for packages

Drawbacks:

  • No longer maintained upstream
  • Does not support newer AUR packages
  • Cannot pass custom makepkg flags
  • Support will be removed in future version of script

Configuration

=Overview=

  • By default settings are located at /etc/xs/auto-update.conf

  • Settings file is (re)generated on every run

  • Older settings will be converted to preserve preferences

  • True and False are 1 and 0 respectively

  • Settings location can be changed by exporting xs_autoupdate_conf environment variable

    • This needs absolute path and filename
    • Warning: whichever file is specified will be overwritten whenever the script runs
=Sample configuration file=
  • NOTE: Blank line at end may be required for last line to be parsed
aur_1helper_str=auto
aur_aftercritical_bool=0
aur_update_freq=3
aur_devel_freq=6
cln_1enable_bool=1
cln_aurbuild_bool=0
cln_aurpkg_num=1
cln_flatpakorphan_bool=1
cln_orphan_bool=1
cln_paccache_num=1
flatpak_update_freq=3
main_country_str=Global,United_States
main_ignorepkgs_str=
main_logdir_str=/var/log/xs
main_perstdir_str=
main_systempkgs_str=
main_testsite_str=www.google.com
notify_1enable_bool=1
notify_errors_bool=1
notify_function_str=auto
notify_lastmsg_num=20
notify_vsn_bool=0
reboot_1enable_num=1
reboot_action_str=reboot
reboot_delayiflogin_bool=1
reboot_delay_num=120
reboot_ignoreusers_str=nobody lightdm sddm gdm
reboot_notifyrep_num=10
repair_db01_bool=1
repair_db02_bool=1
repair_manualpkg_bool=1
repair_pikaur01_bool=1
repair_aurrbld_bool=1
repair_aurrbldfail_freq=32
self_1enable_bool=1
self_branch_str=stable
update_downgrades_bool=1
update_keys_freq=30
update_mirrors_freq=1
zflag:dropbox,tor-browser=--skippgpcheck
=Custom makepkg flags for specific AUR packages=
  • Requires pikaur
  • You can add as many entries as you need
  • All packages listed in one line will be updated at the same time
  • Format: zflag:package1,package2=--flag1,--flag2,--flag3

Individual Settings

aur_1helper_str
  • Default: auto
  • Specifies which AUR helper to use to update AUR packages
  • Current valid values are: auto,none,all,pikaur,apacman
  • auto will use an available AUR helper with the following preference: pikaur > apacman
  • all will run every supported AUR helper found in this order: pikaur, apacman
  • none will not use any AUR helper
aur_aftercritical_bool
  • Default: 0 (False)
  • If set to false, script will skip AUR package updates after critical main system packages have been updated
  • If set to true, script will proceed to update AUR packages, regardless of critical main package updates
aur_update_freq
  • Default: 3
  • Every X days, update AUR packages (-1 disables all AUR updates, including devel)
aur_devel_freq
  • Default: 6
  • Every X days, update "devel" AUR packages (any package that ends in -git, -svn, etc) (-1 to disable)
  • This setting only applies if AUR packages are updated with pikaur
cln_1enable_bool
  • Default: 1 (True)
  • If set to false, disables all cleanup steps
cln_aurpkg_num
  • Default: 1
  • Specifies the number of AUR (built) package versions to keep in cache
  • If set to "-1" all AUR package versions will be kept
  • pikaur cache: /var/cache/pikaur/pkg
  • apacman cache: /var/cache/apacman/pkg
cln_aurbuild_bool
  • Default: 1 (True)
  • If this is True, all AUR package build folders will be deleted when finished
cln_orphan_bool
  • Default: 1 (True)
  • If this is True, obsolete dependencies from main repos will be uninstalled
cln_flatpakorphan_bool
  • Default: 1 (True)
  • If this is True, obsolete flatpak dependencies will be uninstalled
cln_paccache_num
  • Default: 1
  • Specifies the number of repo package versions to keep in cache
  • If set to "-1" all official package versions will be kept (cache is usually /var/cache/pacman/pkg)
flatpak_update_freq
  • Default: 3
  • Every X days, check for Flatpak package updates (-1 to disable)
notify_1enable_bool
  • Default: 1 (True)
  • If true, enables status notifications to active users
notify_function_str
  • Default: auto
  • Specifies which notification method to use
  • Current valid values are: auto,gdbus,desk,send
    • auto: will automatically select the best method
    • gdbus: uses gdbus to create notifications (works on Xfce, Gnome)
    • desk: uses notify-desktop to create notifications (works on Xfce, KDE, and Gnome)
    • send: uses notify-send to create notifications (partial Xfce and KDE support - does not support replacing/dismissing existing notifications, which may result in notification spam)
  • Note: if desk is specified (or if auto is specified and KDE is detected), and an AUR helper is configured, script will attempt to install notify-desktop-git to provide this functionality
notify_lastmsg_num
  • Default: 20
  • Specifies how long (in seconds) the "System update finished" notification is visible before it expires.
  • The "Kernel and/or drivers were updated" message does not expire, regardless of this setting
notify_errors_bool
  • Default: 1 (True)
  • If true, script will state which tasks failed in the "System update finished" notification
notify_vsn_bool
  • Default: 0 (False)
  • If true, the version number of the script will be included in notifications
main_ignorepkgs_str
  • Default: (blank)
  • Packages to ignore, separated by spaces (these are in addition to those stored in pacman.conf)
main_systempkgs_str
  • Default: (blank)
  • Packages to update before any other packages (i.e. archlinux-keyring), separated by spaces
main_inhibit_bool
  • Default: 1 (True)
  • If true, script will inhibit accidental restart/shutdown/hibernate/suspend while the script is updating the system
  • This can be manually overridden with one of the following methods:
    • WARNING: interupting a system update can result in a non-functoinal system! Use with caution!
    • Execute with elevated permissions, i.e. sudo reboot, or sudo systemctl suspend
    • Stop the script with sudo pkill auto-update
  • NOTE: On KDE, while inhibited, selecting shutdown or restart results in a black screen (use different TTY to get back in)
main_logdir_str
  • Default: /var/log/xs
  • Defines the directory where the log will be output
main_perstdir_str
  • Default: (blank)
  • Defines the directory where persistent timestamps are stored. If blank, uses main_logdir_str
main_country_str
  • Default: (blank)
  • If blank, pacman-mirrors --geoip is used
  • Countries separated by commas from which to pull updates
  • See output of pacman-mirrors -l for supported values
main_testsite_str
  • Default: www.google.com
  • Script checks if there is internet access by attempting to ping this address
  • Can also be an IP address
reboot_1enable_num
  • Determines when script should perform "System Power Action" (see reboot_action_str below)
  • Default: 1
  • -1: Disable in all cases
  • 0: Only if rebooting manually may not be possible (system may be in critical state after critical package update)
  • 1: Only after critical system packages have been updated
  • 2: Always reboot, regardless of any updates
reboot_action_str
  • This is the System Power Action the script should take when required
  • Default: reboot
  • reboot: System will be restarted
  • halt: System will be halted (shutdown, with hardware left running)
  • poweroff: System will be powered off (shutdown, with hardware powered off)
reboot_delayiflogin_bool
  • Default: 1 (True)
  • If true, the reboot will be delayed only if a user is logged in. If false, there will always be a delay
reboot_delay_num
  • Default: 120
  • Delay in seconds to wait before rebooting the computer
reboot_notifyrep_num
  • Default: 10
  • Reboot notification is updated every X seconds
  • Works best if reboot_delay_num is evenly divisible by this
reboot_ignoreusers_str
  • Default: nobody lightdm sddm gdm
  • List of users separated by spaces
  • These users will not trigger the reboot delay even if they are logged on
repair_1enable_bool
  • Default: 1 (True)
  • Enables/Disables all repair steps
  • NOTE: If either this, or the individual repair option is disabled, that repair will be ignored
repair_db01_bool
  • Default: 1 (True)
  • If true, the script will detect and attempt to repair missing "desc"/"files" files in package database
  • NOTE: It repairs this by creating the missing files and re-installing the package(s) with overwrite=* specified
  • NOTE: in 3.9.8-rc1 and later, this will attempt to reinstall the local version before installing the latest
repair_db02_bool
  • Default: 1 (True)
  • If true, the script will detect and attempt to redownload corrupt package database files
  • NOTE: It repairs this by removing existing package database files, then running 'pacman -Syy'
repair_keyringpkg_bool
  • Default: 1 (True)
  • If true, the script will detect and attempt to repair outdated keyring packages
repair_manualpkg_bool
  • Default: 1 (True)
  • If true, script will check for and perform critical package changes required for continued updates
  • See Automatic Repair for specific package changes the script supports
repair_pikaur01_bool
  • Default: 1 (True)
  • If true, the script will attempt to re-install pikaur if it is not functioning
  • NOTE: Specifically needed if python is updated
repair_aurrbld_bool
  • Default: 1 (True)
  • If true, the script will attempt to rebuild AUR packages when needed (for example: if their dependencies were updated)
  • Depends on external tool: rebuild-detector (available in official repos)
repair_aurrbldfail_freq
  • Default: 32 (True)
  • After the script finishes attempting to rebuild packages that need it, any packages that still need to be rebuilt are excluded from future runs for this number of days
self_1enable_bool
  • Default: 1 (True)
  • If true, script checks for updates for itself ("self-updates")
self_branch_str
  • Default: stable
  • Script update branch (requires self_1enable_bool be True)
  • Current valid values are: stable, beta
update_downgrades_bool
  • Default: 1 (True)
  • If true, allows pacman to downgrade packages if remote packages are a lesser version than installed
update_mirrors_freq
  • Default: 0
  • Every X days, refreshes mirror list before checking for package updates (-1 to disable)
update_keys_freq
  • Default: 30
  • Every X days, runs pacman-key --refresh-keys before checking for package updates (-1 to disable)