After a LXC container received the pleasure of a distribution release (Debian 10 to Debian 11), the following apt autoremove process failed to remove the old Python version (3.7) from the system and returned the following error:
Removing python3.7 (3.7.3-2+deb10u3) ...
Removing libpython3.7-stdlib:amd64 (3.7.3-2+deb10u3) ...
Removing python3.7-minimal (3.7.3-2+deb10u3) ...
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/python3.7 for writing: Permission denied
update-binfmts: warning: unable to disable binary format python3.7
update-binfmts: exiting due to previous errors
dpkg: error processing package python3.7-minimal (--remove):
installed python3.7-minimal package pre-removal script subprocess returned error exit status 2
dpkg: too many errors, stopping
Errors were encountered while processing:
python3.7-minimal
Processing was halted because there were too many errors.
E:Sub-process /usr/bin/dpkg returned an error code (1)
Let's find out the reason behind this.
Research led to existing bug reports in both Debian and Ubuntu projects:
A brief discussion was also held in LXC Issue #388 where Stéphane Graber, maintainer of LXC and LXD, said:
Right, you may want to try and argue with the package maintainer that update-binfmts shouldn't be fatal. The permission denied is perfectly expected and caused by our apparmor profile as this would otherwise seriously mess up your host.
But where does the Python removal use update-binfmts?
Checking out the Debian pre- and post package scripts in /var/lib/dpkg/info, a grep across the Python 3.7 files quickly returns a result:
root@bullseye:/var/lib/dpkg/info# grep binfmts python3.7-minimal.*
python3.7-minimal.list:/usr/share/binfmts
python3.7-minimal.list:/usr/share/binfmts/python3.7
python3.7-minimal.md5sums:9d48bf1dd4dbda1a67180813849b4745 usr/share/binfmts/python3.7
python3.7-minimal.postinst: if which update-binfmts >/dev/null; then
python3.7-minimal.postinst: update-binfmts --import python3.7
python3.7-minimal.prerm: if which update-binfmts >/dev/null; then
python3.7-minimal.prerm: update-binfmts --package python3.7 --remove python3.7 /usr/bin/python3.7
The pre-removal script (suffixed with .prerm) contains the following if statement:
if which update-binfmts >/dev/null; then
update-binfmts --package python3.7 --remove python3.7 /usr/bin/python3.7
fi
Basically the pre-removal script checks whether the update-binfmts command exists. If it does, it runs the command. But why and what for?
This command, update-binfmts, is part of the binfmt-support package:
root@bullseye:~# dpkg -S /usr/sbin/update-binfmts
binfmt-support: /usr/sbin/update-binfmts
But what is binfmt-support and what does it do?
Quoting the official website of binfmt-support:
The binfmt_misc kernel module, contained in versions 2.1.43 and later of the Linux kernel, allows system administrators to register interpreters for various binary formats based on a magic number or their file extension, and cause the appropriate interpreter to be invoked whenever a matching file is executed. Think of it as a more flexible version of the #! executable interpreter mechanism.
binfmt-support provides an update-binfmts script with which package maintainers can register interpreters to be used with this module without having to worry about writing their own init.d scripts, and which sysadmins can use for a slightly higher-level interface to this module.
I'm not saying that I fully understand the description, but from what I understand, the idea itself sounds not too bad. However the command (update-binfmts) tries to write into procfs, which is shared with the host and is (by security concept) not allowed to be written to from within a LXC container.
This can be verified by running the update-binfmts command from above's prerm script manually:
root@bullseye:~# update-binfmts --package python3.7 --remove python3.7 /usr/bin/python3.7
update-binfmts: warning: unable to open /proc/sys/fs/binfmt_misc/python3.7 for writing: Permission denied
update-binfmts: warning: unable to disable binary format python3.7
update-binfmts: exiting due to previous errors
root@bullseye:~# echo $?
2
The exit code is non-zero, therefore the whole pre-removal script fails and with it the whole apt autoremove process.
As seen in the pre-removal script above, update-binfmts is only executed if the command actually exists on the system. What if we simply remove this package?
root@bullseye:~# apt-get remove binfmt-support
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages were automatically installed and are no longer required:
dbconfig-common dbconfig-mysql dh-python fdisk gnupg-agent guile-2.0-libs libboost-filesystem1.62.0 libboost-iostreams1.62.0 libboost-iostreams1.67.0 libboost-system1.62.0 libboost-system1.67.0 libcurl3-gnutls libdns-export1104
libdns1104 libdns1110 libevent-2.0-5 libevent-2.1-6 libgdbm3 libgeoip1 libhogweed4 libicu57 libicu63 libicu65 libip6tc0 libiptc0 libisc-export1100 libisc1100 libisc1105 libisc160 libisccc140 libisccc161 libjemalloc1 libjs-jquery
libjs-sphinxdoc libjs-underscore libjson-c3 libkyotocabinet16v5 liblogging-stdlog0 liblua5.2-0 liblwres141 liblwres161 liblzo2-2 libmpdec2 libmpfr4 libnettle6 libperl5.28 libpipeline1 libprocps7 libpython2-stdlib libpython2.7
libpython2.7-minimal libpython2.7-stdlib libpython3.5-minimal libpython3.5-stdlib libreadline5 libunistring0 perl-modules-5.28 php-phpseclib python2 python2-minimal python2.7 python2.7-minimal python3-distutils python3-lib2to3
python3.5 python3.5-minimal
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
binfmt-support
0 upgraded, 0 newly installed, 1 to remove and 2 not upgraded.
After this operation, 216 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 26947 files and directories currently installed.)
Removing binfmt-support (2.2.1-1) ...
Processing triggers for mailcap (3.69) ...
The if statement in the pre-removal script, should now return false and the update-binfmts command should not be executed:
root@bullseye:~# which update-binfmts
root@bullseye:~# echo $?
1
Can we now run apt autoremove without running into an error? Let's try it:
root@bullseye:~# apt autoremove
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
dbconfig-common dbconfig-mysql dh-python fdisk gnupg-agent guile-2.0-libs libboost-filesystem1.62.0 libboost-iostreams1.62.0 libboost-iostreams1.67.0 libboost-system1.62.0 libboost-system1.67.0 libcurl3-gnutls libdns-export1104
libdns1104 libdns1110 libevent-2.0-5 libevent-2.1-6 libgdbm3 libgeoip1 libhogweed4 libicu57 libicu63 libicu65 libip6tc0 libiptc0 libisc-export1100 libisc1100 libisc1105 libisc160 libisccc140 libisccc161 libjemalloc1 libjs-jquery
libjs-sphinxdoc libjs-underscore libjson-c3 libkyotocabinet16v5 liblogging-stdlog0 liblua5.2-0 liblwres141 liblwres161 liblzo2-2 libmpdec2 libmpfr4 libnettle6 libperl5.28 libpipeline1 libprocps7 libpython2-stdlib libpython2.7
libpython2.7-minimal libpython2.7-stdlib libpython3.5-minimal libpython3.5-stdlib libreadline5 libunistring0 perl-modules-5.28 php-phpseclib python2 python2-minimal python2.7 python2.7-minimal python3-distutils python3-lib2to3
python3.5 python3.5-minimal
0 upgraded, 0 newly installed, 66 to remove and 2 not upgraded.
After this operation, 232 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 26935 files and directories currently installed.)
Removing dbconfig-mysql (2.0.19) ...
Removing dbconfig-common (2.0.19) ...
Removing dh-python (4.20201102+nmu1) ...
Removing fdisk (2.36.1-8+deb11u1) ...
Removing gnupg-agent (2.2.27-2+deb11u1) ...
Removing guile-2.0-libs:amd64 (2.0.13+1-5.1) ...
Removing libboost-filesystem1.62.0:amd64 (1.62.0+dfsg-4) ...
Removing libboost-iostreams1.62.0:amd64 (1.62.0+dfsg-4) ...
Removing libboost-iostreams1.67.0:amd64 (1.67.0-13+deb10u1) ...
Removing libboost-system1.62.0:amd64 (1.62.0+dfsg-4) ...
Removing libboost-system1.67.0:amd64 (1.67.0-13+deb10u1) ...
Removing libcurl3-gnutls:amd64 (7.74.0-1.3+deb11u1) ...
Removing libdns-export1104 (1:9.11.5.P4+dfsg-5.1+deb10u7) ...
Removing libdns1104:amd64 (1:9.11.5.P4+dfsg-5.1+deb10u7) ...
Removing libdns1110:amd64 (1:9.11.19+dfsg-2.1) ...
Removing libevent-2.0-5:amd64 (2.0.21-stable-3) ...
Removing libevent-2.1-6:amd64 (2.1.8-stable-4) ...
Removing libgdbm3:amd64 (1.8.3-14) ...
Removing liblwres141:amd64 (1:9.10.3.dfsg.P4-12.3+deb9u12) ...
Removing libisccc140:amd64 (1:9.10.3.dfsg.P4-12.3+deb9u12) ...
Removing libhogweed4:amd64 (3.4.1-1+deb10u1) ...
Removing libicu57:amd64 (57.1-6+deb9u5) ...
Removing libicu63:amd64 (63.1-6+deb10u3) ...
Removing libicu65:amd64 (65.1-1+0~20200223.8+debian10~1.gbp519cf3) ...
Removing libip6tc0:amd64 (1.8.2-4) ...
Removing libiptc0:amd64 (1.8.7-1) ...
Removing libisc-export1100:amd64 (1:9.11.5.P4+dfsg-5.1+deb10u7) ...
Removing libisc1100:amd64 (1:9.11.5.P4+dfsg-5.1+deb10u7) ...
Removing libisccc161:amd64 (1:9.11.19+dfsg-2.1) ...
Removing libisc1105:amd64 (1:9.11.19+dfsg-2.1) ...
Removing libisc160:amd64 (1:9.10.3.dfsg.P4-12.3+deb9u12) ...
Removing libjemalloc1 (3.6.0-9.1) ...
Removing libjs-sphinxdoc (3.4.3-2) ...
Removing libjs-jquery (3.5.1+dfsg+~3.5.5-7) ...
Removing libjs-underscore (1.9.1~dfsg-3) ...
Removing libjson-c3:amd64 (0.12.1+ds-2+deb10u1) ...
Removing libkyotocabinet16v5:amd64 (1.2.76-4.2+b1) ...
Removing liblogging-stdlog0:amd64 (1.0.5-2+b2) ...
Removing liblua5.2-0:amd64 (5.2.4-1.1+b3) ...
Removing liblwres161:amd64 (1:9.11.19+dfsg-2.1) ...
Removing liblzo2-2:amd64 (2.10-2) ...
Removing python3.5 (3.5.3-1+deb9u5) ...
Removing libpython3.5-stdlib:amd64 (3.5.3-1+deb9u5) ...
Removing libmpdec2:amd64 (2.4.2-2) ...
Removing libmpfr4:amd64 (3.1.5-1) ...
Removing libnettle6:amd64 (3.4.1-1+deb10u1) ...
Removing libperl5.28:amd64 (5.28.1-6+deb10u1) ...
Removing libpipeline1:amd64 (1.5.3-1) ...
Removing libprocps7:amd64 (2:3.3.15-2) ...
Removing python2 (2.7.18-3) ...
Removing libpython2-stdlib:amd64 (2.7.18-3) ...
Removing libpython2.7:amd64 (2.7.18-8) ...
Removing python2.7 (2.7.18-8) ...
Removing libpython2.7-stdlib:amd64 (2.7.18-8) ...
Removing python2-minimal (2.7.18-3) ...
Removing python2.7-minimal (2.7.18-8) ...
Unlinking and removing bytecode for runtime python2.7
Removing libpython2.7-minimal:amd64 (2.7.18-8) ...
Removing python3.5-minimal (3.5.3-1+deb9u5) ...
Unlinking and removing bytecode for runtime python3.5
Removing libpython3.5-minimal:amd64 (3.5.3-1+deb9u5) ...
Removing libreadline5:amd64 (5.2+dfsg-3+b13) ...
Removing libunistring0:amd64 (0.9.6+really0.9.3-0.1) ...
Removing perl-modules-5.28 (5.28.1-6+deb10u1) ...
Removing php-phpseclib (2.0.30-2) ...
Removing python3-distutils (3.9.2-1) ...
Removing python3-lib2to3 (3.9.2-1) ...
Removing libgeoip1:amd64 (1.6.12-7) ...
Processing triggers for mailcap (3.69) ...
Processing triggers for libc-bin (2.31-13+deb11u3) ...
root@irczsrvc15:~# echo $?
0
Yes, apt autoremove ran through perfectly fine!
Note that a new LXC container, created from the LXC "download" image with Debian Bullseye as distribution release does by default not contain the binfmt-support package:
root@host:~# lxc-create -n bullseye -B lvm --vgname=vglxc --fstype=ext4 --fssize=10G -t download -- -d debian -r bullseye -a amd64
However the older release, Debian Buster, had shipped the binfmt-support package and due to the distribution upgrade took the package with it.
That means that newly created Debian Bullseye LXC containers should not experience that problem in the future.
No comments yet.
AWS Android Ansible Apache Apple Atlassian BSD Backup Bash Bluecoat CMS Chef Cloud Coding Consul Containers CouchDB DB DNS Database Databases Docker ELK Elasticsearch Filebeat FreeBSD Galera Git GlusterFS Grafana Graphics HAProxy HTML Hacks Hardware Icinga Icingaweb Icingaweb2 Influx Internet Java KVM Kibana Kodi Kubernetes LVM LXC Linux Logstash Mac Macintosh Mail MariaDB Minio MongoDB Monitoring Multimedia MySQL NFS Nagios Network Nginx OSSEC OTRS Office PGSQL PHP Perl Personal PostgreSQL Postgres PowerDNS Proxmox Proxy Python Rancher Rant Redis Roundcube SSL Samba Seafile Security Shell SmartOS Solaris Surveillance Systemd TLS Tomcat Ubuntu Unix VMWare VMware Varnish Virtualization Windows Wireless Wordpress Wyse ZFS Zoneminder