GitLab Omnibus package upgrade error: It seems you are upgrading from X to Y

Written by - 0 comments

Published on - Listed in Git Linux

The last time I logged into our on-premise GitLab Community Edition (CE), I was greeted by a security notice with a red alert icon:

GitLab security notice

Yikes! Looks like a critical security fix was released and we need to upgrade.

As the notice states, the currently used version is 16.2.1, which is meanwhile outdated and no security fix is available for the 16.2 release.

Upgrade to latest version: Fail

As this installation uses the GitLab omnibus package, the package upgrade can be executed through a handy apt-get dist-upgrade. At least this works fine for one minor version jump, e.g. from 16.2 to 16.3. But APT by default chooses the latest available version, which is 16.5.1 while I'm writing this.

And this results in the following error at the end of the APT output:

Preparing to unpack .../27-gitlab-ce_16.5.1-ce.0_amd64.deb ...
gitlab preinstall: It seems you are upgrading from 16.2 to 16.5.
gitlab preinstall: It is required to upgrade to the latest 16.3.x version first before proceeding.

gitlab preinstall: Please follow the upgrade documentation at
dpkg: error processing archive /tmp/apt-dpkg-install-mGP23F/27-gitlab-ce_16.5.1-ce.0_amd64.deb (--unpack):
 new gitlab-ce package pre-installation script subprocess returned error exit status 1

The package's preinstall script detected a jump in the version number, not just one minor number but 3.

Yep, you're right, GitLab package. Let's do minor upgrades first.

Note: Big thanks and kudos to the package maintainer(s) at GitLab! Not all deb packages detect such situations and this can sometimes lead to major outages by breaking an upgrade.

Do one upgrade at a time (minor version hopping)

The output from the package nicely showed us we should first upgrade to 16.3.x. Let's search for all 16.x versions currently available in the GitLab repository:

root@gitlab:~# apt-cache show gitlab-ce | egrep "^Version: 16"
Version: 16.5.1-ce.0
Version: 16.5.0-ce.0
Version: 16.4.2-ce.0
Version: 16.4.1-ce.0
Version: 16.4.0-ce.0
Version: 16.3.6-ce.0
Version: 16.3.5-ce.0
Version: 16.3.4-ce.0
Version: 16.3.3-ce.0
Version: 16.3.2-ce.0
Version: 16.3.1-ce.0
Version: 16.3.0-ce.0

Version: 16.2.8-ce.0
Version: 16.2.7-ce.0
Version: 16.2.6-ce.0
Version: 16.2.5-ce.0
Version: 16.2.4-ce.0
Version: 16.2.3-ce.0
Version: 16.2.2-ce.0
Version: 16.2.1-ce.0
Version: 16.2.0-ce.0
Version: 16.1.5-ce.0
Version: 16.1.4-ce.0
Version: 16.1.3-ce.0
Version: 16.1.2-ce.0
Version: 16.1.1-ce.0
Version: 16.1.0-ce.0
Version: 16.0.8-ce.0
Version: 16.0.7-ce.0
Version: 16.0.6-ce.0
Version: 16.0.5-ce.0
Version: 16.0.4-ce.0
Version: 16.0.3-ce.0
Version: 16.0.2-ce.0
Version: 16.0.1-ce.0
Version: 16.0.0-ce.0

The latest available 16.3.x version is therefore 16.3.6.

And what about the gitlab-runner package? Interestingly the upgrade of that package supported the jump over multiple minor versions after the apt-get dist-upgrade:

root@gitlab:~# dpkg -l|grep gitlab
ii  gitlab-ce        16.2.1-ce.0  amd64   GitLab Community Edition (including NGINX, Postgres, Redis)
iU  gitlab-runner    16.5.0       amd64   GitLab Runner

To make the versions consistent, I decided to use the same major/minor release (16.3.x) for both packages. For gitlab-runner, this is currently the 16.3.3 release.

To install a specific version, the release/version tag can be added behind the package name:

root@gitlab:~# apt-get install gitlab-ce=16.3.6-ce.0 gitlab-runner=16.3.3
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
Use 'apt autoremove' to remove it.
Suggested packages:
The following packages will be upgraded:
The following packages will be DOWNGRADED:

1 upgraded, 0 newly installed, 1 downgraded, 0 to remove and 0 not upgraded.
37 not fully installed or removed.
Need to get 1,774 MB of archives.
After this operation, 156 MB disk space will be freed.
Do you want to continue? [Y/n] y

The installation of the package(s) ran through nicely:

     _______ __  __          __
    / ____(_) /_/ /   ____ _/ /_
   / / __/ / __/ /   / __ `/ __ \
  / /_/ / / /_/ /___/ /_/ / /_/ /

Upgrade complete! If your GitLab server is misbehaving try running
  sudo gitlab-ctl restart
before anything else.
If you need to roll back to the previous version you can use the database
backup made during the upgrade (scroll up for the filename).

And now the next version (16.4.x) could be installed. Should, actually, because GitLab 16.3.x is EOL in just 2 days (November 16th 2023).

TL;DR: When upgrading GitLab, make sure to not jump over a release and do one minor version upgrade after another.

Add a comment

Show form to leave a comment

Comments (newest first)

No comments yet.

RSS feed

Blog Tags:

  AWS   Android   Ansible   Apache   Apple   Atlassian   BSD   Backup   Bash   Bluecoat   CMS   Chef   Cloud   Coding   Consul   Containers   CouchDB   DB   DNS   Database   Databases   Docker   ELK   Elasticsearch   Filebeat   FreeBSD   Galera   Git   GlusterFS   Grafana   Graphics   HAProxy   HTML   Hacks   Hardware   Icinga   Influx   Internet   Java   KVM   Kibana   Kodi   Kubernetes   LVM   LXC   Linux   Logstash   Mac   Macintosh   Mail   MariaDB   Minio   MongoDB   Monitoring   Multimedia   MySQL   NFS   Nagios   Network   Nginx   OSSEC   OTRS   Office   PGSQL   PHP   Perl   Personal   PostgreSQL   Postgres   PowerDNS   Proxmox   Proxy   Python   Rancher   Rant   Redis   Roundcube   SSL   Samba   Seafile   Security   Shell   SmartOS   Solaris   Surveillance   Systemd   TLS   Tomcat   Ubuntu   Unix   VMWare   VMware   Varnish   Virtualization   Windows   Wireless   Wordpress   Wyse   ZFS   Zoneminder