The problem#

My personal desktop runs Debian Unstable ("Sid")¹. The nature of running a bleeding edge distro is that things break sometimes. I use Debian Testing/Stable or Ubuntu on my other machines to make my life easier, but I often want access to the latest version of some piece of software and running Debian Unstable is one way to do that. Admittedly, I also do it partially just because fixing things that break is a good way of learning how things work.

The most common kind of problem I run into is that upgrades are not straightforward. For their unstable distro, Debian doesn't make any promises about package dependencies not changing. This is less of a problem when there's an additional package that needs to be installed, but can be complicated when there's conflicts which require removing packages to get an upgrade to go through.

Recently I ran into an extreme version of this problem: trying to upgrade, it proposed uninstalling nearly everything I had installed. Worse, trying to resolve the issue, I got a scary sounding warning that I had uninstalled libssl3:

dpkg: libssl3:amd64: dependency problems, but removing anyway as you requested:
 [...]
 systemd depends on libssl3 (>= 3.0.0).
 sudo depends on libssl3 (>= 3.0.0).
 [...]

Both of those sound important.

The solution#

Luckily, it wasn't as bad as it sounded. Looking at the message, it turned out I had replaced libssl3 with libssl3t64. The latter of which is actually the exact same thing, although the package manager doesn't know that. The reason for the different package name is part of the Debian project to transition to 64-bit time_t, which is required to fix the Year 2038 problem. While on AMD64 and other 64-bit architectures, everything already uses 64-bit time_t, that's not true of all platforms that Debian supports. The way Debian handles ABI transitions like this is to rename the library packages with a suffix (t64 for this one) to ensure the old and new ABI don't get mixed accidentally. Since all of the architectures share the package names, the rename also happens on AMD64 even though there's actual change to match the rename on other platforms where the ABI did change.

Presumably the upgrade will be smoother when done between stable versions, but it really confused apt (which I usually use via wajig):

$ wajig install libssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libegl1 : Depends: libegl-mesa0 but it is not going to be installed
 libreoffice-core : Depends: libgstreamer-plugins-base1.0-0 (>= 1.0.0) but it is not going to be installed
                    Depends: libgstreamer1.0-0 (>= 1.4.0) but it is not going to be installed
                    Depends: liborcus-0.18-0 (>= 0.19.2) but it is not going to be installed
                    Depends: liborcus-parser-0.18-0 (>= 0.19.2) but it is not going to be installed
 wine-development : Depends: wine64-development (>= 8.21~repack-1) but it is not going to be installed or
                             wine32-development (>= 8.21~repack-1)
                    Depends: wine64-development (< 8.21~repack-1.1~) but it is not going to be installed or
                             wine32-development (< 8.21~repack-1.1~)
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

Yeah, no idea what libegl1, libreoffice-core, or wine-development have to do with upgrading libssl-dev, but apt was showing those same packages in the error messages no matter what I tried to upgrade and trying to upgrade those packages didn't work either. Luckily, aptitude was able to handle it somewhat better:

$ sudo aptitude install libssl-dev
The following packages will be upgraded:
  libssl-dev{b}
1 packages upgraded, 0 newly installed, 0 to remove and 1459 not upgraded.
Need to get 2,699 kB of archives. After unpacking 1,122 kB will be used.
The following packages have unmet dependencies:
 libssl-dev : Depends: libssl3t64 (= 3.2.1-3) but it is not going to be installed
The following actions will resolve these dependencies:

     Remove the following packages:
1)     libssl3 [3.1.4-2 (now)]
2)     libssl3:i386 [3.1.4-2 (now)]

     Install the following packages:
3)     libssl3t64 [3.2.1-3 (testing, unstable)]
4)     libssl3t64:i386 [3.2.1-3 (testing, unstable)]



Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
  libssl3t64{a} libssl3t64:i386{a}
The following packages will be REMOVED:
  libssl3{a} libssl3:i386{a}
The following packages will be upgraded:
  libssl-dev
1 packages upgraded, 2 newly installed, 2 to remove and 1457 not upgraded.
Need to get 7,177 kB of archives. After unpacking 2,294 kB will be used.
Do you want to continue? [Y/n/?]

Getting the packages to upgrade involved a lot of calls to aptitude that looked like that: removing a list of libraries and a installing a matching list of new libraries whose names were identical to those removed except with t64 at the end.

The details#

Note on package manager commands#

I actually usually use wajig not apt because it saves needing to remember which of apt-get, apt-cache, dpkg, etc. I need to use and it automatically uses sudo for whichever commands require it. apt handles some, but not all, of that, and I'm used to typing wajig, but I'll use the more standard commands in this post and omit the sudo for brevity.

Figuring out what libssl3t64 is#

Inspecting the package#

My first step was try to get information on libssl3t64. apt show libssl3t64 had an identical description to apt show libssl3. Then I used wajig listfiles to see what was actually in the two packages (apparently this is using apt-file, but the output looks slightly different):

$ wajig listfiles libssl3
libssl3: /usr/lib/x86_64-linux-gnu/engines-3/afalg.so
libssl3: /usr/lib/x86_64-linux-gnu/engines-3/loader_attic.so
libssl3: /usr/lib/x86_64-linux-gnu/engines-3/padlock.so
libssl3: /usr/lib/x86_64-linux-gnu/libcrypto.so.3
libssl3: /usr/lib/x86_64-linux-gnu/libssl.so.3
libssl3: /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so
libssl3: /usr/share/doc/libssl3/NEWS.Debian.gz
libssl3: /usr/share/doc/libssl3/changelog.Debian.gz
libssl3: /usr/share/doc/libssl3/changelog.gz
libssl3: /usr/share/doc/libssl3/copyright

$ wajig listfiles libssl3t64
libssl3t64: /usr/lib/x86_64-linux-gnu/engines-3/afalg.so
libssl3t64: /usr/lib/x86_64-linux-gnu/engines-3/loader_attic.so
libssl3t64: /usr/lib/x86_64-linux-gnu/engines-3/padlock.so
libssl3t64: /usr/lib/x86_64-linux-gnu/libcrypto.so.3
libssl3t64: /usr/lib/x86_64-linux-gnu/libssl.so.3
libssl3t64: /usr/lib/x86_64-linux-gnu/ossl-modules/legacy.so
libssl3t64: /usr/share/doc/libssl3t64/NEWS.Debian.gz
libssl3t64: /usr/share/doc/libssl3t64/changelog.Debian.gz
libssl3t64: /usr/share/doc/libssl3t64/changelog.gz
libssl3t64: /usr/share/doc/libssl3t64/copyright
libssl3t64: /usr/share/lintian/overrides/libssl3t64

# This is the only different filename:
$ cat /usr/share/lintian/overrides/libssl3t64
libssl3t64: package-name-doesnt-match-sonames libssl3

That made it clear that the packages really were doing the same thing, so I hadn't broken anything by replacing libssl3 with libssl3t64, but it didn't explain why.

Searching online#

In order to determine what had happened, I at first tried to look at the Debian package information for libssl3 and libssl3t64, as well as going from there to looking at the bugs and recent mailing list archives. There was one recent bug mentioning the "the t64 transition", but without knowing what that meant, I didn't realize it wasn't something libssl-specific.

After that led nowhere, I tried some web searches for "libssl3 libssl3t64", which eventually led me to this GitHub issue mentioning "64-bit time" and linking to Debian's 64bit-time page which explained what was going on. Although not until after giving up on using DuckDuckGo and searching with Google instead.

Conflict resolution strategy#

The straightforward way to upgrade a Debian system is apt full-upgrade (formerly apt dist-upgrade, which still works). It will try to upgrade all of the packages, installing additional dependencies and removing conflicting packages if necessary. Sometimes it works without needing any extra work, but I always carefully read the list of packages it intends to remove and cancel if it's planning to remove something I want to keep. The apt upgrade command is less aggressive but will simply not upgrade any packages that require other packages to be installed or removed. So it is generally safe to run, but if there's a lot of conflicts, it won't do anything.

Generally if there's a lot of packages to upgrade, trying to dist-upgrade/full-upgrade everything at once is just too much noise, even if it would do the right thing. So instead I try to do rounds of apt upgrade, look at the list of held-back packages, and select one to try to upgrade with apt install foo.

Choosing a package to install#

Generally it's easiest to resolve issues with packages that have fewer dependencies. One way to select such a package is to just guess what package seems simple and see if trying to install it really does get an easy to deal with output from apt. If the package is making apt recommend uninstalling a lot of packages, then I might select one of the packages apt is recommending installing/upgrading and try upgrading just that package. Last, if apt outputs a message saying a dependency package will not be installed, explicitly requesting to install that dependency is likely a good way to make progress on resolving the issues.

One catch is that running apt install foo (sometimes?) marks the package foo as "manually installed", so it will not get removed automatically if the package that depends on it is later uninstalled or changed so it no longer depends on it. To fix that, use apt-mark: run apt-mark auto foo after successfully installing foo to set to back to "automatically installed".

Reviewing packages to remove/install#

Once I have an apt install run with few enough packages being removed that I'm willing to read through them, I'm mainly looking to verify they are all libraries, in which case I can generally trust the package manager is doing the right thing. While almost all packages whose names start with "lib" are libraries, do remember to watch out for libreoffice (better known as LibreOffice).

For any non-libary packages being removed, espeically if I recognize them as something I use, the apt show command will give more information on a package. Sometimes a package will look like one I think should be installed, but apt show will include a message like

This is a transitional package that can safely be removed.

indicating that it is being removed because the actual functionality has moved to a different package (or multiple different packages).

That information would also have helped answer my question about libssl3t64. apt show libssl3t64 includes the line

Provides: libssl3 (= 3.2.1-3)

meaning that the package manager should treat it as basically being the same package.

Force install/removal of packages#

If there's unexpected packages to be removed or installed, one option is modify the request to apt. If apt install foo installs bar and removes baz and I don't want that to happen, I can try running apt install foo bar- baz (notice the minus sign (-) after bar meaning to remove it, not install it). Either apt will come back with a solution that installs the requested packages and not those requested to be removed, or it will show a message about conflicts that may be useful in deciding what to do next.

Resolving conflicts with aptitude#

First, I had not used it before, so I don't know aptitude very well. The offical documentation is a better place to learn how it to use it.

But to solve my problem it was sufficient to use aptitude install instead of apt install and when it proposed a resolution, check if it really was just replacing the libraries with the t64 variants. Generally the first suggestion did not do that, but often I was able to reject a small number of solutions by typing n at the prompt

Accept this solution? [Y/n/q/?] 

a few times and it would propose the desired solution. Note that it would also happily propose nonsense solutions like resolving the conflicts from upgrading a package by not upgrading the package. Luckily, after accepting the solution, it shows it again in a format like apt install explicitly listing which packages will be installed/removed/upgraded and asks for another confirmation before actually doing anything.

Update: APT 3.0 improvements#

UPDATE [2024-05-17]: Soon after I originally published this, a blog post was published on "The new APT 3.0 solver", which sounds like it may make these kinds of upgrades go smoother in the future once it is released.