Debian is going to make an important decision regarding non-free
firmaware.
---------- Forwarded message ----------
From: Steve McIntyre <steve(a)einval.com>
Subject: Firmware - what are we going to do about it?
Date: 2022-04-19T01:27:46+0100
To: debian-devel(a)lists.debian.org
TL;DR: firmware support in Debian sucks, and we need to change this.
See the
"My preference, and rationale" Section below.
In my opinion, the way we deal with (non-free) firmware in Debian is a
mess,
and this is hurting many of our users daily. For a long time we've been
pretending that supporting and including (non-free) firmware on Debian
systems
is not necessary. We don't want to have to provide (non-free) firmware
to our
users, and in an ideal world we wouldn't need to. However, it's very
clearly no
longer a sensible path when trying to support lots of common current
hardware.
Background - why has (non-free) firmware become an issue?
=========================================================
Firmware is the low-level software that's designed to make hardware
devices
work. Firmware is tightly coupled to the hardware, exposing its
features,
providing higher-level functionality and interfaces for other software
to use.
For a variety of reasons, it's typically not Free Software.
For Debian's purposes, we typically separate firmware from software by
considering where the code executes (does it run on a separate
processor? Is it
visible to the host OS?) but it can be difficult to define a single
reliable
dividing line here. Consider the Intel/AMD CPU microcode packages, or
the
U-Boot firmware packages as examples.
In times past, all necessary firmware would normally be included
directly in
devices / expansion cards by their vendors. Over time, however, it has
become
more and more attractive (and therefore more common) for device
manufacturers
to not include complete firmware on all devices. Instead, some devices
just
embed a very simple set of firmware that allows for upload of a more
complete
firmware "blob" into memory. Device drivers are then expected to
provide that
blob during device initialisation.
There are a couple of key drivers for this change:
• Cost: it's typically cheaper to fit smaller flash memory (or no
flash at
all) onto a device. The cost difference may seem small in many
cases, but
reducing the bill of materials (BOM) even by a few cents can make a
substantial difference to the economics of a product. For most
vendors,
they will have to implement device drivers anyway and it's not
difficult to
include firmware in that driver.
• Flexibility: it's much easier to change the behaviour of a device
by simply
changing to a different blob. This can potentially cover lots of
different
use cases:
□ separating deadlines for hardware and software in
manufacturing
(drivers and firmware can be written and shipped later);
□ bug fixes and security updates (e.g. CPU microcode changes);
□ changing configuration of a device for different users or
products
(e.g. potentially different firmware to enable different
frequencies on
a radio product);
□ changing fundamental device operation (e.g. switching between
RAID and
JBOD functionality on a disk controller).
Due to these reasons, more and more devices in a typical computer now
need
firmware to be uploaded at runtime for them to function correctly. This
has
grown:
• Going back 10 years or so, most computers only needed firmware
uploads to
make WiFi hardware work.
• A growing number of wired network adapters now demand firmware
uploads.
Some will work in a limited way but depend on extra firmware to
allow
advanced features like TCP segmentation offload (TSO). Others will
refuse
to work at all without a firmware upload.
• More and more graphics adapters now also want firmware uploads to
provide
any non-basic functions. A simple basic (S)VGA-compatible
framebuffer is
not enough for most users these days; modern desktops expect 3D
acceleration, and a lot of current hardware will not provide that
without
extra firmware.
• Current generations of common Intel-based laptops also need
firmware
uploads to make audio work (see the firmware-sof-signed package).
At the beginning of this timeline, a typical Debian user would be able
to use
almost all of their computer's hardware without needing any firmware
blobs. It
might have been inconvenient to not be able to use the WiFi, but most
laptops
had wired ethernet anyway. The WiFi could always be enabled and
configured
after installation.
Today, a user with a new laptop from most vendors will struggle to use
it at
all with our firmware-free Debian installation media. Modern laptops
normally
don't come with wired ethernet now. There won't be any usable graphics
on the
laptop's screen. A visually-impaired user won't get any audio prompts.
These
experiences are not acceptable, by any measure. There are new computers
still
available for purchase today which don't need firmware to be uploaded,
but they
are growing less and less common.
Current state of firmware in Debian
===================================
For clarity: obviously not all devices need extra firmware uploading
like this.
There are many devices that depend on firmware for operation, but we
never have
to think about them in normal circumstances. The code is not likely to
be Free
Software, but it's not something that we in Debian must spend our time
on as
we're not distributing that code ourselves. Our problems come when our
user
needs extra firmware to make their computer work, and they need/expect
us to
provide it.
We have a small set of Free firmware binaries included in Debian main,
and
these are included on our installation and live media. This is great -
we all
love Free Software and this works.
However, there are many more firmware binaries that are not Free. If we
are
legally able to redistribute those binaries, we package them up and
include
them in the non-free section of the archive. As Free Software
developers, we
don't like providing or supporting non-free software for our users, but
we
acknowledge that it's sometimes a necessary thing for them. This
tension is
acknowledged in the Debian Free Software Guidelines.
This tension extends to our installation and live media. As non-free is
officially not considered part of Debian, our official media cannot
include
anything from non-free. This has been a deliberate policy for many
years.
Instead, we have for some time been building a limited parallel set of
"unofficial non-free" images which include non-free firmware. These
non-free
images are produced by the same software that we use for the official
images,
and by the same team.
There are a number of issues here that make developers and users
unhappy:
1. Building, testing and publishing two sets of images takes more
effort.
2. We don't really want to be providing non-free images at all, from a
philosophy point of view. So we mainly promote and advertise the
preferred
official free images. That can be a cause of confusion for users.
We do
link to the non-free images in various places, but they're not so
easy to
find.
3. Using non-free installation media will cause more installations to
use
non-free software by default. That's not a great story for us, and
we may
end up with more of our users using non-free software and believing
that
it's all part of Debian.
4. A number of users and developers complain that we're wasting their
time by
publishing official images that are just not useful for a lot (a
majority?)
of users.
We should do better than this.
Options
=======
The status quo is a mess, and I believe we can and should do things
differently.
I see several possible options that the images team can choose from
here.
However, several of these options could undermine the principles of
Debian. We
don't want to make fundamental changes like that without the clear
backing of
the wider project. That's why I'm writing this...
1. Keep the existing setup. It's horrible, but maybe it's the best we
can do?
(I hope not!)
2. We could just stop providing the non-free unofficial images
altogether.
That's not really a promising route to follow - we'd be making it
even
harder for users to install our software. While ideologically pure,
it's
not going to advance the cause of Free Software.
3. We could stop pretending that the non-free images are unofficial,
and maybe
move them alongside the normal free images so they're published
together.
This would make them easier to find for people that need them, but
is
likely to cause users to question why we still make any images
without
firmware if they're otherwise identical.
4. The images team technically could simply include non-free into the
official
images, and add firmware packages to the input lists for those
images.
However, that would still leave us with problem 3 from above
(non-free
generally enabled on most installations).
5. We could split out the non-free firmware packages into a new
non-free-firmware component in the archive, and allow a specific
exception
only to allow inclusion of those packages on our official media. We
would
then generate only one set of official media, including those
non-free
firmware packages.
(We've already seen various suggestions in recent years to split up
the
non-free component of the archive like this, for example into
non-free-firmware, non-free-doc, non-free-drivers, etc. Disagreement
(bike-shedding?) about the split caused us to not make any progress
on
this. I believe this project should be picked up and completed. We
don't
have to make a perfect solution here immediately, just something
that works
well enough for our needs today. We can always tweak and improve
the setup
incrementally if that's needed.)
These are the most likely possible options, in my opinion. If you have
a better
suggestion, please let us know!
I'd like to take this set of options to a GR, and do it soon. I want to
get a
clear decision from the wider Debian project as to how to organise
firmware and
installation images. If we do end up changing how we do things, I want
a clear
mandate from the project to do that.
My preference, and rationale
============================
Mainly, I want to see how the project as a whole feels here - this is a
big
issue that we're overdue solving.
What would I choose to do? My personal preference would be to go with
option 5:
split the non-free firmware into a special new component and include
that on
official media.
Does that make me a sellout? I don't think so. I've been passionately
supporting and developing Free Software for more than half my life. My
philosophy here has not changed. However, this is a complex and nuanced
situation. I firmly believe that sharing software freedom with our
users comes
with a responsibility to also make our software useful. If users can't
easily
install and use Debian, that helps nobody.
By splitting things out here, we would enable users to install and use
Debian
on their hardware, without promoting/pushing higher-level non-free
software in
general. I think that's a reasonable compromise. This is simply a
change to
recognise that hardware requirements have moved on over the years.
Further work
============
If we do go with the changes in option 5, there are other things we
could do
here for better control of and information about non-free firmware:
1. Along with adding non-free firmware onto media, when the installer
(or live
image) runs, we should make it clear exactly which firmware
packages have
been used/installed to support detected hardware. We could link to
docs
about each, and maybe also to projects working on Free
re-implementations.
2. Add an option at boot to explicitly disable the use of the non-free
firmware packages, so that users can choose to avoid them.
Acknowledgements
================
Thanks to people who reviewed earlier versions of this document and/or
made
suggestions for improvement, in particular:
• Cyril Brulebois
• Matthew Garrett
• David Leggett
• Martin Michlmayr
• Andy Simpkins
• Neil Williams
--
Steve McIntyre, Cambridge, UK.
steve(a)einval.com
Who needs computer imagery when you've got Brian Blessed?
