mark :: blog :: fedora

[ 1 ]

We're changing the package signing key we use for all new Red Hat products.

Since 1999, all RPM packages in Red Hat products have been gpg signed by the master key "Red Hat, Inc <>" (keyid DB42A60E). I'll call this the legacy signing key for the rest of this article. This signature is one of two security mechanisms we use to ensure that customers can trust the installation of packages and their updates. The other is that the update client, up2date, checks the SSL server signature when it connects to the Red Hat Network to ensure that it only talks to official Red Hat servers; so removing the possibility of a man-in-the-middle attack.

From 2007, all new products will be signed with a different master key, "Red Hat, Inc. (release key) <>" (keyid 37017186). This includes Red Hat Enterprise Linux 5, and any other new products that use RPM packages. The exception to this rule is that any new layered products designed for older versions of Enterprise Linux will still use the legacy key: so for example, a new version of the Application Stack for Red Hat Enterprise Linux 4 will be signed with the legacy key.

The legacy key hasn't been compromised so why change keys? It's all to do with the way the keys are stored and managed internal to Red Hat. The legacy key is a software key and so the key material exists, protected by a passphrase, on a hard disk. When packages need to be signed one of the Red Hat authorised signers manually runs a signing command, this calls rpm --resign which asks for the passphrase then in turn calls out to GNUpg to do the actual signature creation. So the authorised signers not only had the ability to sign with the key, but they also have the ability to read the key material. In theory this means that a malicious internal signer could copy the key, take it away with them, and sign whatever and whenever they wanted. Or, more likely, a clever intruder who gained access to our internal network could perhaps capture the key and passphrase, compromising the key. The risks mean we've had to be really careful who has signing privileges with the legacy key and how the key signing is handled.

The new key, in contrast, was created in a hardware cryptographic device which does not allow the unprotected key material to be exported. This means we can give authorised signers the ability to sign with the key, but no one can ever can get access to the key material itself. This is an important distinction. If for example a current authorised signer switches roles and is no longer responsible for package signing we can instantly revoke their rights and know that they no longer have the ability to sign any more packages with that key.

There was no off-the-shelf solution available for hardware-based RPM key management, so we developed one internally ourselves. We used nCipher nShield hardware security modules (FIPS 140-2 validated) for the key protection along with custom patches I developed to interface RPM/GNUpg to the unit. At the same time we also introduced an extra layer of abstraction to the signing software, so we can authorize signers using their existing internal kerberos credentials.

So, as a customer, you won't really notice any difference. For Red Hat Enterprise Linux 5 you'll find the public keys on our website as well as in the /etc/pki/rpm-gpg/ directory and you'll be prompted when updating or installing new packages for the first time to import that new public key.

This change basically makes it easier for us to protect our signing key and reduce the risk of it being compromised, therefore reducing the chances we'll need to change the key and involve customer effort in the future.

Late last year a reporter contacted me who was interested in the various security features and innovations in Red Hat Enterprise Linux and Fedora. She particularly wanted to know the dates when each first made it into a shipping product. In the end the article was published in a German magazine and was not publically available. It's a shame to waste the work as I don't think this has ever all been collected together into one place before, so here is the table. It's possible I've missed one or two of the features, and I've not broken down the big things like SELinux where we could talk about the number of default policies in each release or the number of binaries compiled PIE, but drop me a mail if you see any issues.

  Fedora Core Red Hat Enterprise Linux
123456 34
2003Nov2004May2004Nov2005Jun2006Mar2006Oct 2003Oct2005Feb
Default requires signed updates YYYYYY YY
NX emulation using segment limits by default YYYYYY since 2004SepY
Support for Position Independent Executables (PIE) YYYYYY since 2004SepY
ASLR for Stack/mmap by default YYYYYY since 2004SepY
ASLR for vDSO (if vDSO enabled) no vDSOYYYYY no vDSOY
Restricted access to kernel memory by default  YYYYY  Y
NX by default for supported processors/kernels  since 2004JunYYYY since 2004SepY
Support for SELinux  YYYYY  Y
SELinux default enabled with targetted policies   YYYY  Y
glibc heap/memory checks by default   YYYY  Y
Support for FORTIFY_SOURCE, used on selected packages   YYYY  Y
All packages compiled using FORTIFY_SOURCE    YYY   
Support for ELF Data Hardening    YYY  Y
All packages compiled with stack smashing protection     YY   
Pointer encryption      Y   
CVE compatible        YY
OVAL compatible        since 2006Maysince 2006May

New: Updated version from 7th January 2008

[ 1 ]

Hi! I'm Mark Cox. This blog gives my thoughts and opinions on my security work, open source, fedora, home automation, and other topics.