mark :: blog :: redhat

20 Sep 2012: Red Hat Security Blog

We now have an official Red Hat Security Blog, and you'll find all my future reports and discussions about security metrics there. In the meantime here are a few already published articles:

18 May 2012: Red Hat and CVRF compatibility

The Common Vulnerability Reporting Framework (CVRF) is a way to share information about security updates in an XML machine-readable format. CVRF 1.1 got released this week and over at Red Hat we've started publishing our security advisories in CVRF format.

Find out more from our FAQ and formatting guide.

18 May 2011: Red Hat Security Advisories in CVRF

Earlier this year, Red Hat joined the Common Vulnerability Reporting Framework (CVRF) working group run by ICASI. The goal of CVRF is to provide a way to share information about security updates in a machine-readable format. Red Hat already produce a version of our security advisories in machine readable format, as OVAL definitions, but these are really designed for automated test tools to determine the need to apply an update. CVRF looked like it would be more useful for providing customers with a machine readable view of our advisories.

After many iterations at getting right schema, CVRF 1.0 was released this week.

Red Hat is not yet providing an official archive with CVRF representations of our advisories, but we have created tools internally to support it and allow us to automatically create CVRF documents based on our advisory database. We've provided a sample set of advisories in CVRF format for download.

Let's take one of these samples, a recent Red Hat Enterprise Linux security advisory and examine how it looks when automatically converted to a CVRF document. Our advisories often fix more than one vulnerability at a time and for more than one version of a product, but for this example we'll keep it simple. RHSA-2010:0888 is an Enterprise Linux 6 update to fix the vulnerability CVE-2010-3864 affecting OpenSSL.

The CVRF for this advisory starts like this:

<?xml version="1.0" encoding="utf-8"?>
<cvrfdoc xmlns="https://www.icasi.org/CVRF/schema/cvrf/1.0"
            xmlns:cvrf="https://www.icasi.org/CVRF/schema/cvrf/1.0" 
            xmlns:cvrf-vuln="https://www.icasi.org/CVRF/schema/vuln/1.0">
  <DocumentTitle xml:lang="en">Red Hat Security Advisory: openssl security update</DocumentTitle>
  <DocumentDistribution xml:lang="en">Copyright © 2011 Red Hat, Inc. All rights reserved.</DocumentDistribution>
  <LegalDisclaimer xml:lang="en">Please see https://www.redhat.com/legal/legal_statement.html</LegalDisclaimer>
  <DocumentType>Security Advisory</DocumentType>
  <DocumentPublisher>Vendor</DocumentPublisher>

The first section, above, is pretty straightforward: we're publishing this CVRF as the authoritative vendor for Red Hat Enterprise Linux, and it's a Security Advisory. All our text fields have a language identifier ("en") as future CVRF advisories could contain localized parts.

  <DocumentTracking>
    <DocumentID>RHSA-2010:0888</DocumentID>
    <DocumentStatus>Final</DocumentStatus>
    <DocumentVersion>1</DocumentVersion>
    <DocumentRevisionHistory>
       <DocumentRevision>
         <RevisionNumber>1.0</RevisionNumber>
         <RevisionDate>2010-11-16T16:08:00+00:00</RevisionDate>
         <RevisionDescription>Current version</RevisionDescription>
       </DocumentRevision>
    </DocumentRevisionHistory>
    <DocumentInitialReleaseDate>2010-11-16T16:08:00+00:00</DocumentInitialReleaseDate>
    <DocumentCurrentReleaseDate>2010-11-16T16:08:00+00:00</DocumentCurrentReleaseDate>

The next section is about document revision history and is a mandatory requirement of CVRF documents. This is tricky for Red Hat to automatically generate from our existing advisories: we use an internal "Push Count" as our revision number, and occasional problems during pushing an advisory live mean that our first real public version is not "1". In addition, when we do release an update to an advisory, we update the description text with details of the changes made, so we don't have this text stored in a separate field. So for now our CVRF "RevisionDescription" sections will not have useful descriptions, but the initial and current release dates will be accurate.

    <DocumentGenerator>
      <Generator>make-cvrf-from-et.pl</Generator>
      <GenerationDate>2010-12-15T10:15:06+00:00</GenerationDate>
      <CVRFVersion>1.00</CVRFVersion>
    </DocumentGenerator>
  </DocumentTracking>

During the design of CVRF we championed this separate "DocumentGenerator" section, which mirrors a similar section in OVAL. This can help us track down any errors in published documents and allows us to regenerate a document if our automated script is altered or fixed without causing a new document revision.

  <DocumentSummary>
    <Summary Title="Topic" Audience="General" xml:lang="en">
Updated openssl packages that fix one security issue are now available for
Red Hat Enterprise Linux 6.

The Red Hat Security Response Team has rated this update as having
important security impact. A Common Vulnerability Scoring System (CVSS)
base score, which gives a detailed severity rating, is available from the
CVE link in the References section.
</Summary>
  </DocumentSummary>
  <DocumentDetails>
    <Details Title="Details" Audience="General" xml:lang="en">
OpenSSL is a toolkit that implements the Secure Sockets Layer (SSL v2/v3)
and Transport Layer Security (TLS v1) protocols, as well as a
full-strength, general purpose cryptography library.

A race condition flaw has been found in the OpenSSL TLS server extension
parsing code, which could affect some multithreaded OpenSSL applications.
Under certain specific conditions, it may be possible for a remote attacker
to trigger this race condition and cause such an application to crash, or
possibly execute arbitrary code with the permissions of the application.
(CVE-2010-3864)

Note that this issue does not affect the Apache HTTP Server. Refer to Red
Hat Bugzilla bug 649304 for more technical details on how to determine if
your application is affected.

Red Hat would like to thank Rob Hulswit for reporting this issue.

All OpenSSL users should upgrade to these updated packages, which contain a
backported patch to resolve this issue. For the update to take effect, all
services linked to the OpenSSL library must be restarted, or the system
rebooted.
</Details>
  </DocumentDetails>

The document summary and details sections above are directly copied from the equivalent sections in our text advisory. This does mean however that the details of the vulnerabilities will get repeated, as later each vulnerability gets its own section with description, CVE name, and acknowledgments.

  <References>
    <RelatedDocument>
       <DocumentURL>https://rhn.redhat.com/errata/RHSA-2010-0888.html</DocumentURL>
       <DocumentDescription>https://rhn.redhat.com/errata/RHSA-2010-0888.html</DocumentDescription>
    </RelatedDocument>
    <RelatedDocument>
       <DocumentURL>https://www.redhat.com/security/updates/classification/#important</DocumentURL>
       <DocumentDescription>https://www.redhat.com/security/updates/classification/#important</DocumentDescription>
    </RelatedDocument>
  </References>

References can appear both at the top level of a CVRF document as well as for each vulnerability. The most important top level reference is the self-reference which links to the full representation of the advisory. In early CVRF drafts this had a separate attribute so that it was possible to extract the self reference URL; we think that would have been useful to still be there.

  <AggregateSeverity>Important</AggregateSeverity>
  <IssuingAuthority VendorID="Red Hat, Inc">secalert@redhat.com</IssuingAuthority>

The final top level directives give our severity rating and issuing authority. The severity is a free-text field and it's up to each vendor to use their own scheme for this. In the earlier references section we already gave a URL that described the severity levels being used; for a future CVRF version we should have that URL referenced as part of the AggregateSeverity tag.

  <cvrf-vuln:Vulnerability>
    <cvrf-vuln:VulnerabilityID><cvrf-vuln:Value SystemName="CVE">CVE-2010-3864</cvrf-vuln:Value></cvrf-vuln:VulnerabilityID>
    <cvrf-vuln:VulnerabilityDetails xml:lang="en">A race condition flaw
    has been found in the OpenSSL TLS server extension parsing code, which could
    affect some multithreaded OpenSSL applications. Under certain specific
    conditions, it may be possible for a remote attacker to trigger this race
    condition and cause such an application to crash, or possibly execute
    arbitrary code with the permissions of the
    application. </cvrf-vuln:VulnerabilityDetails>

We produce one vulnerability section for each CVE fixed, and the vulnerability details are abstracted from our full text description.

    <cvrf-vuln:VendorRemediationStatus>Completed</cvrf-vuln:VendorRemediationStatus>
    <cvrf-vuln:CVE>CVE-2010-3864</cvrf-vuln:CVE>
    <cvrf-vuln:Threat><cvrf-vuln:Impact>Important</cvrf-vuln:Impact></cvrf-vuln:Threat>
    <cvrf-vuln:ProductFamily>Red Hat Enterprise Linux</cvrf-vuln:ProductFamily>
    <cvrf-vuln:Acknowledgment>Red Hat would like to thank Rob Hulswit for reporting this issue.</cvrf-vuln:Acknowledgment>
    <cvrf-vuln:CVSS>
      <cvrf-vuln:CVSSBaseScore>7.6</cvrf-vuln:CVSSBaseScore>
      <cvrf-vuln:CVSSScoringVector>AV:N/AC:H/Au:N/C:C/I:C/A:C</cvrf-vuln:CVSSScoringVector>
    </cvrf-vuln:CVSS>

VendorRemediationStatus will always be "Completed" for all our advisories as they are always released at the same time as a fix. Also included is a per-vulnerability impact rating and CVSS base score. These are currently not included in our HTML advisories, but are available from our CVE database and Bugzilla entries; so it's handy to be able to have them all in the same document for the first time.

    <cvrf-vuln:Remediation>
      <cvrf-vuln:VendorFix xml:lang="en">
Before applying this update, make sure all previously-released errata
relevant to your system have been applied.

This update is available via the Red Hat Network. Details on how to
use the Red Hat Network to apply this update are available at
https://kbase.redhat.com/faq/docs/DOC-11259
</cvrf-vuln:VendorFix>
    </cvrf-vuln:Remediation>
    <cvrf-vuln:AffectedPlatform Name="Red Hat Enterprise Linux">
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Desktop Optional (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-devel-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-perl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-static-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux HPC Node Optional (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-devel-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-perl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-static-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Workstation Optional (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-perl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-static-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Workstation (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-devel-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux HPC Node (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Server Optional (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-perl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-static-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Desktop (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
      <cvrf-vuln:AffectedRelease Name="Red Hat Enterprise Linux Server (v. 6)">
          <cvrf-vuln:Version Type="Fixed">openssl-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-debuginfo-1.0.0-4.el6_0.1</cvrf-vuln:Version>
          <cvrf-vuln:Version Type="Fixed">openssl-devel-1.0.0-4.el6_0.1</cvrf-vuln:Version>
      </cvrf-vuln:AffectedRelease>
    </cvrf-vuln:AffectedPlatform>

The remediation section mirrors our advisories by giving a list of packages and the versions that first contained the fix for the issue. We have multiple variants of Red Hat Enterprise Linux 6, and some vulnerabilities will affect packages only shipped in some subset of variants, so the duplication is required.

    <cvrf-vuln:References>
      <cvrf-vuln:RelatedDocument>
        <cvrf-vuln:DocumentURL>https://www.redhat.com/security/data/cve/CVE-2010-3864.html</cvrf-vuln:DocumentURL>
        <cvrf-vuln:DocumentDescription>CVE-2010-3864</cvrf-vuln:DocumentDescription>
      </cvrf-vuln:RelatedDocument>
      <cvrf-vuln:RelatedDocument>
        <cvrf-vuln:DocumentURL>https://bugzilla.redhat.com/show_bug.cgi?id=649304</cvrf-vuln:DocumentURL>
        <cvrf-vuln:DocumentDescription>bz#649304: CVE-2010-3864 OpenSSL TLS extension parsing race condition</cvrf-vuln:DocumentDescription>
      </cvrf-vuln:RelatedDocument>
    </cvrf-vuln:References>
  </cvrf-vuln:Vulnerability>
</cvrfdoc>

Finally, the per-vulnerability reference section links to our CVE database for each CVE, and the Red Hat bug database for more technical details of the vulnerability and how it was addressed.

Our example advisory only had one vulnerability, but the whole vulnerability section is repeated where multiple are addressed.

Overall we've been really pleased with the way CVRF has turned out for it's first release, and we think it will be a useful way for vendors such as us to provide machine-readable advisories to customers as well as to tool vendors, filling a gap between our OVAL XML and our text and HTML advisories. We'll be trialling CVRF advisories later this year, drop an email to secalert@redhat.com if you have comments on our CVRF samples.

11 Nov 2010: Why Red Hat Enterprise Linux 6 has a new package signing key

Starting with Red Hat Enterprise Linux 6 we have switched to using SHA-256 signatures on all RPM packages and to a 4096-bit RSA signing key.

We've done this because it is current best practice to migrate away from MD5 and SHA-1 hashes due to various flaws found in them. Those flaws don't yet directly pose a threat to package signing however, and therefore our existing shipped products which used these older hashes will continue to use their existing keys until they reach their end of life.

A similar switch to stronger signing was already made in Fedora 11. This switch involved some changes to the RPM application.

So what this means is that we used new signing keys for both the beta and final release packages for Red Hat Enterprise Linux 6. Those keys were created and are protected by a hardware security module, as we've done with previous keys.

Details and fingerprint of the new key, #fd431d51.

Also in the Red Hat Enterprise Linux 6 distribution we've started to simplify the layout of the key files in the /etc/pki/rpm-gpg/ directory:

RPM-GPG-KEY-redhat-beta : Both the old and new beta keys
RPM-GPG-KEY-redhat-release : Both the new signing key and the auxiliary key
RPM-GPG-KEY-redhat-legacy-release : The signing key used for EL5
RPM-GPG-KEY-redhat-legacy-former : The signing key used for products before EL5
RPM-GPG-KEY-redhat-legacy-rhx : The signing key used for RHX

The auxiliary key mentioned above is for emergency use. We created it some time ago on a new standalone machine, took a hardcopy printout of the private key and passphrase, stored them separately and securely, and destroyed the software copies. We've planned for many eventualities, but in the unlikely event we lose the ability to sign with the hardware key we could retrieve the printout, type in the key, and continue to sign updates.

07 Oct 2009: CVE-2009-2408 (NULL in certificate) timeline

There have been quite a few stories over the last couple of weeks about the NULL character certificate flaw, such as this one from The Register.

The stories center around how open source software such as Firefox was able to produce updates to correct this issue just a few days after the Blackhat conference, while Microsoft still hasn't fixed it and are "investigating a possible vulnerability in Windows presented during Black Hat".

But the actual timeline is missing from these stories.

The NULL character certificate flaw (CVE-2009-2408) was actually disclosed by two researchers working independantly who both happened to present the work at the same conference, Blackhat, in July this year. Dan Kaminsky mentioned it as part of a series of PKI flaws he disclosed. Marlinspike had found the same flaw, but was able to demonstrate it in practice by managing to get a trusted Certificate Authority to sign such a malicious certificate.

The flaw was no Blackhat surprise; Dan Kaminsky actually found this issue many months ago and responsibly reported the issues to vendors including Red Hat, Microsoft, and Mozilla. We found out about this issue on 25th February 2009 and worked with Dan and some of the upstream projects on these issues in advance, so we had plenty of time to prepare updates and this is why we were able to have them ready to release just after the disclosure.

03 Mar 2009: Source of Vulnerabilities

From time to time I publish metrics on vulnerabilities that affect Red Hat Enterprise Linux. One of the more interesting metrics looks at how far in advance we know about the vulnerabilities we fix, and from where we get that information. This post is abstracted from the upcoming "4 years of Enterprise Linux 4" risk report

For every fixed vulnerability across every package and every severity in Enterprise Linux 4 AS in the first 4 years of its life, we determined if the flaw was something we knew about a day or more in advance of it being publicly disclosed, and how we found out about the flaw.

A graph showing the information sources

For vulnerabilities which are already public when we first hear about them we still track the source as it's a useful internal indicator on where the security response team should focus their efforts.

A graph showing the information sources

So from this data, Red Hat knew about 51% of the security vulnerabilities that we fixed at least a day in advance of them being publicly disclosed. For those issues, the average notice was 21 calendar days, although the median was much lower, with half the private issues having advance notice of 9 days or less.

A graph showing the information
sources