Add unittests for net::TestRootCerts and regenerate test certificates

Regenerate the root certificate and end-entity certificates used for various tests in net_unittests, now that Windows can temporarily trust certificates for the duration of tests.

In addition, add unittests for net::TestRootCerts, which depend on the certs being tested not being trusted by the system beforehand.

BUG=8470, 5552
TEST=TestRootCertsTest.*
TBR=zelidrag

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=207443

[Ryan Sleevi, 2010-12-06 01:47:10]

wtc: here's the regenerated certificates, split into a new CL per your comments
on http://codereview.chromium.org/4646001/

I looked at certificates issued by Thawte, Verisign, and GeoTrust when deciding
on the list of attributes and how to mark them critical vs non-critical.

There is at least one place remaining that depends on the (old) root/EE
certificate, and that's in WebKit:

http://codesearch.google.com/codesearch/p?hl=en#OAMlx_jo-ck/src/third_party/W...

I'm not sure the appropriate way to resolve that, or if it will have any affect
on layout tests. There's also
http://codesearch.google.com/codesearch/p?hl=en#OAMlx_jo-ck/tools/buildbot/sl...
- which suggests there may be one or more UI tests depending on that
certificate. I'm not sure it's usage though, as I can't find any other
references to it.

[wtc, 2010-12-08 19:58:21]

LGTM.  Please do not check this in until the following
issues have been addressed.

1. Please address the comment marked with IMPORTANT below.

2. We'll need to regenerate httpd2.pem used for WebKit
layout tests.  Please try your CL on the layout_win,
layout_mac, layout_linux try bots.

3. I don't know what ssl_ui_test_root_ca.crt is for.
Please look in the SVN history to find out whom we should
ask.

http://codereview.chromium.org/5535006/diff/5001/net/base/test_root_certs_uni...
File net/base/test_root_certs_unittest.cc (right):

http://codereview.chromium.org/5535006/diff/5001/net/base/test_root_certs_uni...
net/base/test_root_certs_unittest.cc:64: TEST(TestRootCertsTest, OverridesTrust)
{
Nit: OverridesTrust => OverrideTrust
for consistency with the two unit test names above.

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/RE...
File net/data/ssl/certificates/README (right):

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/RE...
net/data/ssl/certificates/README:37: - root_ca_cert.crt : The testing root CA
used to sign the test
Typo: root_ca_cert.crt => root_ca_cert.pem

Please note that root_ca_cert.pem also contains the private
key.

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
File net/data/ssl/certificates/expired_cert.pem (right):

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
net/data/ssl/certificates/expired_cert.pem:34: TLS Web Server Authentication,
TLS Web Client Authentication
We should not need TLS Web Client Authentication, but
I see that many real server certs have TLS Web Client
Authentication, too.  Do you know why?

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
net/data/ssl/certificates/expired_cert.pem:61:
BOb9LqmU+IpO/6+JrEdMhXkZ8GgyTKrAamyf3E1Q
Do you know why the new cert is much shorter?  Just curious.

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/op...
File net/data/ssl/certificates/openssl_cert.cnf (right):

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/op...
net/data/ssl/certificates/openssl_cert.cnf:18: # TODO(wtc): Deprecate de facto,
but not de jure, matching of iPAddress in
Are you sure?  I think we can put
  CN = Test Server
here.

http://codereview.chromium.org/5535006/diff/5001/tools/valgrind/valgrind_test.py
File tools/valgrind/valgrind_test.py (left):

http://codereview.chromium.org/5535006/diff/5001/tools/valgrind/valgrind_test...
tools/valgrind/valgrind_test.py:337: logging.warning('sudo
update-ca-certificates')
IMPORTANT: I don't know what this code is for.  Is it safe
to remove it?

Since the root_ca_cert.crt file is renamed, we'll need to at
least update the file name.

Similarly for the removal of the equivalent code in
tools/wine_valgrind/chrome_tests.sh.

[Ryan Sleevi, 2010-12-09 05:49:55]

+cc joth, as an FYI since I'm adding a TODO(joth)

+review maruel & nsylvain, as I'm going to hold off committing until I hear from
one of them regarding ssl_ui_test_root.crt SVN history for the file shows only
three touches: nsylvain and maruel moving the file, and initial.commit
importing. maruel, nsylvain, could you explain how this file is used?

+review dpranke to see if the the WebKit cert needs to change, as he's been the
one touching it upstream (see
http://trac.webkit.org/log/trunk/WebKitTools/Scripts/webkitpy/layout_tests/po...
). The comment on the initial commit,
http://trac.webkit.org/changeset/54091/trunk/WebKitTools/Scripts/webkitpy/lay...
, suggests that the goal is to eventually use the same cert the normal WebKit
tests uses, and *NOT* the Chromium specific test cert. dpranke, can you confirm?

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
File net/data/ssl/certificates/expired_cert.pem (right):

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
net/data/ssl/certificates/expired_cert.pem:34: TLS Web Server Authentication,
TLS Web Client Authentication
On 2010/12/08 19:58:22, wtc wrote:
> We should not need TLS Web Client Authentication, but
> I see that many real server certs have TLS Web Client
> Authentication, too.  Do you know why?

I'm not certain. I've assumed it's for protocols that require server-to-server
communication, in particular so that a certificate can be mapped to a peer,
whether the peer is acting as the TLS server or as the TLS client.

I agree that we should not need it, but added it to match real server certs
(which all of the ones I looked at included it, save for encrypted.google.com,
which has no EKU and is thus valid for everything)

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/ex...
net/data/ssl/certificates/expired_cert.pem:61:
BOb9LqmU+IpO/6+JrEdMhXkZ8GgyTKrAamyf3E1Q
On 2010/12/08 19:58:22, wtc wrote:
> Do you know why the new cert is much shorter?  Just curious.

Signature size. The new root is only 1024 bits, because it no longer represents
a sensitive key.

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/op...
File net/data/ssl/certificates/openssl_cert.cnf (right):

http://codereview.chromium.org/5535006/diff/5001/net/data/ssl/certificates/op...
net/data/ssl/certificates/openssl_cert.cnf:18: # TODO(wtc): Deprecate de facto,
but not de jure, matching of iPAddress in
On 2010/12/08 19:58:22, wtc wrote:
> Are you sure?  I think we can put
>   CN = Test Server
> here.

I've updated the comment to reflect the two open bugs related to this:

http://crbug.com/20276 - TODO(wtc) and http://crbug.com/62973 - TODO(joth)

http://codereview.chromium.org/5535006/diff/5001/tools/valgrind/valgrind_test.py
File tools/valgrind/valgrind_test.py (left):

http://codereview.chromium.org/5535006/diff/5001/tools/valgrind/valgrind_test...
tools/valgrind/valgrind_test.py:337: logging.warning('sudo
update-ca-certificates')
On 2010/12/08 19:58:22, wtc wrote:
> IMPORTANT: I don't know what this code is for.  Is it safe
> to remove it?
> 
> Since the root_ca_cert.crt file is renamed, we'll need to at
> least update the file name.
> 
> Similarly for the removal of the equivalent code in
> tools/wine_valgrind/chrome_tests.sh.

Wine's way of implementing trusted root certs. Rather than using the registry,
they iterate through a series of paths, looking for certificate files. This is
akin to the TestRootCerts check that used to exist, just an added layer.

The list of paths is at
http://source.winehq.org/source/dlls/crypt32/rootstore.c#L454

The implementation is at
http://source.winehq.org/source/dlls/crypt32/rootstore.c#L704

[M-A Ruel, 2010-12-09 14:08:49]

On 2010/12/09 05:49:55, Ryan Sleevi wrote:
> +review maruel & nsylvain, as I'm going to hold off committing until I hear
from
> one of them regarding ssl_ui_test_root.crt SVN history for the file shows only
> three touches: nsylvain and maruel moving the file, and initial.commit
> importing. maruel, nsylvain, could you explain how this file is used?

SSL interstitial tests IIRC. Note that there is a copy in
src.chromium.org/viewvc/chrome/trunk/tools/build/slave/cert that we use to
preinstall it on the windows slaves.

[Nicolas Sylvain, 2010-12-09 16:41:12]

On Thu, Dec 9, 2010 at 6:08 AM, <maruel@chromium.org> wrote:

> On 2010/12/09 05:49:55, Ryan Sleevi wrote:
>
>> +review maruel & nsylvain, as I'm going to hold off committing until I
>> hear
>>
> from
>
>> one of them regarding ssl_ui_test_root.crt SVN history for the file shows
>> only
>> three touches: nsylvain and maruel moving the file, and initial.commit
>> importing. maruel, nsylvain, could you explain how this file is used?
>>
>
> SSL interstitial tests IIRC. Note that there is a copy in
> src.chromium.org/viewvc/chrome/trunk/tools/build/slave/cert that we use to
> preinstall it on the windows slaves.


ssl_ui_test_root.crt is just a copy of the one in net i believe.  You should
confirm, but it should be the same thing. It would be good to update that
one too..

that said... the bots will need the new cert... and installing a new cert is
a heavily manual process. We will need to connect to our 200 bots one by one
to install it... and until that, your tests would fail.  I'm not sure how to
proceed here. Do we really need to change the cert? If so, we should move
this discussion to chrome-infrastructure-team and drop some of the CCs...
we'll need to do some planning.

Nicolas


>
> http://codereview.chromium.org/5535006/
>

[Ryan Sleevi, 2010-12-09 16:44:27]

nsylvain:

Sorry I wasn't clear in the original +review : This is actually a second part of
an existing CL, http://codereview.chromium.org/4646001/ , which makes it no
longer necessary to add/update the Windows bots with the root cert.

Am I understanding correctly that the purpose of the file is to make it
available for pre-install? In which case, it should be able to be deleted, as
that step is no longer necessary.

[Nicolas Sylvain, 2010-12-09 16:57:23]

On Thu, Dec 9, 2010 at 8:44 AM, <rsleevi@chromium.org> wrote:

> nsylvain:
>
> Sorry I wasn't clear in the original +review : This is actually a second
> part of
> an existing CL, http://codereview.chromium.org/4646001/ , which makes it
> no
> longer necessary to add/update the Windows bots with the root cert.
>
> Am I understanding correctly that the purpose of the file is to make it
> available for pre-install? In which case, it should be able to be deleted,
> as
> that step is no longer necessary.


Yes, it can be deleted!

Thanks!

Nicolas


>
> http://codereview.chromium.org/5535006/
>

[M-A Ruel, 2010-12-13 16:22:57]

lgtm

[Ryan Sleevi, 2010-12-14 07:33:15]

ping dpranke: Is the WebKit file used in production? Or is it an upstream
contribution of what Chromium use[s/d]. The try bots are green without touching
it, but my fear would be that it's using the old cert and passing because of the
system configured trust on Win (which this CL plus it's parent seeks to
eliminate).

[Dirk Pranke, 2010-12-14 20:35:20]

Whoops, sorry for missing the comments earlier. I'm 99.8% certain the
upstream code is not using the downstream cert, so you should be free
to remove it.

I had forgotten all about this particular change. I don't see any real
reason why we shouldn't just move to the webkit cert, so I'll try that
out today and see what happens.

-- Dirk

On Mon, Dec 13, 2010 at 11:33 PM,  <rsleevi@chromium.org> wrote:
> ping dpranke: Is the WebKit file used in production? Or is it an upstream
> contribution of what Chromium use[s/d]. The try bots are green without
> touching
> it, but my fear would be that it's using the old cert and passing because of
> the
> system configured trust on Win (which this CL plus it's parent seeks to
> eliminate).
>
> http://codereview.chromium.org/5535006/
>

[Dirk Pranke, 2013-05-24 19:53:55]

cleanup time ... do we care about this patch at this point?

[Ryan Sleevi, 2013-05-24 19:54:29]

On 2013/05/24 19:53:55, Dirk Pranke wrote:
> cleanup time ... do we care about this patch at this point?

Yup! It's not abandoned - but Chrome Frame makes everything harder.

[Ryan Sleevi, 2013-06-17 23:37:07]

So it seems every HTTPS test in Chrome Frame is disabled at the moment.

Given that CF is being retired (
http://blog.chromium.org/2013/06/retiring-chrome-frame.html ), are there any
objections to landing this?

[Dirk Pranke, 2013-06-18 00:05:30]

seems reasonable to me. Do we need to update
LayoutTests/http/conf/webkit-httpd.pem as well?

[Ryan Sleevi, 2013-06-18 00:07:27]

On 2013/06/18 00:05:30, Dirk Pranke wrote:
> seems reasonable to me. Do we need to update
> LayoutTests/http/conf/webkit-httpd.pem as well?

As far as I can tell, this is only used by the Apache config, which Chromium
doesn't support (in favour of using lighttpd). Things seem to be passing on all
the trybots I've tried, as well as running run-webkit-tests - can you think of
anything that might exercise this path?

[Dirk Pranke, 2013-06-18 00:12:31]

On 2013/06/18 00:07:27, Ryan Sleevi wrote:
> On 2013/06/18 00:05:30, Dirk Pranke wrote:
> > seems reasonable to me. Do we need to update
> > LayoutTests/http/conf/webkit-httpd.pem as well?
> 
> As far as I can tell, this is only used by the Apache config, which Chromium
> doesn't support (in favour of using lighttpd). Things seem to be passing on
all
> the trybots I've tried, as well as running run-webkit-tests - can you think of
> anything that might exercise this path?

We use apache on mac and linux. We only use lighttpd on win.

Given that you're not touching the layout tests file, I'm not sure whether I
would expect any of the layout tests to be affected.

[Ryan Sleevi, 2013-06-18 20:46:34]

On 2013/06/18 00:12:31, Dirk Pranke wrote:
> On 2013/06/18 00:07:27, Ryan Sleevi wrote:
> > On 2013/06/18 00:05:30, Dirk Pranke wrote:
> > > seems reasonable to me. Do we need to update
> > > LayoutTests/http/conf/webkit-httpd.pem as well?
> > 
> > As far as I can tell, this is only used by the Apache config, which Chromium
> > doesn't support (in favour of using lighttpd). Things seem to be passing on
> all
> > the trybots I've tried, as well as running run-webkit-tests - can you think
of
> > anything that might exercise this path?
> 
> We use apache on mac and linux. We only use lighttpd on win.
> 
> Given that you're not touching the layout tests file, I'm not sure whether I
> would expect any of the layout tests to be affected.

wtc: Ping?

[wtc, 2013-06-18 22:51:44]

Patch set 13 LGTM. I'm sorry that I didn't know I should
review this CL again.

https://codereview.chromium.org/5535006/diff/113041/net/cert/test_root_certs_...
File net/cert/test_root_certs_android.cc (right):

https://codereview.chromium.org/5535006/diff/113041/net/cert/test_root_certs_...
net/cert/test_root_certs_android.cc:21: empty_ = false;

This is a bug fix, right?

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
File net/data/ssl/certificates/ok_cert.pem (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
net/data/ssl/certificates/ok_cert.pem:37: Not After : Jun 16 19:52:02 2023 GMT

This should ideally be very far into the future. 2023 is only 10 years away.
Note that httpd2.pem is valid until 2060.

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
File net/data/ssl/certificates/root_ca_cert.pem (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
net/data/ssl/certificates/root_ca_cert.pem:28: -----BEGIN CERTIFICATE-----

Should we include the human-readable output of the certificate?

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/scripts/ca.cnf
File net/data/ssl/scripts/ca.cnf (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/scripts/ca.c...
net/data/ssl/scripts/ca.cnf:88: CN = $ENV::CA_COMMON_NAME

Does CA_COMMON_NAME need a default?

[Ryan Sleevi, 2013-06-18 23:57:59]

https://codereview.chromium.org/5535006/diff/113041/net/cert/test_root_certs_...
File net/cert/test_root_certs_android.cc (right):

https://codereview.chromium.org/5535006/diff/113041/net/cert/test_root_certs_...
net/cert/test_root_certs_android.cc:21: empty_ = false;
On 2013/06/18 22:51:45, wtc wrote:
> 
> This is a bug fix, right?

Yes, picked up by the TestRootCerts test.

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
File net/data/ssl/certificates/ok_cert.pem (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
net/data/ssl/certificates/ok_cert.pem:37: Not After : Jun 16 19:52:02 2023 GMT
On 2013/06/18 22:51:45, wtc wrote:
> 
> This should ideally be very far into the future. 2023 is only 10 years away.
> Note that httpd2.pem is valid until 2060.

Thanks for pointing out httpd2.pem, the script missed that.

The 10 years was intentional, as something less than Y2038. On the upside, this
is now automated, and no longer requires touching infrastructure, so it's not a
big deal to regen.

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
File net/data/ssl/certificates/root_ca_cert.pem (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/certificates...
net/data/ssl/certificates/root_ca_cert.pem:28: -----BEGIN CERTIFICATE-----
On 2013/06/18 22:51:45, wtc wrote:
> 
> Should we include the human-readable output of the certificate?

Yup! Done.

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/scripts/ca.cnf
File net/data/ssl/scripts/ca.cnf (right):

https://codereview.chromium.org/5535006/diff/113041/net/data/ssl/scripts/ca.c...
net/data/ssl/scripts/ca.cnf:88: CN = $ENV::CA_COMMON_NAME
On 2013/06/18 22:51:45, wtc wrote:
> 
> Does CA_COMMON_NAME need a default?

Intentionally omitted, just to make sure that the caller is supplying a unique
name.

[commit-bot: I haz the power, 2013-06-18 23:58:31]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/rsleevi@chromium.org/5535006/127041

[commit-bot: I haz the power, 2013-06-19 00:12:09]

Retried try job too often on chromium_presubmit for step(s) presubmit
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=chromium_p...

[Ryan Sleevi, 2013-06-19 00:31:42]

zel, jam: TBR for one line file name change

[commit-bot: I haz the power, 2013-06-19 00:32:10]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/rsleevi@chromium.org/5535006/127041

[commit-bot: I haz the power, 2013-06-19 00:49:00]

Retried try job too often on chromium_presubmit for step(s) presubmit
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=chromium_p...

[commit-bot: I haz the power, 2013-06-19 00:55:53]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/rsleevi@chromium.org/5535006/132042

[commit-bot: I haz the power, 2013-06-19 03:23:23]

Retried try job too often on linux_rel for step(s) browser_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=linux_rel&...

[jam, 2013-06-19 17:03:58]

chrome lgtm

[commit-bot: I haz the power, 2013-06-19 17:16:02]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/rsleevi@chromium.org/5535006/132042

[commit-bot: I haz the power, 2013-06-19 19:27:15]

Retried try job too often on linux_aura for step(s) browser_tests
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=linux_aura...

[commit-bot: I haz the power, 2013-06-20 01:42:14]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/rsleevi@chromium.org/5535006/114064

[commit-bot: I haz the power, 2013-06-20 16:28:46]

Change committed as 207443