HTTPS SPDF Web Services

Introduction

The SPDF web services are required to comply with U.S. government standards for secure communcations with federal web sites. While this should not require any special software (beyond an up-to-date browser) for access to simple HTML pages, it may require additional preparations for non-browser based access to the SPDF web services. Typically, this will involve changing any URLs beginning with “http: ” to “https: ” and installing updated Certificate Authority (CA) certificates in older programming run-time environments or disabling the verification of the certificate. Run-time environments released/updated after the summer of 2016 may already have the necessary certificates so it is best to test your environment before making any changes.

Note
The rest of this page contains several links to certificates that your browser is likely to already possess. Simply clicking the certificate link may result in your browser attempting to install the certificate in your browser. You may have to use the “save link as ” feature of your browser or use other software (for example, curl or wget) to download the certificate.

The required CA certificates are available from:

Let ‘s Encrypt Certificates
- Intermediate Authority X3: [pem ]
  - This can be used to add a single certificate to an older collection of certificates.
Mozilla.org bundle which includes the above Let ‘s Encrypt Authority X3 certificate.
- This can be used to replace an entire older collection of certificates with a newer collection after being processed by the cURL mk-ca-bundle.pl script.

If you encounter problems or have questions about the SPDF web services, e-mail spdf-support.

Instructions for common web service clients

Below are instructions to enable common programming run-time environments to interact with the SPDF web services over HTTPS. The environments covered include the following:

cURL
wget
Java
IDL
- IDLnetURL
- Java Bridge
Perl
Python
- httplib2

This does not cover all possible run-time environments used to implement web service clients. But these instructions maybe helpful for other environments as well.

cURL Client

Older versions of cURL may need a newer certificate bundle. To overcome this problem, do one of the following:

Disable verification of the certificates with the -k/--insecure option to the curl command like this
- $ curl [--insecure] https://sscweb.gsfc.nasa.gov/WS/sscr/2/application.wadl | xmllint --format -
Alternatively, obtain and use a newer certificate bundle.
1. Obtain a new certificate bundle using one of these procedures:
  - Run the cURL mk-ca-bundle.pl script include with the source for cURL.
  - Alternatively, download the results of running mk-ca-bundle.pl that is available at ca-bundle.crt.
2. Add the --cacert option to all uses of cURL like this
  - $ curl [--cacert ca-bundle.crt] https://sscweb.gsfc.nasa.gov/WS/sscr/2/application.wadl | xmllint --format -
  - Alternatively, you can avoid using the --cacert option by setting the environment variable named CURL_CA_BUNDLE to the path of the ca-bundle.crt file.

wget Client

Older versions of wget are know to have problems evaluating the multi-domain certficate used for our sites. If you you encounter the error

        `ERROR: certificate common name 'spdf.gsfc.nasa.gov' doesn't match 
        requested host name 'cdaweb.gsfc.nasa.gov'`

you should upgrade your wget. Version 1.12 and newer are known to work.

Older versions of wget/openssl also do not support newer versions of the SSL/TLS protocols. If you get an error like

        `OpenSSL: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure` 
        `Unable to establish SSL connection. `

you will need to upgrade your version of wget and/or openssl.

Wget may also not have a new enough certificate bundle. To overcome this problem, do one of the following:

Obtain and use a newer certficate bundle as described below:
1. Obtained a new certificate bundle as described in cURL above.
2. Use the new certificate bundle in one of the following ways:
  - Create a file named $HOME/.wgetrc containing the line
```
ca_certificate=ca-bundle.crt
```
    Where ca-bundle.crt is the full-pathname of the new certficate bundle file. Now use wget as usual. For example
    - wget -O - https://sscweb.gsfc.nasa.gov/WS/sscr/2/application.wadl | xmllint --format -
  - Alternately, add the --ca-certificate option to the wget command like this
    - wget --ca-certificate=ca-bundle.crt -O - https://sscweb.gsfc.nasa.gov/WS/sscr/2/application.wadl | xmllint --format -
Alternatively, you can disable verification of the certificate like this
- wget [--no-check-certificate] -O - https://sscweb.gsfc.nasa.gov/WS/sscr/2/application.wadl | xmllint --format -

Java Client

Java versions after 1.7.0_111, and 1.8.0_102 should not require any special preparations. For earlier versions of Java, do the following

Download this CA certificate file https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem.
$ cd JAVA_HOME/jre
- Where JAVA_HOME is the root directory of your Java installation.
$ bin/keytool -import -trustcacerts -file lets-encrypt-x3-cross-signed.pem -keystore lib/security/cacerts
- Where lets-encrypt-x3-cross-signed.pem is the CA certificate file downloaded in step 1.
Now, any Java application that are run with this installation of Java should be able to use the web services over HTTPS.

If your Java client code runs in an application server, you may, depending upon the specific application server, have to update the applications server ‘s trusted CA keystore. The location of the keystore will vary depending upon the application server.

To debug a TLS issue, you might try setting the following system property

javax.net.debug=SSL

IDL Client

Interactive Data Language (IDL) versions after 8.5.0 should not require any special preparations. These instructions describe how to prepare older versions of IDL to access the SPDF web services.

There are multiple ways for an IDL client to make web services calls. Two common ways are with the IDLnetURL object and through the IDL-Java bridge. Instructions for each of these cases are describe separately below.

IDLnetURL IDL Client

For an IDL application that use IDLnetURL to access the web services, do one of the following:

IDL versions less than 8.4 are not capable of verifying our SSL certificate. If using an IDL version less than 8.4, do one of the following:
- Upgrade to a newer version of IDL.
- Programatically disable verification of the certficate by setting the SSL_VERIFY_PEER property to 0 like this
  - idlNetUrlObj->SetProperty, SSL_VERIFY_PEER=0.
- Replace the libcurl.(so|dll|dylib) file with the one from version 8.4 or higher.
Update the CA certficate file as described below:
1. Create or download the ca-bundle.crt file as described in cURL above.
2. Replace the <IDL_DIR>/bin/bin.<platform>/ca-bundle.crt file with the one obtained in step 1 above.
  - Instead of replacing the ca-bundle.crt file in your IDL installation, you could programatically set the SSL_CERTIFICATE_FILE IDLnetURL property to the fully-qualified path of the downloaded ca-bundle.crt file.
3. Now, any IDL applications that use this installation of IDL and the IDLnetURL object should be able to use these web services over HTTPS.
Alternatively, programatically disable verification of the certficate by setting the SSL_VERIFY_PEER property to 0 like this
- idlNetUrlObj->SetProperty, SSL_VERIFY_PEER=0.

IDL Java Bridge Client

For an IDL application that use the Java bridge to access the web services, do the following:

Download this CA certificate file https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem
$ cd <IDL_DIR>/idlde/bin.<platform>/jre
- Where <IDL_DIR> is the directory where IDL is installed and <platform> is your platform.
$ bin/keytool -import -trustcacerts -file lets-encrypt-x3-cross-signed.pem -keystore lib/security/cacerts
- Where lets-encrypt-x3-cross-signed.pem is the file obtained in step 1.
Now, any IDL applications that use this installation of IDL and the Java bridge should be able to use these web services over HTTPS.

Perl

For a Perl application, the location of the default trusted CA file varies on different platforms so the following simply overrides the default with an updated file. The following are general instructions that will work in most configurations.

Create or download the ca-bundle.crt file as described in cURL above.
Set the PERL_LWP_SSL_CA_FILE environment variable to the ca-bundle.crt file before running your Perl program.

Python

For a Python application, the location of the default trusted CA file varies on different platforms and libraries. The following are suggestions for specific libraries.

httplib2

For python code that utilizes the httplib2 (http.client in Python 3) library, verification of the certificate may be disabled by adding the disable_ssl_certificate_validation=True parameter to the constructor. For example,

    import httplib2

    client = httplib2.Http(".cache", disable_ssl_certificate_validation=True)

Questions/Comments

Questions or comments concerning these Web services should be sent to SPDF Feedback.