Skip to content

ESGF_OpenDAP

Jump to bottom Edit New page
Stephen Pascoe edited this page Apr 9, 2014 · 6 revisions
Wiki Reorganisation
This page has been classified for reorganisation. It has been given the category REVISE.
This page contains useful content but needs revision. It may contain out of date or inaccurate content.

How to publish and secure OpenDAP access in the Earth System Grid

Federation

Publishing

The standard method of publishing data to an ESGF Data Node is to use the Publisher application to create THREDDS catalogs that are served by a Thredds Data Server (TDS), which among other things is also an OpenDAP server. To enable OpenDAP access when publishing data, the Publisher configuration file _ esg.ini _ needs to be instructed to associate services of type "OpenDAP" to single files and aggregations like so:

thredds_file_services =
        HTTPServer | /thredds/fileServer/ | TDSatJPL | fileservice
        OpenDAP | /thredds/dodsC/ | OpenDAPatJPL | fileservice

thredds_aggregation_services =
        OpenDAP | /thredds/dodsC/ | gridded

where "fileservice" is the name of the compound service used when running the Publisher, for example:

esgscan_directory --project obs4cmip5 -o obs4cmip5.txt /esg/archive
esgpublish --map ./obs4cmip5.txt --project obs4cmip5 --service fileservice
esgpublish --map ./obs4cmip5.txt --project obs4cmip5 --noscan --thredds --publish --service fileservice

Security

OpenDAP access through the TDS can be secured by using the ESGF Authentication and Authorization filters (you can still have the legacy Token filter to secure full file access, but that filter won't help with OpenDAP requests). All configuration is done through the TDS web application descriptor file _ web.xml _ . First, make sure that the TDS OpenDAP servlet is enabled:

  <!-- NetCDF/OPeNDAP server -->
  <servlet>
    <display-name>THREDDS/NetCDF DODS Server</display-name>
    <servlet-name>Opendap</servlet-name>
    <servlet-class>thredds.server.opendap.OpendapServlet</servlet-class>
    <load-on-startup>2</load-on-startup>
  </servlet>
  ...................................
  <servlet-mapping>
    <servlet-name>Opendap</servlet-name>
    <url-pattern>/dodsC/*</url-pattern>
  </servlet-mapping>

Then configure the following filter chain in exactly this order . Change the URLs to match the names of the servers hosting your Openid Relying Party and Authorization Service applications. The critical piece of configuration consists in mapping the Authentication Filter to intercept OpenDAP requests, which end in extensions .ascii , .dods , .das , .dds .

  <!-- Token Filter.
       This is needed only if your are authorizing access to files via a Gateway configured with token-based access.
  -->
  <filter>
    <filter-name>AuthorizationTokenValidationFilter</filter-name>
    <filter-class>eske.web.filters.security.AuthorizationTokenValidationFilter</filter-class>
    <init-param>
      <param-name>ESG-JPL</param-name>
      <param-value>https://esg-gateway.jpl.nasa.gov/remote/hessian/guest/secure/authorizationTokenService</param-value>
    </init-param>
  </filter>
  <filter-mapping>
    <filter-name>AuthorizationTokenValidationFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

  <!--
       Authentication Filter.
       This filter will redirect to the configured Openid Relying Party the first time a non-authenticated user makes a request.
  -->
  <filter>
    <filter-name>authenticationFilter</filter-name>
    <filter-class>esg.orp.app.AuthenticationFilter</filter-class>
    <init-param>
      <param-name>policyServiceClass</param-name>
      <param-value>esg.orp.app.tds.TDSPolicyService</param-value>
    </init-param>
    <init-param>
      <param-name>openidRelyingPartyUrl</param-name>
      <param-value>https://esg-datanode.jpl.nasa.gov/OpenidRelyingParty/home.htm</param-value>
    </init-param>
    <init-param>
      <param-name>trustoreFile</param-name>
      <param-value>/usr/local/tomcat/conf/esg-truststore.ts</param-value>
    </init-param>
    <init-param>
      <param-name>trustorePassword</param-name>
      <param-value>*</param-value>
    </init-param>
    <init-param>
      <param-name>trimURIRegEx</param-name>
      <param-value>\.ascii.*,\.dods.*,\.dds.*,\.das.*</param-value>
    </init-param>
  </filter>
  <filter-mapping>
    <filter-name>authenticationFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

  <!--- Authorization Filter.
        This filter will attempt authorization for the requested URL by contacting the configured Authorization Service.
        For compatibility with the Gateway authorization model,
        when requesting authorization, the OpenDAP extension ".dods", ".das", ".dds", ".ascii" is converted to ".nc".
  -->
  <filter>
    <filter-name>authorizationFilter</filter-name>
    <filter-class>esg.orp.app.AuthorizationFilter</filter-class>
    <init-param>
      <param-name>authorizationServiceClass</param-name>
      <!-- <param-value>esg.datanode.security.app.NoAuthorizationServiceFilterCollaborator</param-value> -->
      <param-value>esg.orp.app.SAMLAuthorizationServiceFilterCollaborator</param-value>
    </init-param>
    <init-param>
      <param-name>authorizationServiceUrl</param-name>
      <!-- <param-value>https://esg-gateway.jpl.nasa.gov/saml/soap/secure/authorizationService.htm</param-value> -->
      <param-value>https://esg-datanode.jpl.nasa.gov/esgf-security/saml/soap/secure/authorizationService.htm</param-value>
    </init-param>
  </filter>
  <filter-mapping>
    <filter-name>authorizationFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

  <!--
      Metrics filter.
      This has no bearing on securing OpenDAP access, but if enabled, it must go after all the security filters.
  -->
  <filter>
    <filter-name>AccessLoggingFilter</filter-name>
    <filter-class>esg.node.filters.AccessLoggingFilter</filter-class>
    <init-param>
      <param-name>db.driver</param-name>
      <param-value>org.postgresql.Driver</param-value>
    </init-param>
    <init-param>
      <param-name>db.protocol</param-name>
      <param-value>jdbc:postgresql:</param-value>
    </init-param>
    <init-param>
      <param-name>db.host</param-name>
      <param-value>localhost</param-value>
    </init-param>
    <init-param>
      <param-name>db.port</param-name>
      <param-value>5432</param-value>
    </init-param>
    <init-param>
      <param-name>db.database</param-name>
      <param-value>esgcet</param-value>
    </init-param>
    <init-param>
      <param-name>db.user</param-name>
      <param-value>*</param-value>
    </init-param>
    <init-param>
      <param-name>db.password</param-name>
      <param-value>*</param-value>
    </init-param>
    <init-param>
      <param-name>extensions</param-name>
      <param-value>.nc</param-value>
    </init-param>
  </filter>
  <filter-mapping>
    <filter-name>AccessLoggingFilter</filter-name>
    <url-pattern>/*</url-pattern>
  </filter-mapping>

Finally, you need to make sure your TDS WEB-INF/lib directory is configured with the latest version of the ESGF security jars. Specifically, the following jars will work:

-rw-r--r-- 1 root     root  30150 Apr 24 14:26 esgf-node-manager-common-0.4.7.jar
-rw-r--r-- 1 root     root  22954 Apr 24 14:26 esgf-node-manager-filters-0.4.7.jar
-rw-r--r-- 1 cinquini ipp  417460 Feb 16 14:17 esgf-security-1.1.2.jar
-rw-r--r-- 1 cinquini ipp   43953 Feb 23 07:28 esg-orp-1.0.3.jar

Testing

Once you have configured secure OpenDAP access in the TDS, and published some data with OpenDAP endpoints, you should be able to obtain data through OpenDAP in one of three possible ways:

Add a custom footer
Add a custom sidebar
Clone this wiki locally