Merge pull request #144 from oxelson/main

remotetest migration & removal of blended products

README.md

@@ -10,12 +10,11 @@ Configuration files
 - kept in git repo TdsConfig
 
 - each machine has their own subdirectory:
-  - idd : common to all (4.6) idd servers
+  - idd : common to all idd servers
   - rdavm : at CISL/RDA
-  - remotetest : where? opendap/netcdf C test server
   - startup: this should be the minimal configuration for running a thredds server
   - thredds : atm-nwsc (/opt/tds) (idd + casestudies)
-  - threddsTest : atm-nwsc (/opt/tds-test) (idd + casestudies + preprocessed)
+  - threddsTest : atm-nwsc (/opt/tds-test) (idd + casestudies + preprocessed + remotetest)
   - threddsDev : lead (/opt/tds-dev)
   - awsL2 : jetstream machine serving AWS Level2 radar archive data
 
@@ -30,7 +29,7 @@ Configuration files
    The build and upload of config.zip files is automatically performed by GitHub Actions for
    all changes committed to the repository.
 
-- config.zip hosted on [Unidata Nexus](https://artifacts.unidata.ucar.edu/#browse/browse:downloads-tds-config)
+- config.zip hosted on the [Unidata artifacts server](https://artifacts.unidata.ucar.edu/#browse/browse:downloads-tds-config)
 
 - to make changes to config files:
   - Edit files, e.g. `thredds/*`

idd/idd/satellite.xml

@@ -188,6 +188,8 @@
         <addDatasetSize/>
       </datasetScan>
     </dataset><!-- End of NESDIS NOAA20 data -->
+
+    <!--
     <dataset name="Hyrdology Products">
       <metadata inherited="true">
         <serviceName>nongrid</serviceName>
@@ -211,7 +213,7 @@
         </filter>
         <addDatasetSize/>
       </datasetScan>
-    </dataset><!-- End of NESDIS Blended data -->
+    </dataset>  End of NESDIS Blended data -->
   </dataset><!-- End of Satellite Data -->
 </catalog>
 

remotetest/README.md

@@ -1,96 +1,12 @@
-Setup Instructions For The remotetest.unidata.ucar.edu Tomcat Server
+remotetest.unidata.ucar.edu host now combined with thredds-test server 
 ====
+** This method of updating the remotetest server is defunct and should not be used! **
 
-When installing a new thredds server on
-_remotetest.unidata.ucar.edu_, the process
-is as follows. This overrides any other instructions.
+The remotetest.unidata.ucar.edu is no longer a standalone host.  
 
-For purposes of this process, it is assumed that the content and logs
-directories are in _/opt/remotetest_
-on the machine _remotetest.unidata.ucar.edu_.
+The `dts.war` and `d4ts.war` files are hosted on the same tomcat instance as the thredds-test server. 
 
-1. Enter the directory _TdsConfig/remotetest_.
-2. Edit as needed the following files:
-   * _thredds/threddsConfig.xml_
-   * _thredds/catalog.xml_
-3. Run the shell script 'content.sh' to construct the file
-   'content.tar'
-4. Copy _content.tar_ to the _remotetest.unidata.ucar.edu_ machine.
-5. Stop the tomcat server.
-6. Move to the directory _/opt/remotetest/content_.
-7. Untar content.tar into _/opt/remotetest/content_.
-8. (optional) clear logs: _rm -f /opt/remotetest/logs/*_
-9. Build and install server side keystore and truststore (see below)
-10. (optional) modify *$CATALINA_HOME/conf/server.xml* (see below)
-11. (optional) modify *$CATALINA_HOME/conf/tomcat-users.xml* (see below)
-12. Restart the tomcat server.
+* The `defunct` directory contains the old files and documentation for updating this host.
+* The `thredds` directory contains the data used by this host.
 
-Optional Changes to $CATALINA_HOME/conf/server.xml
-----
-On a one time basis, you will need to modify *$CATALINA_HOME/conf/server.xml*
-to support SSL and also to support a password protected dataset.
-
-1. Insert the following <Connector> into conf/server.xml
-<pre>
-	&lt;Connector
-		port="8443"
-		maxThreads="150"
-		SSLEnabled="true"
-		scheme="https"
-		secure="true" 
-		clientAuth="want"
-		sslProtocol="TLS"
-		keyAlias="1"
-		keystoreFile="conf/ServerKeystore.jks"
-		keystorePass="password"
-		keyPass="password"
-	/&gt;
-</pre>
-Note the 'clientAuth="want"' line. This tells tomcat to attempt
-to use client-side keys but if not successful, ignore and continue.
-See below with respect to the 'keystoreFile' line.
-
-Optional Changes to $CATALINA_HOME/conf/tomcat-users.xml
-----
-Since access to a restricted dataset is used as a test,
-the tomcat-users.xml file must be modified to include the following lines.
-<pre>
-	<role rolename="restrictedDatasetUser"/>
-	<role rolename="tiggeData"/>
-	<user username="tiggeUser"
-		password="tigge"
-	    roles="restrictedDatasetUser,tiggeData"/>
-</pre>
-The roles defined here must be consistent with _content/thredds/catalog.xml_.
-
-Depending on your tomcat installation, the tiggeUser password may
-need to be encrypted. You will also need to ensure that 'https:'
-access is enabled either in tomcat (see above) or apache httpd.
-
-Key/Trust Store Construction and Installation:
-----
-If you are running a standalone Tomcat (typically using Intellij),
-and in order to support server keys and (optionally) client side keys
-the shell script certs.sh must executed. This will construct
-JKS format key and trust stores.
-
-1. Run certs.sh to produce
-   * ServerKeystore.jks
-   * ClientKeystore.jks
-   * ClientTruststore.jks
-2. Put ServerKeystore.jks and ClientTruststore.jks into ${CATALINA_HOME}/conf
-
-Notes:
------
-1. Verify that all the datasets referenced in catalog.xml
-   are in place in /opt/remotetest/content/thredds/public.
-   Note that the "locations" in catalog.xml are not actual paths:
-   effectively the "content/" part refers to
-    /opt/remotetest/content/thredds/public
-
-2. If you plan to modify /opt/remotetest/content, make sure
-   you keep a backup copy.
-
-3. Make sure that the following environment variable in setenv.sh is set:
-     CONTENT_ROOT="-Dtds.content.root.path=/opt/remotetest/content"
-
+Please reference the `threddstest/catalog.xml` file for `threddstest/remotetest` directory for more information.

remotetest/defunct/README.md

@@ -0,0 +1,9 @@
+remotetest.unidata.ucar.edu host now combined with thredds-test server 
+====
+** This method of updating the remotetest server is defunct and should not be used! **
+
+The remotetest.unidata.ucar.edu is no longer a standalone host.  
+
+The `dts.war` and `d4ts.war` files are hosted on the same tomcat instance as the thredds-test server. 
+
+Please reference the `threddstest/catalog.xml` file for `threddstest/remotetest` directory for more information.

remotetest/defunct/README.md.old

@@ -0,0 +1,96 @@
+Setup Instructions For The remotetest.unidata.ucar.edu Tomcat Server
+====
+
+When installing a new thredds server on
+_remotetest.unidata.ucar.edu_, the process
+is as follows. This overrides any other instructions.
+
+For purposes of this process, it is assumed that the content and logs
+directories are in _/opt/remotetest_
+on the machine _remotetest.unidata.ucar.edu_.
+
+1. Enter the directory _TdsConfig/remotetest_.
+2. Edit as needed the following files:
+   * _thredds/threddsConfig.xml_
+   * _thredds/catalog.xml_
+3. Run the shell script 'content.sh' to construct the file
+   'content.tar'
+4. Copy _content.tar_ to the _remotetest.unidata.ucar.edu_ machine.
+5. Stop the tomcat server.
+6. Move to the directory _/opt/remotetest/content_.
+7. Untar content.tar into _/opt/remotetest/content_.
+8. (optional) clear logs: _rm -f /opt/remotetest/logs/*_
+9. Build and install server side keystore and truststore (see below)
+10. (optional) modify *$CATALINA_HOME/conf/server.xml* (see below)
+11. (optional) modify *$CATALINA_HOME/conf/tomcat-users.xml* (see below)
+12. Restart the tomcat server.
+
+Optional Changes to $CATALINA_HOME/conf/server.xml
+----
+On a one time basis, you will need to modify *$CATALINA_HOME/conf/server.xml*
+to support SSL and also to support a password protected dataset.
+
+1. Insert the following <Connector> into conf/server.xml
+<pre>
+	&lt;Connector
+		port="8443"
+		maxThreads="150"
+		SSLEnabled="true"
+		scheme="https"
+		secure="true" 
+		clientAuth="want"
+		sslProtocol="TLS"
+		keyAlias="1"
+		keystoreFile="conf/ServerKeystore.jks"
+		keystorePass="password"
+		keyPass="password"
+	/&gt;
+</pre>
+Note the 'clientAuth="want"' line. This tells tomcat to attempt
+to use client-side keys but if not successful, ignore and continue.
+See below with respect to the 'keystoreFile' line.
+
+Optional Changes to $CATALINA_HOME/conf/tomcat-users.xml
+----
+Since access to a restricted dataset is used as a test,
+the tomcat-users.xml file must be modified to include the following lines.
+<pre>
+	<role rolename="restrictedDatasetUser"/>
+	<role rolename="tiggeData"/>
+	<user username="tiggeUser"
+		password="tigge"
+	    roles="restrictedDatasetUser,tiggeData"/>
+</pre>
+The roles defined here must be consistent with _content/thredds/catalog.xml_.
+
+Depending on your tomcat installation, the tiggeUser password may
+need to be encrypted. You will also need to ensure that 'https:'
+access is enabled either in tomcat (see above) or apache httpd.
+
+Key/Trust Store Construction and Installation:
+----
+If you are running a standalone Tomcat (typically using Intellij),
+and in order to support server keys and (optionally) client side keys
+the shell script certs.sh must executed. This will construct
+JKS format key and trust stores.
+
+1. Run certs.sh to produce
+   * ServerKeystore.jks
+   * ClientKeystore.jks
+   * ClientTruststore.jks
+2. Put ServerKeystore.jks and ClientTruststore.jks into ${CATALINA_HOME}/conf
+
+Notes:
+-----
+1. Verify that all the datasets referenced in catalog.xml
+   are in place in /opt/remotetest/content/thredds/public.
+   Note that the "locations" in catalog.xml are not actual paths:
+   effectively the "content/" part refers to
+    /opt/remotetest/content/thredds/public
+
+2. If you plan to modify /opt/remotetest/content, make sure
+   you keep a backup copy.
+
+3. Make sure that the following environment variable in setenv.sh is set:
+     CONTENT_ROOT="-Dtds.content.root.path=/opt/remotetest/content"
+

threddsTest/catalog.xml

@@ -16,4 +16,8 @@
     <catalogRef xlink:href="casestudies/catalog.xml" xlink:title="Unidata case studies" name=""/>
     <catalogRef xlink:href="preprocessed/catalog.xml" xlink:title="Preprocessed data" name=""/>
   </dataset>
+
+  <dataset name="DAP and DAP4 Tests">
+    <catalogRef xlink:href="remotetest/catalog.xml" xlink:title="Test Data" name=""/>
+  </dataset>
 </catalog>

threddsTest/remotetest/catalog.xml

@@ -0,0 +1,43 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<catalog xmlns="http://www.unidata.ucar.edu/namespaces/thredds/InvCatalog/v1.0"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xmlns:ncml="http://www.unidata.ucar.edu/namespaces/netcdf/ncml-2.2" version="1.2">
+
+  <service name="all" base="" serviceType="compound">
+    <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/" />
+    <service name="dap4" serviceType="DAP4" base="/thredds/dap4/" />
+    <service name="http" serviceType="HTTPServer" base="/thredds/fileServer/" />
+    <!--service name="wcs" serviceType="WCS" base="/thredds/wcs/" /-->
+    <!--service name="wms" serviceType="WMS" base="/thredds/wms/" /-->
+    <!--service name="ncss" serviceType="NetcdfSubset" base="/thredds/ncss/grid/" /-->
+  </service>
+
+  <service name="dap" base="" serviceType="compound">
+    <service name="odap" serviceType="OpenDAP" base="/thredds/dodsC/" />
+    <service name="dap4" serviceType="DAP4" base="/thredds/dap4/" />
+  </service>
+
+  <datasetScan name="Standard Test Data" ID="testDatasetScan" path="testdata" location="content/testdata/">
+    <metadata inherited="true">
+      <serviceName>all</serviceName>
+      <dataType>Grid</dataType>
+    </metadata>
+  </datasetScan>
+
+  <datasetScan name="OPeNDAP Test Data" ID="testDodsScan" path="testdods" location="content/testdods/">
+    <metadata inherited="true">
+      <serviceName>all</serviceName>
+      <dataType>Grid</dataType>
+    </metadata>
+  </datasetScan>
+
+<!--
+  <datasetScan name="Restricted Test Data" ID="testrestricted" path="restrict" location="content/restrict/" restrictAccess="tiggeData">
+    <metadata inherited="true">
+      <serviceName>all</serviceName>
+      <dataType>Grid</dataType>
+    </metadata>
+  </datasetScan>
+-->
+
+</catalog>