Esri/geoportal-server-harvester

Harvesting THREDD's metadata

mfforte opened this issue · 12 comments

mfforte commented

Hi All,

What "Type" broker should be used to harvest from THREDD's data server here:
https://chlthredds.erdc.dren.mil/thredds/catalog/catalog.html

I did not see any mention of THREDD's in the harvester broker dropdown list.

Thanks,
Mike

mhogeweg commented

hi Mike. We have not implemented THREDDS yet in the new harvester. I will add this to the backlog. meanwhile, I was looking into this particular THREDDS catalog and noticed some issues. See for example this subset of the catalog:
https://chlthredds.erdc.dren.mil/thredds/catalog/wis/Atlantic/ST41001/1980/catalog.xml

I picked a random data set:
https://chlthredds.erdc.dren.mil/thredds/catalog/wis/Atlantic/ST41001/1980/catalog.html?dataset=wisDatasetScan/Atlantic/ST41001/1980/WIS-ocean_waves_ST41001_198001.nc

but then when accessing the ISO 19115 metadata representation I got an error:
https://chlthredds.erdc.dren.mil/thredds/iso/wis/Atlantic/ST41001/1980/WIS-ocean_waves_ST41001_198001.nc?catalog=https://chlthredds.erdc.dren.mil/thredds/catalog/wis/Atlantic/ST41001/1980/catalog.html&dataset=wisDatasetScan/Atlantic/ST41001/1980/WIS-ocean_waves_ST41001_198001.nc

Throwable exception handled : org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoSuchMethodError: ucar.nc2.dataset.NetcdfDataset.getCoordinateAxes()Ljava/util/List;
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1055)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
...
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoSuchMethodError: ucar.nc2.dataset.NetcdfDataset.getCoordinateAxes()Ljava/util/List;
	at thredds.server.metadata.util.ThreddsExtentUtil.doGetExtent(ThreddsExtentUtil.java:213)
	at thredds.server.metadata.util.ThreddsExtentUtil.getExtent(ThreddsExtentUtil.java:457)
...
mfforte commented
mhogeweg commented

hi, we have implemented THREDDS support. See this pull request from @pandzel
#123

mfforte commented
mfforte commented

Hello Marten and Piotr,

What is the best way to update the webapp with the recently implemented THREDDS harvester support? 
I recognize the files that have been updated and their paths, however locating the same folder/filename within Tomcat/webapp has been challenging.

Previously, I downloaded the war file.

Thanks,
Mike

pandzel-zz commented

The best way is to build application by yourself. It could be done with these steps:

Steps

  • Clone repository: git clone https://github.com/Esri/geoportal-server-harvester.git,
  • Change working directory to the clonet repository: cd geoportal-server-harvester,
  • Build the project: mvn clean install.

Your new shiny war will be in geoportal-server-harvester\geoportal-application\geoportal-harvester-war\target folder.

Tools needed

mhogeweg commented

my workflow for updates, while maybe not fully sanctioned has worked for many years:

  • download .war
  • rename to .zip
  • unzip into folder
  • stop tomcat
  • move old app folder out of webapps (assuming that's where you deploy)
  • move new folder into webapps
  • start tomcat
mfforte commented

Thanks Piotr and Marten - I will give your instructions try.

mfforte commented
mhogeweg commented

yes, this would be configured in the 'nodes' property in app-context.xml.

mfforte commented