[Esip-documentation] information on related files
Jim Biard
jbiard at cicsnc.org
Thu Sep 20 08:28:22 EDT 2018
Hi.
THREDDS aggregation won't be of much use will it? Nan is saying that
this data may well not be netCDF format.
Jim
On 9/19/18 5:38 PM, Armstrong, Edward M (398G) via Esip-documentation wrote:
> Hi Nan:
>
> I reviewed the findings of the 2015 NASA ESDSWG group on Virtual Collections and they were a set of use cases (including similar to yours) with some capabilities for solutions including THREDDS aggregations and event-based search capability for datasets (and implicitly granules for those datasets). So your proposal on virtual THREDDS aggregations seems a good path forward. But it would likely need manual curation to maintain as new granules are created.
>
>
> On 9/19/18, 2:09 PM, "Armstrong, Edward M (398G)" <Edward.M.Armstrong at jpl.nasa.gov> wrote:
>
> Hi Nan:
>
> I didn't think about THREDDS virtual aggregation but that might be an approach that works.....
>
> A few years ago there was a NASA ESDSWG on this subject (virtual collections). I'll look back and see if there are any recommendations that could help. I believe this type of THREDDS aggregation was one of them.
>
>
>
> On 9/19/18, 12:47 PM, "Esip-documentation on behalf of Nan Galbraith via Esip-documentation" <esip-documentation-bounces at lists.esipfed.org on behalf of esip-documentation at lists.esipfed.org> wrote:
>
> Thanks Ed, Dave, and Aleksandar.
>
> These are all possibilities, although I was hoping there would be an
> attribute
> in some convention, somewhere, that was more specific. Both 'references',
> and 'metadata_link' seem to be intended to provide information about the
> NetCDF
> file in which they're found, and 'external_variables' is meant to list
> 'variables which
> are named by attributes in the file'.
>
> What we were hoping to do was point to external data that was collected
> at the same
> place and time as the data in the file, but which is not IN the file for
> various reasons.
> This might be because it's not time series data, or because it's from
> another domain
> where NetCDF is unknown.
>
> Making this information machine-readable would require it to be pretty
> specific, and
> might even require more than one attribute - one giving the location,
> one describing
> the remote content, and maybe one describing how the data sets are related.
>
> Maybe the terminology that THREDDS uses would be a good place to start.
> I think
> I've got some THREDDS documentation ... somewhere around here. Also,
> NcML has the
> concept of Aggregation (to create "virtual" NetCDF datasets, according
> to the NcML page
> at unidata), but I'm not sure how far that's gone since I last looked at it.
>
> Thanks again -
> Nan
>
>
> On 9/19/18 11:48 AM, Aleksandar Jelenak via Esip-documentation wrote:
> > Hello!
> >
> > There's also this:
> > http://cfconventions.org/Data/cf-conventions/cf-conventions-1.7/cf-conventions.html#external-variables
> >
> > Apparently this approach is chosen for the CMIP6 data and is lax on
> > the specifics by design. The context where to find files with those
> > referenced external netCDF variables is to be provided in some
> > unspecified manner.
> >
> > It is possible to include links to related files on THREDDS dataset
> > pages but this is outside of file. The metadata_link keeps the
> > connection in the file but requires that the application code handle
> > ISO XML.
> >
> > -Aleksandar
> >
> >> On Sep 19, 2018, at 11:32 AM, Armstrong, Edward M (398G) via
> >> Esip-documentation <esip-documentation at lists.esipfed.org
> >> <mailto:esip-documentation at lists.esipfed.org>> wrote:
> >>
> >> Hi:
> >> I would recommend the ACDD attribute “references”:
> >> references
> >>
> >> Published or web-based references that describe the data or methods
> >> used to produce it. Recommend URIs (such as a URL or DOI) for papers
> >> or other references. This attribute is defined in the CF conventions.
> >>
> >> In CF global attributes there is also a “comment” attribute where
> >> such info could be placed in a generic sense.
> >> I have often thought of this use case too and its an area where ACDD
> >> or CF could be improved.
> >> The metadata_link could point to an ISO 19115 record where multiple
> >> citations could point to other datasets cleanly.
> >> *From:*Esip-documentation on behalf of David Neufeld
> >> *Reply-To:*David Neufeld <david.neufeld at noaa.gov
> >> <mailto:david.neufeld at noaa.gov>>
> >> *Date:*Wednesday, September 19, 2018 at 8:18 AM
> >> *To:*Nan Galbraith <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>>
> >> *Subject:*Re: [Esip-documentation] [thredds] information on related files
> >> Hi Nan,
> >> I'd recommend using a 'metadata_link' attribute.
> >> http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_1-3#Additional_Metadata:_metadata_link_attribute
> >> -Dave
> >> On Wed, Sep 19, 2018 at 8:56 AM, Nan Galbraith <ngalbraith at whoi.edu
> >> <mailto:ngalbraith at whoi.edu>> wrote:
> >>> Hi ESIP, NetCDF, and THREDDS folks -
> >>>
> >>> First, I'm sorry for the cross posting. I have a hard time figuring
> >>> out which NetCDF list does what; there are 'best practices' threads
> >>> on several lists, interspersed with install and development threads.
> >>>
> >>> The OceanSITES project would like to find the most useful way to
> >>> provide a link within a NetCDF file to an external dataset.
> >>>
> >>> > Another piece I am missing ... is an optional field, that link to a
> >>> > data DOI. Why? Here's an example: We operate an OceanSITES site off
> >>> > the Cape Verdes and the sediment trap data is ... not included in an
> >>> > netcdf file but processed and stored elsewhere
> >>> >https://doi.pangaea.de/10.1594/PANGAEA.863969
> >>>
> >>> Does THREDDS have a standard way to provide links to related or
> >>> co-located data? Is this something we discussed as part of ESIP's
> >>> ACDD recommendations? I've been searching through my store
> >>> of old messages, and haven't come up with anything.
> >>>
> >>> Thanks very much for any advice on this!
> >>>
> >>> - Nan Galbraith
> >>>
> >>>
> >>>
>
>
> --
> *******************************************************
> * Nan Galbraith Information Systems Specialist *
> * Upper Ocean Processes Group Mail Stop 29 *
> * Woods Hole Oceanographic Institution *
> * Woods Hole, MA 02543 (508) 289-2444 *
> *******************************************************
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> https://lists.esipfed.org/mailman/listinfo/esip-documentation
>
>
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> https://lists.esipfed.org/mailman/listinfo/esip-documentation
--
CICS-NC <http://www.cicsnc.org/> Visit us on
Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA National Centers for Environmental Information <http://ncdc.noaa.gov/>
/formerly NOAA’s National Climatic Data Center/
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org <mailto:jbiard at cicsnc.org>
o: +1 828 271 4900
/Connect with us on Facebook for climate
<https://www.facebook.com/NOAANCEIclimate> and ocean and geophysics
<https://www.facebook.com/NOAANCEIoceangeo> information, and follow us
on Twitter at @NOAANCEIclimate <https://twitter.com/NOAANCEIclimate> and
@NOAANCEIocngeo <https://twitter.com/NOAANCEIocngeo>. /
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.esipfed.org/pipermail/esip-documentation/attachments/20180920/124833ee/attachment-0001.html>
More information about the Esip-documentation
mailing list