[Esip-documentation] on documenting instrument and platform metadata
Ge Peng - NOAA Affiliate
ge.peng at noaa.gov
Thu Mar 19 12:00:13 EDT 2015
Actually, I think that Jim’s suggestion of developing ACUP - the Attribute
Conventions for Usage and Provenance is a great idea. Hopefully, it will be
picked up by someone and developed by the group in not so distant future.
--- Peng
On Thu, Mar 19, 2015 at 11:25 AM, Jim Biard via Esip-documentation <
esip-documentation at lists.esipfed.org> wrote:
> Hi.
>
> I think that Nan's last comment is well worth considering in relation to
> ACDD. Some metadata are focused more on provenance, and some are focused on
> discovery. The primary purpose of ACDD is to capture discovery metadata,
> not provenance metadata. That doesn't mean that provenance metadata is
> unimportant - it can, in fact, be critical to proper use, but that doesn't
> mean that ACDD needs to address it.
>
> Perhaps someone should start developing ACUP - the Attribute Conventions
> for Usage and Provenance. (grin)
>
> Grace and peace,
>
> Jim
>
>
> On 3/17/15 11:30 AM, Nan Galbraith via Esip-documentation wrote:
>
> Hi Aaron -
>
> Having been involved in the oceansites specification on this, I agree
> that
> no one solution always works. I adopted (and try to follow, and promote)
> the NODC templates as closely as possible. Their development of the
> ancillary
> variable 'instrument' was brilliant, in my opinion.
>
> My datasets range from met files where all the data is from a single
> logger but each of 6 or 8 variables might need its sensor described at
> the variable level, to depth merged temperature files where there's only
> one data variable but 30 instruments (and 4 or 5 different models). In
> the met
> files, variable attributes would work, but I use 'instrument' ancillary
> variables
> so I can link data variables recorded by a single sensor module (e.g.
> Inst_HRH
> is linked to data variables HRH and ATMP).
>
> The depth merged files show the real beauty of the instrument variable,
> though;
> since it can have a depth dimension, you can describe the different
> instruments
> deployed at different depths. There's no good way to do that using
> attributes, as
> far as I've seen - lists break down, especially when data is sub-sampled
> i.e. in thredds.
>
> And, although much of this isn't yet at a stage where software can use it,
> any and all
> information that you have about the sensors and platform should still be
> recorded
> wherever it seems most appropriate to you, since it might otherwise be
> lost. Re-formatting
> this kind of info (if and when a useful standard emerges) will be much
> less effort
> than trying to track down something that was not recorded because there
> didn't
> seem like a standard way to encode it.
>
> If ACDD can't find and understand instrument variables yet, maybe that's
> something
> that could be looked at in another rev. Or, maybe instruments aren't a
> common
> discovery concept - I really don't know.
>
> Cheers -
> Nan
>
>
> On 3/16/15 6:41 PM, Aaron Sweeney via Esip-documentation wrote:
>
> Thanks very much for the guidance, John. I think I may stick with the
> NODC or OceanSites platform and instrument variables to define the
> particular instances.
>
> That being said, for my two use cases, I think I can still capture the
> type of platform and instrument in the ACDD global attributes. This
> amounts to:
>
> :platform = "DART"
> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version 8.0"
> :instrument = "BOTTOM PRESSURE GAUGES"
> :instrument_vocabulary = "NASA/GCMD Instrument Keywords. Version 8.0"
>
> and
>
> :platform = "COASTAL STATIONS"
> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version 8.0"
> :instrument = "TIDE GAUGES"
> :instrument_vocabulary = "NASA/GCMD Instrument Keywords. Version 8.0"
>
> Cordially,
> Aaron
>
> On 03/16/2015 02:43 PM, John Graybeal wrote:
>
> What great questions! The following thoughts are my own, and are
> unrelated to my previous ACDD activities.
>
> Clearly some data files will have data from multiple platforms and
> instruments; whereas others will get all their variables from a single
> platform/instrument; and some will have data so heavily processed as to be
> barely traceable to the source platforms/instruments. Each of the
> specifications is oriented toward somewhat different scenarios, and none of
> them are yet great at representing all the scenarios.
>
> Let's consider ACDD first. Note that this specification provides
> _recommendations_ for attributes, not _requirements_; and the platform and
> instrument attributes are only 'Suggested'. These give a broad-brush
> categorization of the platform(s) or instrument(s) that collected a data
> set; if it seems reasonable to specify that information for the whole data
> set, use these attributes to do so. (If it isn't useful/applicable, then
> don't worry about these attributes.) The most precise way to give that
> info is to use a controlled vocabulary (see
> https://marinemetadata.org/references?filter0=**ALL**&filter1%5B%5D=129&filter1%5B%5D=141
> <https://marinemetadata.org/references?filter0=**ALL**&filter1[]=129&filter1[]=141> for
> a partial list of vocabularies), and typically I'd expect a type vocabulary
> here (with e.g. "CTD", "Mooring") not a specific instance.
>
> If you *are* following ACDD, I don't recommend putting the names of
> variables containing information on platforms and instruments in the global
> attributes 'platform' and 'instrument'. Without ACDD describing that as a
> possibility, it's highly unlikely any software will know how to treat that,
> and some people will get confused too. The existence of the variables will
> be pretty self-explanatory, so just leave out those ACDD attributes. (I see
> this conflicts with NODC and OceanSITES recommendations, to my surprise;
> perhaps others will offer another view here.)
>
> Whether to put the information in a global instrument variable, or
> attributes of each data variable, or both, seems to me a matter of
> circumstances and taste. Let's stipulate the best solution is
> human-recognizable, computer-parseable, extensible, and unambiguous. Thus,
> global attributes are inadequate. Assuming a sophisticated data system 5
> years from now would look in both variables and data attributes, then the
> model of 'one variable per instrument or platform', and association from
> the data variables to those variables or platforms seems best.
>
> But given that the examples in each of the different profiles still
> seems inconsistent, likely you should follow the guidance most close to
> your work. If you aren't particularly close to any of them, I'd do it a bit
> differently still. But I'll hold that thought unless you want to extend the
> discussion in that direction.
>
> John
>
>
> On Mar 16, 2015, at 11:56, Aaron Sweeney via Esip-documentation <
> esip-documentation at lists.esipfed.org> wrote:
>
> Hi, folks,
>
> I’m struggling with how to capture platform and instrument
> information in the appropriate places in netCDF files. The NODC templates
> suggest creating separate variables for platforms and instruments, with
> multiple variable attributes (make, model, serial number, precision,
> accuracy, etc.), and associating these platform and instrument variables
> with physical variables via an “ancillary_variable” variable attribute
> attached to the physical variable. Along these lines, SeaDataNet suggests
> including variable attributes for instrument (sdn_instrument_urn and
> sdn_instrument_name), and OceanSites suggests either the use of a separate
> instrument variable (with make, model, serial number attributes) or
> capturing make, model and serial number as variable attributes contained
> directly within the relevant physical variable. But ACDD-1.3 only has
> single *global* attributes for platform and instrument (and
> vocabularies). The NODC templates suggest placing the names of the
> variables containing information on platforms and instruments in the global
> attributes: instrument and platform. I like NODC’s and OceanSites’
> “associative” approach (i.e. “this instrument goes with this physical
> variable”), but the use of variable names in global attributes conflicts
> with ACDD-1.3.
>
> It seems that in order to comply with ACDD-1.3, I need to capture
> instruments and platforms at a global level (dropping their association
> with physical measurement variables), but in order to comply with
> OceanSites and others, I need to capture instrument and platform
> information in separate variables and associate these with physical
> measurement variables via an ancillary_variable attribute.
>
> In order to comply with both, in the interest of enabling
> interoperability, I seem to need to repeat instrument and platform metadata
> in two different places.
>
> Any thoughts or guidance?
>
> References:
> NODC netCDF timeseries template:
> http://www.nodc.noaa.gov/data/formats/netcdf/v1.1/timeSeriesOrthogonal.cdl
> OceanSITES Data Format Reference Manual (See Appendix 2 for sensor
> variables):
> http://www.oceansites.org/docs/oceansites_data_format_reference_manual.pdf
> SeaDataNet Data Transports Manual (See Section 4.2.2 Co-ordinate
> Variables):
> http://www.seadatanet.org/content/download/16251/106283/file/SDN2_D85_WP8_Datafile_formats.pdf
>
> Thanks,
> Aaron
>
>
> --
> *******************************************************
> * Nan Galbraith Information Systems Specialist *
> * Upper Ocean Processes Group Mail Stop 29 *
> * Woods Hole Oceanographic Institution *
> * Woods Hole, MA 02543 (508) 289-2444 *
> *******************************************************
>
>
>
>
>
> _______________________________________________
> Esip-documentation mailing listEsip-documentation at lists.esipfed.orghttp://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
>
> --
> [image: CICS-NC] <http://www.cicsnc.org/> Visit us on
> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
> 151 Patton Ave, Asheville, NC 28801
> e: jbiard at cicsnc.org
> o: +1 828 271 4900
>
>
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
>
--
*Ge Peng, Ph.D*
*Research Scholar*
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
151 Patton Ave, Asheville, NC 28801
ge.peng at noaa.gov
o: +1 828 257 3009
f: +1 828 257 3002
Following CICS-NC on Facebook <http://www.facebook.com/cicsnc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150319/e5690a5b/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hjdfgbcg.png
Type: image/png
Size: 11847 bytes
Desc: not available
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150319/e5690a5b/attachment-0001.png>
More information about the Esip-documentation
mailing list