[Esip-documentation] on documenting instrument and platform metadata
Bob Simons - NOAA Federal
bob.simons at noaa.gov
Fri Mar 20 11:16:44 EDT 2015
Isn't "use" metadata CF's realm? I know Ethan Davis has a proposal in to
EarthCube to expand the use of CF to other earth science domains. Shouldn't
we leave "use" metadata to CF?
Is there a real need for Yet Another Standard? If you want new "use"
metadata attributes, wouldn't it be best to work with the CF group to add
them to CF?
The nice thing about standards is that there are so many of them to choose
from.
-- Andrew S. Tanenbaum
On Thu, Mar 19, 2015 at 9:58 AM, John Graybeal via Esip-documentation <
esip-documentation at lists.esipfed.org> wrote:
> I have always seen the Venn diagram of these (provenance, use, discovery)
> as overlapping. Interesting to pursue further.
>
> If there is a call today I'm afraid i will have to miss it.
>
> John
>
> On Mar 19, 2015, at 9:00 AM, Ge Peng - NOAA Affiliate via
> Esip-documentation <esip-documentation at lists.esipfed.org> wrote:
>
> Actually, I think that Jim’s suggestion of developing ACUP - the Attribute
> Conventions for Usage and Provenance is a great idea. Hopefully, it will be
> picked up by someone and developed by the group in not so distant future.
>
>
> --- Peng
>
> On Thu, Mar 19, 2015 at 11:25 AM, Jim Biard via Esip-documentation <
> esip-documentation at lists.esipfed.org> wrote:
>
>> Hi.
>>
>> I think that Nan's last comment is well worth considering in relation to
>> ACDD. Some metadata are focused more on provenance, and some are focused on
>> discovery. The primary purpose of ACDD is to capture discovery metadata,
>> not provenance metadata. That doesn't mean that provenance metadata is
>> unimportant - it can, in fact, be critical to proper use, but that doesn't
>> mean that ACDD needs to address it.
>>
>> Perhaps someone should start developing ACUP - the Attribute Conventions
>> for Usage and Provenance. (grin)
>>
>> Grace and peace,
>>
>> Jim
>>
>>
>> On 3/17/15 11:30 AM, Nan Galbraith via Esip-documentation wrote:
>>
>> Hi Aaron -
>>
>> Having been involved in the oceansites specification on this, I agree
>> that
>> no one solution always works. I adopted (and try to follow, and
>> promote)
>> the NODC templates as closely as possible. Their development of the
>> ancillary
>> variable 'instrument' was brilliant, in my opinion.
>>
>> My datasets range from met files where all the data is from a single
>> logger but each of 6 or 8 variables might need its sensor described at
>> the variable level, to depth merged temperature files where there's only
>> one data variable but 30 instruments (and 4 or 5 different models). In
>> the met
>> files, variable attributes would work, but I use 'instrument' ancillary
>> variables
>> so I can link data variables recorded by a single sensor module (e.g.
>> Inst_HRH
>> is linked to data variables HRH and ATMP).
>>
>> The depth merged files show the real beauty of the instrument variable,
>> though;
>> since it can have a depth dimension, you can describe the different
>> instruments
>> deployed at different depths. There's no good way to do that using
>> attributes, as
>> far as I've seen - lists break down, especially when data is sub-sampled
>> i.e. in thredds.
>>
>> And, although much of this isn't yet at a stage where software can use
>> it, any and all
>> information that you have about the sensors and platform should still be
>> recorded
>> wherever it seems most appropriate to you, since it might otherwise be
>> lost. Re-formatting
>> this kind of info (if and when a useful standard emerges) will be much
>> less effort
>> than trying to track down something that was not recorded because there
>> didn't
>> seem like a standard way to encode it.
>>
>> If ACDD can't find and understand instrument variables yet, maybe that's
>> something
>> that could be looked at in another rev. Or, maybe instruments aren't a
>> common
>> discovery concept - I really don't know.
>>
>> Cheers -
>> Nan
>>
>>
>> On 3/16/15 6:41 PM, Aaron Sweeney via Esip-documentation wrote:
>>
>> Thanks very much for the guidance, John. I think I may stick with the
>> NODC or OceanSites platform and instrument variables to define the
>> particular instances.
>>
>> That being said, for my two use cases, I think I can still capture the
>> type of platform and instrument in the ACDD global attributes. This
>> amounts to:
>>
>> :platform = "DART"
>> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version 8.0"
>> :instrument = "BOTTOM PRESSURE GAUGES"
>> :instrument_vocabulary = "NASA/GCMD Instrument Keywords. Version 8.0"
>>
>> and
>>
>> :platform = "COASTAL STATIONS"
>> :platform_vocabulary = "NASA/GCMD Platform Keywords. Version 8.0"
>> :instrument = "TIDE GAUGES"
>> :instrument_vocabulary = "NASA/GCMD Instrument Keywords. Version 8.0"
>>
>> Cordially,
>> Aaron
>>
>> On 03/16/2015 02:43 PM, John Graybeal wrote:
>>
>> What great questions! The following thoughts are my own, and are
>> unrelated to my previous ACDD activities.
>>
>> Clearly some data files will have data from multiple platforms and
>> instruments; whereas others will get all their variables from a single
>> platform/instrument; and some will have data so heavily processed as to be
>> barely traceable to the source platforms/instruments. Each of the
>> specifications is oriented toward somewhat different scenarios, and none of
>> them are yet great at representing all the scenarios.
>>
>> Let's consider ACDD first. Note that this specification provides
>> _recommendations_ for attributes, not _requirements_; and the platform and
>> instrument attributes are only 'Suggested'. These give a broad-brush
>> categorization of the platform(s) or instrument(s) that collected a data
>> set; if it seems reasonable to specify that information for the whole data
>> set, use these attributes to do so. (If it isn't useful/applicable, then
>> don't worry about these attributes.) The most precise way to give that
>> info is to use a controlled vocabulary (see
>> https://marinemetadata.org/references?filter0=**ALL**&filter1%5B%5D=129&filter1%5B%5D=141
>> <https://marinemetadata.org/references?filter0=**ALL**&filter1[]=129&filter1[]=141> for
>> a partial list of vocabularies), and typically I'd expect a type vocabulary
>> here (with e.g. "CTD", "Mooring") not a specific instance.
>>
>> If you *are* following ACDD, I don't recommend putting the names of
>> variables containing information on platforms and instruments in the global
>> attributes 'platform' and 'instrument'. Without ACDD describing that as a
>> possibility, it's highly unlikely any software will know how to treat that,
>> and some people will get confused too. The existence of the variables will
>> be pretty self-explanatory, so just leave out those ACDD attributes. (I see
>> this conflicts with NODC and OceanSITES recommendations, to my surprise;
>> perhaps others will offer another view here.)
>>
>> Whether to put the information in a global instrument variable, or
>> attributes of each data variable, or both, seems to me a matter of
>> circumstances and taste. Let's stipulate the best solution is
>> human-recognizable, computer-parseable, extensible, and unambiguous. Thus,
>> global attributes are inadequate. Assuming a sophisticated data system 5
>> years from now would look in both variables and data attributes, then the
>> model of 'one variable per instrument or platform', and association from
>> the data variables to those variables or platforms seems best.
>>
>> But given that the examples in each of the different profiles still
>> seems inconsistent, likely you should follow the guidance most close to
>> your work. If you aren't particularly close to any of them, I'd do it a bit
>> differently still. But I'll hold that thought unless you want to extend the
>> discussion in that direction.
>>
>> John
>>
>>
>> On Mar 16, 2015, at 11:56, Aaron Sweeney via Esip-documentation <
>> esip-documentation at lists.esipfed.org> wrote:
>>
>> Hi, folks,
>>
>> I’m struggling with how to capture platform and instrument
>> information in the appropriate places in netCDF files. The NODC templates
>> suggest creating separate variables for platforms and instruments, with
>> multiple variable attributes (make, model, serial number, precision,
>> accuracy, etc.), and associating these platform and instrument variables
>> with physical variables via an “ancillary_variable” variable attribute
>> attached to the physical variable. Along these lines, SeaDataNet suggests
>> including variable attributes for instrument (sdn_instrument_urn and
>> sdn_instrument_name), and OceanSites suggests either the use of a separate
>> instrument variable (with make, model, serial number attributes) or
>> capturing make, model and serial number as variable attributes contained
>> directly within the relevant physical variable. But ACDD-1.3 only has
>> single *global* attributes for platform and instrument (and
>> vocabularies). The NODC templates suggest placing the names of the
>> variables containing information on platforms and instruments in the global
>> attributes: instrument and platform. I like NODC’s and OceanSites’
>> “associative” approach (i.e. “this instrument goes with this physical
>> variable”), but the use of variable names in global attributes conflicts
>> with ACDD-1.3.
>>
>> It seems that in order to comply with ACDD-1.3, I need to capture
>> instruments and platforms at a global level (dropping their association
>> with physical measurement variables), but in order to comply with
>> OceanSites and others, I need to capture instrument and platform
>> information in separate variables and associate these with physical
>> measurement variables via an ancillary_variable attribute.
>>
>> In order to comply with both, in the interest of enabling
>> interoperability, I seem to need to repeat instrument and platform metadata
>> in two different places.
>>
>> Any thoughts or guidance?
>>
>> References:
>> NODC netCDF timeseries template:
>> http://www.nodc.noaa.gov/data/formats/netcdf/v1.1/timeSeriesOrthogonal.cdl
>> OceanSITES Data Format Reference Manual (See Appendix 2 for sensor
>> variables):
>> http://www.oceansites.org/docs/oceansites_data_format_reference_manual.pdf
>> SeaDataNet Data Transports Manual (See Section 4.2.2 Co-ordinate
>> Variables):
>> http://www.seadatanet.org/content/download/16251/106283/file/SDN2_D85_WP8_Datafile_formats.pdf
>>
>> Thanks,
>> Aaron
>>
>>
>> --
>> *******************************************************
>> * Nan Galbraith Information Systems Specialist *
>> * Upper Ocean Processes Group Mail Stop 29 *
>> * Woods Hole Oceanographic Institution *
>> * Woods Hole, MA 02543 (508) 289-2444 *
>> *******************************************************
>>
>>
>>
>>
>>
>> _______________________________________________
>> Esip-documentation mailing listEsip-documentation at lists.esipfed.orghttp://lists.deltaforce.net/mailman/listinfo/esip-documentation
>>
>>
>> --
>> <hjdfgbcg.png> <http://www.cicsnc.org/> Visit us on
>> Facebook <http://www.facebook.com/cicsnc> *Jim Biard*
>> *Research Scholar*
>> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
>> North Carolina State University <http://ncsu.edu/>
>> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
>> 151 Patton Ave, Asheville, NC 28801
>> e: jbiard at cicsnc.org
>> o: +1 828 271 4900
>>
>>
>>
>>
>> _______________________________________________
>> Esip-documentation mailing list
>> Esip-documentation at lists.esipfed.org
>> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>>
>>
>
>
> --
> *Ge Peng, Ph.D*
> *Research Scholar*
> Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
> North Carolina State University <http://ncsu.edu/>
> NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
> 151 Patton Ave, Asheville, NC 28801
> ge.peng at noaa.gov
> o: +1 828 257 3009
> f: +1 828 257 3002
>
> Following CICS-NC on Facebook <http://www.facebook.com/cicsnc>
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
>
> _______________________________________________
> Esip-documentation mailing list
> Esip-documentation at lists.esipfed.org
> http://lists.deltaforce.net/mailman/listinfo/esip-documentation
>
>
--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A (New!)
Monterey, CA 93940 (New!)
Phone: (831)333-9878 (New!)
Fax: (831)648-8440
Email: bob.simons at noaa.gov
The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150320/58ebf48c/attachment-0001.html>
More information about the Esip-documentation
mailing list