[Esip-documentation] ACDD-1.3 documentation change request: Descriptions of "resolution" attributes
Ted Habermann
thabermann at hdfgroup.org
Thu Feb 26 10:03:12 EST 2015
Aaron,
Thanks for starting this discussion… It is great to have new eyes on this work. Sorry that I was in a meeting and traveling so am a bit slow in responding.
First, we need to admit that, in general, units are a mess. The general approach for the uom attribute is that it is implemented as an href to a units dictionary. The question is – which one? The primary goal of the translation that is done in ncISO is to cross a chasm between attributes in netCDF and the same content in ISO. The units is one place that that is a difficult task because the netCDF community (that we are trying to help) has a well established and completely functional approach to units in udunits. Unfortunately, this approach was developed and implemented (AFAIK) with a desktop application in mind (and it works really well there). It is not easily matched up with the more web-based approach that is used in gml where the units catalog is accessible in a restful way using the unit string as an identifier. When I created ncISO my goal was to facilitate getting as much content as I could get across the chasm. Units was left for future work. Several people have tried this work at several times. The most complete attempt that I know of was done by Bob in his typical careful and thorough way (see http://coastwatch.pfeg.noaa.gov/erddap/convert/units.html).
I’m not really sure that I understand what you call an inconsistency, but I think it reflects the inconsistency in approach between udunits and gml. That is an inconsistency between two very well established approaches in two very large communities. We can not fix that in a general way, but Bob’s work could be a light shining in the right direction.
The definition of horizontal resolution in ACDD has never been close to the definition that you want, which I would call precision or accuracy, not resolution. The long history of netCDF is clearly dominated by grids and, in that context, resolution means cell size and always has. We talk all the time about 1 degree grids, or 1km grids. This, BTW, is another problem with resolution. If you look at how resolution is described in any large collection of records from NOAA or NASA, it is incredibly inconsistent. Some numbers (with or without units) some with X in the middle (25X25 or is it 25 X 25?), some adjectives (high, low), some with degrees and km in the same string… Making these consistently machine readable and consistently defined across a large collection is a daunting task (I have been there more than once)… BTW, this is why GCMD created a controlled vocabulary for temporal, horizontal, and vertical resolution (their historic approach to enforcing consistency). The values of their keywords nicely illustrate the nature of the problem (note they include point resolution):
"1 km - < 10 km or approximately .01 degree - < .09 degree"
"1 meter - < 30 meters"
"10 km - < 50 km or approximately .09 degree - < .5 degree"
"100 km - < 250 km or approximately 1 degree - < 2.5 degrees"
"100 meters - < 250 meters"
"250 km - < 500 km or approximately 2.5 degrees - < 5.0 degrees"
"250 meters - < 500 meters"
"30 meters - < 100 meters"
"50 km - < 100 km or approximately .5 degree - < 1 degree"
"500 km - < 1000 km or approximately 5 degrees - < 10 degrees"
"500 meters - < 1 km"
"< 1 meter"
"> 1000 km or > 10 degrees"
"Point Resolution"
Anyway, that is the background… ncISO is a very functional tool but it still implements the 95-5 rule. We get you 95% of the way there but there may be some tweaks required… Welcome to the 5%…
Ted
==== Ted Habermann ========================
Director of Earth Science, The HDF Group
Voice: (217) 531-4202
Email: thabermann at hdfgroup.org
==== HDF: Software That Powers Science ====
From: "esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>" <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>>
Reply-To: Aaron Sweeney - NOAA Affiliate <aaron.sweeney at noaa.gov<mailto:aaron.sweeney at noaa.gov>>
Date: Wednesday, February 25, 2015 at 7:51 PM
To: "Bob.Simons at noaa.gov<mailto:Bob.Simons at noaa.gov>" <Bob.Simons at noaa.gov<mailto:Bob.Simons at noaa.gov>>
Cc: "esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>" <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>>
Subject: Re: [Esip-documentation] ACDD-1.3 documentation change request: Descriptions of "resolution" attributes
Thanks for the background on the naming, John. To be clear, Bob, I did not ask for an attribute to be renamed.
Perhaps it's worth pointing out that I, as a new netCDF user and creator, am attempting to the best of my ability to read and understand all relevant documentation, and when questions arise, I am reaching out to a community of experts through this mailing list.
Perhaps some discussion of my efforts thus far will help toward mutual understanding. The netCDF files I am creating are of point feature type (station timeseries data) and comply with the CF conventions for discrete sampling geometries and for standard names. This work was greatly assisted by the existence of template CDL files (and decision tree!) put together by the NODC netCDF Team (thanks, Ajay, and others).
I am also endeavoring to comply with the most recent ACDD documentation (v1.3). I found an inconsistency in the ACDD-1.3 documentation regarding geospatial_[lat|lon|vertical]_resolution. (To sum up, the example in the Description is a text string with a numerical value and uom, while the attribute crosswalk to ISO 19115-2, as documented at http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_(ACDD)_Mappings, maps this attribute to a gco:Measure, which can only contain a numeric value.) This inconsistency came to light when I attempted to generate ISO 19115-2 metadata from my netCDF via TDS (and in particular ncISO) and discovered that the ISO did not validate. I traced the problem back to ACDD, hence my appeal to all of you for clarity.
In doing so, it came to my attention that the intent of the geospatial_[lat|lon|vertical]_resolution was to describe an aspect of gridded data (thanks again, John). Here, I must express some apprehension, since I am not working with a grid feature type, but rather a point feature type. I do not know what the solution is. Should I not even attempt to use geospatial_[lat|lon|vertical]_resolution in my netCDF files?
Cordially,
Aaron
On Wed, Feb 25, 2015 at 6:18 PM, Bob Simons - NOAA Federal <bob.simons at noaa.gov<mailto:bob.simons at noaa.gov>> wrote:
For ACDD 1.4, I welcome suggestions for new attributes.
For ACDD 1.4, I welcome suggestions for changes to the definitions of existing attributes that clarify or improve the wording or add useful examples, but I discourage changes that change the meaning or usage of attributes. It is a standard. We generally accept it as is.
I do not welcome suggestions for changing the names of current attributes. Everybody has that kind of suggestion, but our suggestions often disagree. The current names are used by software and datasets. It's best not to even think about changing them unless there is some absolutely compelling case/need.
I do not welcome suggestions for ACDD version 1.3.1. We haven't done "second decimal" versions before. I think it is a bad idea to ratify new versions of standards frequently. We should debate this and vote on this before we start accepting suggestions for ACDD 1.3.1.
If there is a significant disagreement (as arose here) between how some software is using an attribute and how the attribute is defined in a ratified version of ACDD, I think the software should be changed, not ACDD. (And I'm one of those software guys.) It is hard to imagine situations where that isn't true. That's what standards are for. That's what standards are all about.
On Wed, Feb 25, 2015 at 4:43 PM, David Neufeld - NOAA Affiliate <david.neufeld at noaa.gov<mailto:david.neufeld at noaa.gov>> wrote:
Bob I think you've made a good summary about how standards development and discussions often occur in our community, however I also like to think that new ideas and contributions are welcomed and even encouraged! A discussion list seems like the right place to put those ideas forward even though they may not ultimately be accepted.
Dave
On Wed, Feb 25, 2015 at 2:07 PM, Bob Simons - NOAA Federal via Esip-documentation <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>> wrote:
Aaron, please understand that ACDD, like CF, is a standard. Successive versions of the standard are hammered out by seemingly endless discussions and numerous votes (with the hope of reaching consensus) by people in the community. Once a version of a standard is ratified (as ACDD 1.3 is), you should treat it as if that version were written in stone. Software (like ncISO, THREDDS, ERDDAP) is written to work according to the standard, not the other way around (unless it is something new that is being added to the standard). And if you don't like an attribute's name, (to be blunt) tough. It is what it is. You could have lobbied for some other name before the attribute first made it into ACDD. But now that it is in ACDD, please don't advocate changing it just because you think a slightly different name is more appropriate. Lots of software and thousands of datasets have been written to follow the ACDD standard. They shouldn't have to change just because you think a slightly different name is more appropriate. That's why we try hard to get standards right the first time. That's why we try very hard not to make changes to successive versions of ACDD (and other standards) that conflict with previous versions.
On Wed, Feb 25, 2015 at 12:29 PM, Aaron Sweeney via Esip-documentation <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>> wrote:
Hi, John,
Why are these attributes not called "geospatial_[lat|lon|vertical]_spacing", rather than "geospatial_[lat|lon|vertical]_resolution", if, in fact, the intention, as indicated in their Descriptions, is to capture the "targeted spacing of points"?
My first inclination, upon reading the name of the attribute "geospatial_[lat|lon|vertical]_resolution", was to interpret it to mean "how well the numerical value of [lat|lon|vertical] is resolved."
Cordially,
Aaron
On 02/25/2015 01:19 PM, John Graybeal via Esip-documentation wrote:
I am trying to find a good ncISO rep to be a part of this discussion -- with luck David Neufeld will jump in soon, or people can suggest another rep, perhaps from Unidata?
Not sure yet if we're having a call tomorrow, for now I am assuming we are.
I'm not sure that many current users are using this value the way it is documented: "Information about the targeted spacing of points". That does not mean either the accuracy or precision of the value, but the (intended) distance between points in a grid of values -- but often these values appear in data sets for single fixed stations, which seems invalid.
And as Jim B points out (thank you), the original ISO mapping assumes the grid sizes are georectified -- but some/many grids in earth science are laid out in non-angular constant steps (meters). I assumed we want to support those cases, and no one found fault with the examples -- but maybe they are flawed, if georectified is a constraint. Alternatively, units of meters (or plain, non-geo degree) could be mapped to non-georectified ISO grid resolution terms.
John
On Feb 25, 2015, at 10:59, Jim Biard via Esip-documentation <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>> wrote:
Hi.
I checked this as well, and there should clearly be both a floating-point number and a units for a resolution entry. As Aaron showed in his email, the units go in a 'uom' attribute on the gco:Measure element, and the number is the value of the element. If ncISO isn't doing this, then that sure appears to be an ncISO bug.
There is another issue, of a sort, that this points up. The geospatial_(lat/lon)_resolution attributes in the documentation Aaron listed allow for units that are nonsensical (to me, anyway), such as meters for a quantity that should be restricted to angular units. You can get away with it for latitude, but there is no way to create a longitude axis on a grid that has constant steps in non-angular units.
Grace and peace,
Jim
On 2/25/15 12:32 PM, Aaron Sweeney via Esip-documentation wrote:
Hi, Bob et al.,
I believe that these resolutions are currently mapped to ISO 19115-2 gco:Measures<https://geo-ide.noaa.gov/wiki/index.php?title=Measure>, as in:
geospatial_lat_resolution -->
<gco:Measure uom="[geospatial_lat_unit]">[geospatial_lat_resolution]</gco:Measure>
geospatial_lon_resolution -->
<gco:Measure uom="[geospatial_lon_unit]">[geospatial_lon_resolution]</gco:Measure>
geospatial_vertical_resolution -->
<gco:Measure uom="[geospatial_vertical_unit">[geospatial_vertical_resolution]</gco:Measure>
The GEO-IDE wiki page on gco:Measure (link above) indicates that the value should be an "XML Schema double."
Also, the earlier ACDD-1.1 documentation includes the ISO 19115-2 path as:
/gmi:MI_Metadata/gmd:spatialRepresentationInfo/gmd:MD_Georectified/gmd:axisDimensionProperties/gmd:MD_Dimension/gmd:resolution/gco:Measure
Cordially,
Aaron
On 02/25/2015 10:17 AM, Bob Simons - NOAA Federal wrote:
Is that an error in ACDD or in ncISO? (I don't know. I think it is
debatable.) It is probably easier to change ncISO than to change ACDD
(at least until the next version which is probably far off) and all
datasets which follow the recommendations of the ACDD documentation.
On Wed, Feb 25, 2015 at 9:11 AM, Aaron Sweeney via Esip-documentation
<esip-documentation at lists.esipfed.org><mailto:esip-documentation at lists.esipfed.org> wrote:
Hi, folks,
I want to bring to your attention a problem with the ACDD-1.3
documentation that is leading to the creation of invalid ISO 19115-2
metadata records.
Specifically, I am referring to the Descriptions of the following
Suggested global attributes: geospatial_lat_resolution,
geospatial_lon_resolution, and geospatial_vertical_resolution. For
reference and clarity, I've included these attributes (as well as their
related units attributes) and their descriptions from the ACDD-1.3
documentation below this message.
There are two problems with the Description of these "resolutions."
The first is the recommendation to include both a numerical value and unit.
The ncISO tool expects these resolutions to be numeric only. This leads to
the creation of invalid ISO 19115-2 records. The second problem is that the
Description does not require the unit of resolution to be the same as the
corresponding "geospatial_[lat|lon|vertical]_unit."
I would advocate that the Description of resolution explicitly state
that a numeric value is expected and that the value be expressed in the same
unit as specified in the corresponding "geospatial_[lat|lon|vertical]_unit"
attribute.
Cordially,
Aaron
----relevant ACDD-1.3 documentation follows----
geospatial_lat_units: Units for the latitude axis described in
"geospatial_lat_min" and "geospatial_lat_max" attributes. These are presumed
to be "degree_north"; other options from udunits may be specified instead.
geospatial_lat_resolution: Information about the targeted spacing of points
in latitude. Recommend describing resolution as a number value followed by
the units. Examples: '100 meters', '0.1 degree'
geospatial_lon_units: Units for the longitude axis described in
"geospatial_lon_min" and "geospatial_lon_max" attributes. These are presumed
to be "degree_east"; other options from udunits may be specified instead.
geospatial_lon_resolution: Information about the targeted spacing of points
in longitude. Recommend describing resolution as a number value followed by
units. Examples: '100 meters', '0.1 degree'
geospatial_vertical_units: Units for the vertical axis described in
"geospatial_vertical_min" and "geospatial_vertical_max" attributes. The
default is EPSG:4979 (height above the ellipsoid, in meters); other vertical
coordinate reference systems may be specified. Note that the common
oceanographic practice of using pressure for a vertical coordinate, while
not strictly a depth, can be specified using the unit bar. Examples:
'EPSG:5829' (instantaneous height above sea level), 'EPSG:5831'
(instantaneous depth below sea level).
geospatial_vertical_resolution: Information about the targeted vertical
spacing of points. Example: '25 meters'
--
Aaron D. Sweeney
Water Level Data Manager
Cooperative Institute for Research in Environmental Sciences (CIRES)
University of Colorado at Boulder
and
NOAA National Geophysical Data Center
Marine Geology and Geophysics Division
325 Broadway, E/GC3
Boulder, CO 80305-3328
Phone: 303-497-4797, Fax: 303-497-6513
DISCLAIMER: The contents of this message are mine personally and do not
necessarily reflect any position of NOAA.
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>http://lists.deltaforce.net/mailman/listinfo/esip-documentation
--
Aaron D. Sweeney
Water Level Data Manager
Cooperative Institute for Research in Environmental Sciences (CIRES)
University of Colorado at Boulder
and
NOAA National Geophysical Data Center
Marine Geology and Geophysics Division
325 Broadway, E/GC3
Boulder, CO 80305-3328
Phone: 303-497-4797<tel:303-497-4797>, Fax: 303-497-6513<tel:303-497-6513>
DISCLAIMER: The contents of this message are mine personally and do not necessarily reflect any position of NOAA.
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>http://lists.deltaforce.net/mailman/listinfo/esip-documentation
--
<igjdgbcb.png><http://www.cicsnc.org/>Visit us on
Facebook <http://www.facebook.com/cicsnc> Jim Biard
Research Scholar
Cooperative Institute for Climate and Satellites NC <http://cicsnc.org/>
North Carolina State University <http://ncsu.edu/>
NOAA's National Climatic Data Center <http://ncdc.noaa.gov/>
151 Patton Ave, Asheville, NC 28801
e: jbiard at cicsnc.org<mailto:jbiard at cicsnc.org>
o: +1 828 271 4900<tel:%2B1%20828%20271%204900>
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>
http://lists.deltaforce.net/mailman/listinfo/esip-documentation
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>http://lists.deltaforce.net/mailman/listinfo/esip-documentation
--
Aaron D. Sweeney
Water Level Data Manager
Cooperative Institute for Research in Environmental Sciences (CIRES)
University of Colorado at Boulder
and
NOAA National Geophysical Data Center
Marine Geology and Geophysics Division
325 Broadway, E/GC3
Boulder, CO 80305-3328
Phone: 303-497-4797<tel:303-497-4797>, Fax: 303-497-6513<tel:303-497-6513>
DISCLAIMER: The contents of this message are mine personally and do not necessarily reflect any position of NOAA.
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>
http://lists.deltaforce.net/mailman/listinfo/esip-documentation
--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A (New!)
Monterey, CA 93940 (New!)
Phone: (831)333-9878<tel:%28831%29333-9878> (New!)
Fax: (831)648-8440<tel:%28831%29648-8440>
Email: bob.simons at noaa.gov<mailto:bob.simons at noaa.gov>
The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><
_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>
http://lists.deltaforce.net/mailman/listinfo/esip-documentation
--
Sincerely,
Bob Simons
IT Specialist
Environmental Research Division
NOAA Southwest Fisheries Science Center
99 Pacific St., Suite 255A (New!)
Monterey, CA 93940 (New!)
Phone: (831)333-9878<tel:%28831%29333-9878> (New!)
Fax: (831)648-8440<tel:%28831%29648-8440>
Email: bob.simons at noaa.gov<mailto:bob.simons at noaa.gov>
The contents of this message are mine personally and
do not necessarily reflect any position of the
Government or the National Oceanic and Atmospheric Administration.
<>< <>< <>< <>< <>< <>< <>< <>< <><
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.deltaforce.net/pipermail/esip-documentation/attachments/20150226/166ac355/attachment-0001.html>
More information about the Esip-documentation
mailing list