[Esip-documentation] to Re: ACDD convention (2 things)

[John Graybeal]

It might be time for a new specification that is a fresh start building on ACDD. I think several of us felt at the time of the last that we dropped a number of important improvements to maintain historical consistency. The computing world has changed a lot in the last 20 years, and with the new marinedata group in RDA and many other initiatives, there might be critical mass to do the necessary hard work. 

John

Sent from my iPhone

> On May 27, 2021, at 11:31, Bob Simons - NOAA Federal <bob.simons at noaa.gov> wrote:
> 
> 
> NaN,
> 
> As before I will strongly recommend that we don't change existing attribute names. 
> There are now many large systems that have adopted ACDD 1.3, and probably millions of files that now have "id" and many other ACDD 1.3 attribute names. 
> Those files aren't going to be changed. So changing an attribute name in the next version of ACDD just adds confusion to users and software that depends on ACDD since they would now have to look for the old attribute name and the new attribute name.
> This is what happened with the bad (in my opinion) decision (it slipped by me or I would have objected) to change from "acknowlegment" (the US spelling in previous versions of ACDD) to "acknowlegement" (the British spelling in ACDD 1.3). 
> Now there are 10's of 1000's of files that have one and 10's of 1000's of files that have the other. What a mess.
> Plus, any single person might choose to rename several of the attribute names if they were in charge of doing it over from scratch by themselves. Open that up to lots of people and you would be debating changing almost everything. What a mess that would be!
> We debated the attribute names in the past (other than the names Ethan defined when he created ACDD 1.0 by himself). Let's not consider re-debating them now.
> 
> And yes, we can and must insist that data providers in other groups "look up the definitions of our terms". That's what compliance is all about -- not just using the attributes names but using them correctly according to the definition in the specification.  The whole point of the specification is to list the terms and their definitions so that they can be used correctly. If the definitions aren't important or needed, and if we didn't expect people to read the definitions, why did we spend so much time arguing over them?
> 
> Finally, regarding "the 5th most important": I don't think the order of terms in the summary of terms implies a ranking of importance. The only ranking is by the explicitly stated groups "Highly Recommended", "Recommended" and "Suggested". 
> 
> Best wishes.
> 
> 
>> On Thu, May 27, 2021 at 11:30 AM Nan Galbraith <ngalbraith at whoi.edu> wrote:
>> Hi all - 
>> 
>> I also agree, although I don't recall why we didn't set a minimum 
>> standard for compliance.
>> 
>> On another detail about ACDD, the OceanSITES NetCDF spec follows
>> ACDD, and lists ACDD in our 'Conventions' attribute (along with CF and 
>> OceanSITES). We're currently trying to translate our metadata into a
>> standard created by the OceanOPS group for GOOS (Global Ocean Observing
>> System), and I ran into a detail that really boggles my mind. 
>> 
>> The recommended ACDD attribute 'id' - presumably the 5th most important of 
>> our attributes, based on its location in the table, is defined as 'An identifier for 
>> the data set...'  Why we didn't name it 'data_set_id' is beyond me. It's already 
>> been misread as a platform ID by the GOOS translators, and I had to refer back to 
>> our format spec, which quotes ACDD, to understand why that wasn't working 
>> as they'd expected it to.
>> 
>> If this convention is ever updated, just in case I'm not involved, I hereby 
>> recommend that this term be changed to something more meaningful; we
>> can't expect people outside the ESIP community to look up the definitions of
>> our terms, so they should be explicit. 
>> 
>> Cheers - Nan
>> 
>> 
>> On 5/12/21 11:24 AM, John Graybeal via Esip-documentation wrote:
>>> And I fully agree with this perspective also, thanks Bob! (And hi!)
>>> 
>>> john
>>> 
>>>> On May 12, 2021, at 8:20 AM, Bob Simons - NOAA Federal via Esip-documentation <esip-documentation at lists.esipfed.org> wrote:
>>>> 
>>>> My personal answer to your questions is:
>>>> 
>>>> I think you may have a misunderstanding of the ACDD attributes with regard to compliance. ACDD (like CF) defines a set of attributes. Yes, they are categorized as  "highly recommended", "recommended" or "suggested", but note that none are "required". So one might say that, technically, a dataset with none of the ACDD attributes is compliant with ACDD. But it's better to say that a file or dataset is compliant if it uses the ACDD attributes (hopefully all of the "highly recommended"  and "recommended" and many of the others) in a way that is consistent with the attribute definitions. It is not an error or a sign of non-compliance if a dataset doesn't have one or more of the ACDD attributes. Note that some of the attributes simply are not relevant to some files, so those attributes simply shouldn't be used for that file. In that case, their absence is not an error. Also, ACDD (like CF) allows the file to have other attributes, perhaps from other conventions, so the presence of non-ACDD attributes is not an error or sign of non-compliance.. 
>>>> 
>>>> Regarding "the convention does not specify whether data is compliant with ACDD,"
>>>> Basically correct. And there is no official ESIP ACDD compliance checker which looks at a file or dataset's metadata to determine its compliance. However, other groups have made compliance checkers (i.e., software): search the web for these. I think NOAA's IOOS has a compliance checker which includes ACDD. NOAA's NCEI's checker may also include ACDD checking. Note that compliance checkers mostly just say "better" for files that have more of the ACDD attributes (especially the "highly recommended" ones), and "worse" for files that have fewer ACDD attributes, which is what the checker's authors are seeking, but not strictly what the ACDD convention says. And note that compliance checkers currently aren't actually smart enough to evaluate if an attribute value is in compliance with the attribute's specification or to evaluate the quality of the metadata (e.g., does the "title" do a good job of describing the dataset or is it a cryptic code that only the creator understands?). In that sense, it will take AI to make a checker that tests true compliance. I think the only true non-compliance that one of the current checkers might catch is if an ACDD attribute is misspelled or has the wrong data type (e.g., text when a number is expected).
>>>> 
>>>> I hope that helps.
>>>> 
>>>> Best wishes.
>>>> 
>>>> 
>>> 
>>> ----------------------
>>> John Graybeal
>>> Administrator—ESIP Community Ontology Repository
>>> jbgraybeal at sonic.net
>>> 
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> Esip-documentation mailing list
>>> To start a new topic: Esip-documentation at lists.esipfed.org
>>> To unsubscribe and manage prefs: https://lists.esipfed.org/mailman/listinfo/esip-documentation
>> 
>> -- 
>> *******************************************************
>> * Nan Galbraith        Information Systems Specialist *
>> * Upper Ocean Processes Group            Mail Stop 29 *
>> * Woods Hole Oceanographic Institution                *
>> * Woods Hole, MA 02543                 (508) 289-2444 *
>> *******************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.esipfed.org/pipermail/esip-documentation/attachments/20210527/e6b8fa60/attachment.htm>