[Esip-documentation] ACDD 2-3 question

Wed May 21 20:15:10 EDT 2014

That is a typo,

My point was if you just use date_content_modified you are indicating data, metadata and/or format changed.  No way to differentiate between the first two.

The use case by Nan seemed to indicate changing data is very important.  Therefore the user should/must/ use date_values_modified  attribute.  If they do not you have no idea if it was metadata or data changed in file.

Explicitly separating these attributes to describe data vs metadata would avoid this possible confusion.

My two cents…...

On May 21, 2014, at 4:54 PM, John Graybeal <jbgraybeal at mindspring.com<mailto:jbgraybeal at mindspring.com>> wrote:

Is it possible you misread, or we mistyped (or you mistyped)?  I don't think there is a 'data_values_modified' -- I think the two names offered are as you suggest.

Or is your concern that date_content_modified includes changes to the data, and you propose it include just the metadata?  We (small group of we) didn't see a strong use case for this to stand alone, but that the use case for the last time that *anything* in the file was changed was very strong (that's the one lots of people want).

John

On May 21, 2014, at 15:42, "Armstrong, Edward M (398M) via Esip-documentation" <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>> wrote:

Hello all:
I still have some problems with date_content_modified vs data_values_modified because they both contain “data” modified references

Would it not be better to explicitly have a one attribute to refer to data changes (date_values_modified)  and another for metadata changes (date_content_modified )

On May 21, 2014, at 3:30 PM, John Graybeal via Esip-documentation <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>> wrote:

Yay! Love all the comments.

As we seem to have much attention at this moment, Iet me summarize the status: this version has been under discussion/development for over a year, and there is summary documentation (still painfully detailed) of many of the issues that arose, and how most of them were resolved, at the Talk (Discussion) page of the Working document<http://wiki.esipfed.org/index.php/Talk:Attribute_Convention_for_Data_Discovery_1-2_Working>. No obligation to read; just appreciate that the history is there, for the most part. (If you do read it, you'll find other tricky bits too...)

Philip:  Which version is currently a candidate for receiving review comments, and how can one submit comments on that version? I'd rather not take the time reviewing and submitting comments for the wrong version. I am looking at the version history table on this page: http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery

The 'Working' document Anna asked about (the last one in the revision list you cite) is the one that is a candidate for receiving review comments. Comments have been requested on this version on several occasions, so we are all coming to this same point now.

The description of the 1.2beta document is a little off, sorry; that document is the 'final form' document that tracks the working document, so you can see what it would look like in final form. I tried to fix the poor descriptions, but my access to the site has again gone south, so it will have to be later.

Philip: I didn't realize that some of the date attributes had been re-named/purposed in 1-2. Is a rationale being captured somewhere explaining these kinds of changes between versions?

It is my understanding that this email list is the principle 'somewhere' for discussion, and my recollection that this question was discussed briefly on the thread, A Long Time Ago. I know it was discussed on a phone call, and there are several comments on it in the previously mentioned Talk page for the Working document (to which people have been pointed in email and the phone calls).

Ted: I that the intent of these dates has always been to describe important dates for the data in the file rather than the metadata in the file.

The problem IMHO was that this intent -- to consider the values as data, whereas the metadata is not data -- was not clear from the name or description. ("date_created: The date on which the data was created."). Also, please consider your next sentence below….

We went through several clarification rounds on the new definitions -- being unambiguous is hard. Still room for improvement I'm sure, or for rejection of the new approach.

Ted: Whether a data provider chooses to alter the date_modified when they update metadata is up to the provider. If they don't think it is important to users, they don't change the date_modified.

This is explicitly ambiguous, and therefore not useful to those of us who want to know what the date means. It also negates your previous statement. Which helps explain the varied answers, when I asked different people what it meant.

Ted: These new names differentiate between content and values. Does that mean that the data in the file is not content?

Content means all file content, values means only that content which specify netCDF file values. I think these terms are unambiguous. I do not know what your term 'data'  in your question means, but I think I've given you enough information for you to know the answer?

Alternatively, if more people know exactly what 'data' means than know what 'content' and 'values' mean, these definitions are definitely worse.

Anna: However, I don't quite understand why date_product_generated is suggested and the other two are recommended.

My take: It is metadata about the packaging/distribution process, rather than metadata about the content.

Derrick: I think we need to just call a vote and tally up the results.

Do you still feel that way given the sudden waxing of this topic?

Thanks again everyone for all the input.

John

From: Philip Jones - NOAA Affiliate via Esip-documentation <esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>>
Subject: Re: [Esip-documentation] ACDD 2-3 question
Date: May 21, 2014 1:30:40 PM PDT
To: ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>
Cc: esip-documentation at lists.esipfed.org<mailto:esip-documentation at lists.esipfed.org>
Reply-To: Philip Jones - NOAA Affiliate <philip.jones at noaa.gov<mailto:philip.jones at noaa.gov>>

All,

I had raised similar questions as Anna's on ACDD version status at the March telecon. Which version is currently a candidate for receiving review comments, and how can one submit comments on that version? I'd rather not take the time reviewing and submitting comments for the wrong version. I am looking at the version history table on this page: http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery

Also, I didn't realize that some of the date attributes had been re-named/purposed in 1-2. Is a rationale being captured somewhere explaining these kinds of changes between versions?

Thanks.

Phil

From: Ted Habermann <thabermann at hdfgroup.org<mailto:thabermann at hdfgroup.org>>
Subject: Re: [Esip-documentation] ACDD 2-3 question
Date: May 21, 2014 1:04:58 PM PDT
To: "ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>" <ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>>
Cc: "Armstrong, Edward M (398M)" <Edward.M.Armstrong at jpl.nasa.gov<mailto:Edward.M.Armstrong at jpl.nasa.gov>>, "John Graybeal" <jbgraybeal at mindspring.com<mailto:jbgraybeal at mindspring.com>>, Dave Neufeld <david.neufeld at noaa.gov<mailto:david.neufeld at noaa.gov>>, Derrick Snowden <derrick.snowden at noaa.gov<mailto:derrick.snowden at noaa.gov>>, Anna Milan <anna.milan at noaa.gov<mailto:anna.milan at noaa.gov>>, Steve Hankin <steven.c.hankin at noaa.gov<mailto:steven.c.hankin at noaa.gov>>, Aleksandar Jelenak <aleksandar.jelenak at noaa.gov<mailto:aleksandar.jelenak at noaa.gov>>, Richard Signell <rsignell at usgs.gov<mailto:rsignell at usgs.gov>>, "dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>" <dblodgett at usgs.gov<mailto:dblodgett at usgs.gov>>, Bob Simons <bob.simons at noaa.gov<mailto:bob.simons at noaa.gov>>, Ethan Davis <edavis at unidata.ucar.edu<mailto:edavis at unidata.ucar.edu>>

Nan,

I that the intent of these dates has always been to describe important dates for the data in the file rather than the metadata in the file. Whether a data provider chooses to alter the date_modified when they update metadata is up to the provider. If they don't think it is important to users, they don't change the date_modified. These new names differentiate between content and values. Does that mean that the data in the file is not content? Seems like a can of worms to me...

I think somer of our guiding principles are "cause no confusion" and "don't fix it if it ain't broken".  I'm not convinced that this is broken enough to fix...

Ted

On May 21, 2014, at 13:08, Anna Milan - NOAA Federal <anna.milan at noaa.gov<mailto:anna.milan at noaa.gov>> wrote:

Although more wordy, I'm comfortable with the concepts/definitions of the new terms:

date_content_modified
The date on which any of the provided content, including data, metadata, and presented format, was last changed (ISO 8601 format)
date_values_modified
The date on which the provided data values were last changed; excludes metadata and formatting changes (ISO 8601 format)
date_product_generated
The date on which this data file or product was produced/distributed (ISO 8601 format). While this date is like a file timestamp, the date_content_modified and date_values_modified should be used to assess the age of the contents of the file or product.

However, I don't quite understand why date_product_generated is suggested and the other two are recommended. I would expect it to be the other way around. On the other hand,  I also agree with Ted that 'date_created' and 'date_modified' are sufficient and waaaay more straightforward.

Anna
*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
Anna.Milan at noaa.gov<mailto:Anna.Milan at noaa.gov>, 303-497-5099
NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma
*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*

On Wed, May 21, 2014 at 1:53 PM, Nan Galbraith <ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>> wrote:
Hi Ted -

Actually, no, in the context of NetCDF files, I really don't understand
date_modified and date_created.  That might be because my files are
never 'modified', we just write new files whenever we need to change
anything. The original date when the first NetCDF file containing the
data is not something we hang onto, either.

We've had lots of conversations about if/why we need 2 'file dates',
and this is what we came up with:

The end user may not care about the last time I re-wrote a file if all I was
doing was (e.g.) updating the contact info for the instrument's manufacturer,
or updating from ACDD-1.1 to ACDD-1.2, but he might want to know if the
data he's got on his desktop is exactly the same as what I've got on my server
now.

Hence, 2 dates. With the terms 'modified' and  'created' he would have no way
to know; with 'content_modified' and 'values_modified' it's clear.

I think John may have come up with these terms, so I just want to check...
a change to the data would cause both date_content_modified and
date_values_modified to be updated, and a change to some metadata
would require just  date_content_modified to be changed. Is that what you
had in mind, John? I think I missed that part of the discussion ... or I was
at sea and not paying attention.

Cheers -
Nan

On 5/21/14 3:34 PM, Ted Habermann wrote:
All,

I don't remember a discussion of these new terms (#2) and feel like they are more, rather than less, confusing. Everyone understands date_modified and date_created...

Ted

On May 21, 2014, at 1:23 PM, Armstrong, Edward M (398M) <Edward.M.Armstrong at jpl.nasa.gov<mailto:Edward.M.Armstrong at jpl.nasa.gov>> wrote:

hi all:

That does look like the correct working version site.

I’m familiar with point #2. Guess I have not been paying attention. But the fields seem pretty explanatory.

date_content_modified  by itself would indicate no data values were changed. You would need to add date_values_modified  in that case too.

On May 21, 2014, at 12:15 PM, Nan Galbraith <ngalbraith at whoi.edu<mailto:ngalbraith at whoi.edu>> wrote:

Hi All  -

Ah, yes, ACDD ...  where do we stand with these terms,
and with the working version of the document?

I don't recall how or when date_content_modified and
date_values_modified were arrived at, but they're a little
more clear than the old date_modified and date_created,
so that seems pretty good.

Are we sticking with these new terms? Can we move ahead
with finalizing the working version,  or should this be asked
on the Esip-documentation list, or ... have we given up?

Thanks - Nan

On 5/21/14 2:06 PM, Anna Milan - NOAA Federal via Esip-documentation wrote:
Hi All,

I'm helping DSCOVR define their netCDF elements and am using the guidance from this ACDD version:

http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_%28ACDD%29_Working

1. Is this the best version to be using? (They will NOT be using groups)

2. Are the date_modified, date_created, date_issued fields deprecated in favor of the following elements date_content_modified, date_values_modified, date_product_generated fields?

Thanks in advance for your response!

Anna
*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
Anna.Milan at noaa.gov<mailto:Anna.Milan at noaa.gov> <mailto:Anna.Milan at noaa.gov>, 303-497-5099<tel:303-497-5099>
NOAA/NESDIS/NGDC

http://www.ngdc.noaa.gov/metadata/emma
*~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*

--
*******************************************************
* Nan Galbraith        Information Systems Specialist *
* Upper Ocean Processes Group            Mail Stop 29 *
* Woods Hole Oceanographic Institution                *
* Woods Hole, MA 02543                 (508) 289-2444<tel:%28508%29%20289-2444> *
*******************************************************

John Graybeal
jbgraybeal at mindspring.com<mailto:jbgraybeal at mindspring.com>

_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>
http://www.lists.esipfed.org/mailman/listinfo/esip-documentation

-ed

Ed Armstrong
JPL Physical Oceanography DAAC
818 519-7607

_______________________________________________
Esip-documentation mailing list
Esip-documentation at lists.esipfed.org<mailto:Esip-documentation at lists.esipfed.org>
http://www.lists.esipfed.org/mailman/listinfo/esip-documentation

John Graybeal
jbgraybeal at mindspring.com<mailto:jbgraybeal at mindspring.com>

-ed

Ed Armstrong
JPL Physical Oceanography DAAC
818 519-7607

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140522/a7255f78/attachment-0001.html>