[Esip-documentation] ACDD 2-3 question
Nan Galbraith via Esip-documentation
esip-documentation at lists.esipfed.org
Thu May 22 10:29:04 EDT 2014
The assumption is that the *date_content_modified* would always be
changed when a file is modified (or first written); if the geophysical
data itself is modified, *date_values_modified* would also need to be
updated. So, content_modified will never be before values_modified.
Are we agreed that we need a date representing any changes to the
geophysical data values and another date that's the time the file was
written (by the data originator, that is)?
If we're at that point, the attribute names might be easier.
- file_date - the date on which this version of the file was created
or modified, including geophysical values, quality values, metadata,
or formatting (this is the date of the last time the 'original' copy of
this file was changed in any way)
- file_geophysical_values_date - the date on which the geophysical
data in the file was modified
My concern is not that content includes data, but that there are 'values'
including QC variables and other flag variables that would make
this distinction ... squishy. We should decide (and state) whether 'values'
includes these ancillary variables.
Thanks -
Nan
On 5/21/14 8:15 PM, Armstrong, Edward M (398M) via Esip-documentation wrote:
> That is a typo,
>
> My point was if you just use *date_content_modified *you are
> indicating data, metadata and/or format changed. No way to
> differentiate between the first two.
>
> The use case by Nan seemed to indicate changing data is very
> important. Therefore the user should/must/ use date_values_modified
> attribute. If they do not you have no idea if it was metadata or
> data changed in file.
>
> Explicitly separating these attributes to describe data vs metadata
> would avoid this possible confusion.
>
> My two cents......
>
>
> On May 21, 2014, at 4:54 PM, John Graybeal <jbgraybeal at mindspring.com
> <mailto:jbgraybeal at mindspring.com>> wrote:
>
>> Is it possible you misread, or we mistyped (or you mistyped)? I
>> don't think there is a 'data_values_modified' -- I think the two
>> names offered are as you suggest.
>>
>> Or is your concern that date_content_modified /includes/ changes to
>> the data, and you propose it include just the metadata? We (small
>> group of we) didn't see a strong use case for this to stand alone,
>> but that the use case for the last time that *anything* in the file
>> was changed was very strong (that's the one lots of people want).
>>
>> John
>>
>>
>> On May 21, 2014, at 15:42, "Armstrong, Edward M (398M) via
>> Esip-documentation" <esip-documentation at lists.esipfed.org
>> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>>
>>> Hello all:
>>> I still have some problems with date_content_modified vs
>>> data_values_modified because they both contain "data" modified
>>> references
>>>
>>> Would it not be better to explicitly have a one attribute to refer
>>> to data changes (date_values_modified) and another for metadata
>>> changes (*date_content_modified )*
>>>
>>>
>>> On May 21, 2014, at 3:30 PM, John Graybeal via Esip-documentation
>>> <esip-documentation at lists.esipfed.org
>>> <mailto:esip-documentation at lists.esipfed.org>> wrote:
>>>
>>>> Yay! Love all the comments.
>>>>
>>>> As we seem to have much attention at this moment, Iet me summarize
>>>> the status: this version has been under discussion/development for
>>>> over a year, and there is summary documentation (still painfully
>>>> detailed) of many of the issues that arose, and how most of them
>>>> were resolved, at the Talk (Discussion) page of the Working
>>>> document
>>>> <http://wiki.esipfed.org/index.php/Talk:Attribute_Convention_for_Data_Discovery_1-2_Working>.
>>>> No obligation to read; just appreciate that the history is there,
>>>> for the most part. (If you do read it, you'll find other tricky
>>>> bits too...)
>>>>
>>>>> *Philip*: Which version is currently a candidate for receiving
>>>>> review comments, and how can one submit comments on that version?
>>>>> I'd rather not take the time reviewing and submitting comments for
>>>>> the wrong version. I am looking at the version history table on
>>>>> this page:
>>>>> http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery
>>>>
>>>> The 'Working' document Anna asked about (the last one in the
>>>> revision list you cite) is the one that is a candidate for
>>>> receiving review comments. Comments have been requested on this
>>>> version on several occasions, so we are all coming to this same
>>>> point now.
>>>>
>>>> The description of the 1.2beta document is a little off, sorry;
>>>> that document is the 'final form' document that tracks the working
>>>> document, so you can see what it would look like in final form. I
>>>> tried to fix the poor descriptions, but my access to the site has
>>>> again gone south, so it will have to be later.
>>>>
>>>>> *Philip*: I didn't realize that some of the date attributes had
>>>>> been re-named/purposed in 1-2. Is a rationale being captured
>>>>> somewhere explaining these kinds of changes between versions?
>>>>
>>>> It is my understanding that this email list is the principle
>>>> 'somewhere' for discussion, and my recollection that this question
>>>> was discussed briefly on the thread, A Long Time Ago. I know it was
>>>> discussed on a phone call, and there are several comments on it in
>>>> the previously mentioned Talk page for the Working document (to
>>>> which people have been pointed in email and the phone calls).
>>>>
>>>>> *Ted*: I that the intent of these dates has always been to
>>>>> describe important dates for the *data* in the file rather than
>>>>> the metadata in the file.
>>>>
>>>> The problem IMHO was that this intent -- to consider the values as
>>>> data, whereas the metadata is not data -- was not clear from the
>>>> name or description. ("date_created: The date on which the data was
>>>> created."). Also, please consider your next sentence below....
>>>>
>>>> We went through several clarification rounds on the new definitions
>>>> -- being unambiguous is hard. Still room for improvement I'm sure,
>>>> or for rejection of the new approach.
>>>>
>>>>> *Ted*: Whether a data provider chooses to alter the date_modified
>>>>> when they update metadata is up to the provider. If they don't
>>>>> think it is important to users, they don't change the date_modified.
>>>>
>>>> This is explicitly ambiguous, and therefore not useful to those of
>>>> us who want to know what the date means. It also negates your
>>>> previous statement. Which helps explain the varied answers, when I
>>>> asked different people what it meant.
>>>>
>>>>> *Ted*: These new names differentiate between content and values.
>>>>> Does that mean that the data in the file is not content?
>>>>
>>>> Content means all file content, values means only that content
>>>> which specify netCDF file values. I think these terms are
>>>> unambiguous. I do not know what your term 'data' in your question
>>>> means, but I think I've given you enough information for you to
>>>> know the answer?
>>>>
>>>> Alternatively, if more people know exactly what 'data' means than
>>>> know what 'content' and 'values' mean, these definitions are
>>>> definitely worse.
>>>>
>>>>> *Anna*: However, I don't quite understand why
>>>>> date_product_generated is /suggested /and the other two are
>>>>> /recommended./
>>>>
>>>> My take: It is metadata about the packaging/distribution process,
>>>> rather than metadata about the content.
>>>>
>>>>> *Derrick*: I think we need to just call a vote and tally up the
>>>>> results.
>>>>
>>>> Do you still feel that way given the sudden waxing of this topic?
>>>>
>>>> Thanks again everyone for all the input.
>>>>
>>>> John
>>>>
>>>>> *From: *Philip Jones - NOAA Affiliate via Esip-documentation
>>>>> <esip-documentation at lists.esipfed.org
>>>>> <mailto:esip-documentation at lists.esipfed.org>>
>>>>> *Subject: **Re: [Esip-documentation] ACDD 2-3 question*
>>>>> *Date: *May 21, 2014 1:30:40 PM PDT
>>>>> *To: *ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>
>>>>> *Cc: *esip-documentation at lists.esipfed.org
>>>>> <mailto:esip-documentation at lists.esipfed.org>
>>>>> *Reply-To: *Philip Jones - NOAA Affiliate <philip.jones at noaa.gov
>>>>> <mailto:philip.jones at noaa.gov>>
>>>>>
>>>>> All,
>>>>>
>>>>> I had raised similar questions as Anna's on ACDD version status at
>>>>> the March telecon. Which version is currently a candidate for
>>>>> receiving review comments, and how can one submit comments on that
>>>>> version? I'd rather not take the time reviewing and submitting
>>>>> comments for the wrong version. I am looking at the version
>>>>> history table on this page:
>>>>> http://wiki.esipfed.org/index.php/Category:Attribute_Conventions_Dataset_Discovery
>>>>>
>>>>> Also, I didn't realize that some of the date attributes had been
>>>>> re-named/purposed in 1-2. Is a rationale being captured somewhere
>>>>> explaining these kinds of changes between versions?
>>>>>
>>>>> Thanks.
>>>>>
>>>>> Phil
>>>>
>>>>> *From: *Ted Habermann <thabermann at hdfgroup.org
>>>>> <mailto:thabermann at hdfgroup.org>>
>>>>> *Subject: **Re: [Esip-documentation] ACDD 2-3 question*
>>>>> *Date: *May 21, 2014 1:04:58 PM PDT
>>>>> *To: *"ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>"
>>>>> <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>>
>>>>> *Cc: *"Armstrong, Edward M (398M)"
>>>>> <Edward.M.Armstrong at jpl.nasa.gov
>>>>> <mailto:Edward.M.Armstrong at jpl.nasa.gov>>, "John Graybeal"
>>>>> <jbgraybeal at mindspring.com <mailto:jbgraybeal at mindspring.com>>,
>>>>> Dave Neufeld <david.neufeld at noaa.gov
>>>>> <mailto:david.neufeld at noaa.gov>>, Derrick Snowden
>>>>> <derrick.snowden at noaa.gov <mailto:derrick.snowden at noaa.gov>>, Anna
>>>>> Milan <anna.milan at noaa.gov <mailto:anna.milan at noaa.gov>>, Steve
>>>>> Hankin <steven.c.hankin at noaa.gov
>>>>> <mailto:steven.c.hankin at noaa.gov>>, Aleksandar Jelenak
>>>>> <aleksandar.jelenak at noaa.gov
>>>>> <mailto:aleksandar.jelenak at noaa.gov>>, Richard Signell
>>>>> <rsignell at usgs.gov <mailto:rsignell at usgs.gov>>,
>>>>> "dblodgett at usgs.gov <mailto:dblodgett at usgs.gov>"
>>>>> <dblodgett at usgs.gov <mailto:dblodgett at usgs.gov>>, Bob Simons
>>>>> <bob.simons at noaa.gov <mailto:bob.simons at noaa.gov>>, Ethan Davis
>>>>> <edavis at unidata.ucar.edu <mailto:edavis at unidata.ucar.edu>>
>>>>>
>>>>> Nan,
>>>>>
>>>>> I that the intent of these dates has always been to describe
>>>>> important dates for the *data* in the file rather than the
>>>>> metadata in the file. Whether a data provider chooses to alter the
>>>>> date_modified when they update metadata is up to the provider. If
>>>>> they don't think it is important to users, they don't change the
>>>>> date_modified. These new names differentiate between content and
>>>>> values. Does that mean that the data in the file is not content?
>>>>> Seems like a can of worms to me...
>>>>>
>>>>> I think somer of our guiding principles are "cause no confusion"
>>>>> and "don't fix it if it ain't broken". I'm not convinced that
>>>>> this is broken enough to fix...
>>>>>
>>>>> Ted
>>>>
>>>> On May 21, 2014, at 13:08, Anna Milan - NOAA Federal
>>>> <anna.milan at noaa.gov <mailto:anna.milan at noaa.gov>> wrote:
>>>>
>>>>> Although more wordy, I'm comfortable with the concepts/definitions
>>>>> of the new terms:
>>>>>
>>>>> date_content_modified
>>>>>
>>>>> The date on which any of the provided content, including data,
>>>>> metadata, and presented format, was last changed (ISO 8601 format)
>>>>>
>>>>> date_values_modified
>>>>>
>>>>> The date on which the provided data values were last changed;
>>>>> excludes metadata and formatting changes (ISO 8601 format)
>>>>>
>>>>> date_product_generated
>>>>> The date on which this data file or product was
>>>>> produced/distributed (ISO 8601 format). While this date is
>>>>> like a file timestamp, the date_content_modified and
>>>>> date_values_modified should be used to assess the age of the
>>>>> contents of the file or product.
>>>>>
>>>>> However, I don't quite understand why date_product_generated
>>>>> is /suggested / and the other two are /recommended. /I would
>>>>> expect it to be the other way around. On the other hand, I
>>>>> also agree with Ted that 'date_created' and 'date_modified'
>>>>> are sufficient and waaaay more straightforward.
>>>>>
>>>>>
>>>>>
>>>>
>>>> Anna
>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>> Anna.Milan at noaa.gov <mailto:Anna.Milan at noaa.gov>, 303-497-5099
>>>> NOAA/NESDIS/NGDC
>>>>
>>>> http://www.ngdc.noaa.gov/metadata/emma
>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>
>>>>
>>>> On Wed, May 21, 2014 at 1:53 PM, Nan Galbraith <ngalbraith at whoi.edu
>>>> <mailto:ngalbraith at whoi.edu>> wrote:
>>>>
>>>> Hi Ted -
>>>>
>>>> Actually, no, in the context of NetCDF files, I really don't
>>>> understand
>>>> date_modified and date_created. That might be because my files
>>>> are
>>>> never 'modified', we just write new files whenever we need to
>>>> change
>>>> anything. The original date when the first NetCDF file
>>>> containing the
>>>> data is not something we hang onto, either.
>>>>
>>>> We've had lots of conversations about if/why we need 2 'file
>>>> dates',
>>>> and this is what we came up with:
>>>>
>>>> The end user may not care about the last time I re-wrote a file
>>>> if all I was
>>>> doing was (e.g.) updating the contact info for the instrument's
>>>> manufacturer,
>>>> or updating from ACDD-1.1 to ACDD-1.2, but he might want to
>>>> know if the
>>>> data he's got on his desktop is exactly the same as what I've
>>>> got on my server
>>>> now.
>>>>
>>>> Hence, 2 dates. With the terms 'modified' and 'created' he
>>>> would have no way
>>>> to know; with 'content_modified' and 'values_modified' it's clear.
>>>>
>>>> I think John may have come up with these terms, so I just want
>>>> to check...
>>>> a change to the data would cause both date_content_modified and
>>>> date_values_modified to be updated, and a change to some metadata
>>>> would require just date_content_modified to be changed. Is that
>>>> what you
>>>> had in mind, John? I think I missed that part of the discussion
>>>> ... or I was
>>>> at sea and not paying attention.
>>>>
>>>> Cheers -
>>>> Nan
>>>>
>>>>
>>>>
>>>>
>>>> On 5/21/14 3:34 PM, Ted Habermann wrote:
>>>>> All,
>>>>>
>>>>> I don't remember a discussion of these new terms (#2) and feel
>>>>> like they are more, rather than less, confusing. Everyone
>>>>> understands date_modified and date_created...
>>>>>
>>>>> Ted
>>>>>
>>>>>
>>>>> On May 21, 2014, at 1:23 PM, Armstrong, Edward M (398M)
>>>>> <Edward.M.Armstrong at jpl.nasa.gov
>>>>> <mailto:Edward.M.Armstrong at jpl.nasa.gov>> wrote:
>>>>>
>>>>>> hi all:
>>>>>>
>>>>>> That does look like the correct working version site.
>>>>>>
>>>>>> I'm familiar with point #2. Guess I have not been paying
>>>>>> attention. But the fields seem pretty explanatory.
>>>>>>
>>>>>> *date_content_modified *by itself would indicate no data
>>>>>> values**were changed. You would need to add
>>>>>> date_values_modified in that case too.
>>>>>>
>>>>>>
>>>>>> On May 21, 2014, at 12:15 PM, Nan Galbraith
>>>>>> <ngalbraith at whoi.edu <mailto:ngalbraith at whoi.edu>> wrote:
>>>>>>
>>>>>>> Hi All -
>>>>>>>
>>>>>>> Ah, yes, ACDD ... where do we stand with these terms,
>>>>>>> and with the working version of the document?
>>>>>>>
>>>>>>> I don't recall how or when date_content_modified and
>>>>>>> date_values_modified were arrived at, but they're a little
>>>>>>> more clear than the old date_modified and date_created,
>>>>>>> so that seems pretty good.
>>>>>>>
>>>>>>> Are we sticking with these new terms? Can we move ahead
>>>>>>> with finalizing the working version, or should this be asked
>>>>>>> on the Esip-documentation list, or ... have we given up?
>>>>>>>
>>>>>>> Thanks - Nan
>>>>>>>
>>>>>>>
>>>>>>> On 5/21/14 2:06 PM, Anna Milan - NOAA Federal via
>>>>>>> Esip-documentation wrote:
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I'm helping DSCOVR define their netCDF elements and am
>>>>>>>> using the guidance from this ACDD version:
>>>>>>>>
>>>>>>>> http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_%28ACDD%29_Working
>>>>>>>>
>>>>>>>>
>>>>>>>> 1. Is this the best version to be using? (They will NOT be
>>>>>>>> using groups)
>>>>>>>>
>>>>>>>> 2. Are the date_modified, date_created, date_issued fields
>>>>>>>> deprecated in favor of the following elements
>>>>>>>> date_content_modified, date_values_modified,
>>>>>>>> date_product_generated fields?
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks in advance for your response!
>>>>>>>>
>>>>>>>> Anna
>>>>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>>>>> Anna.Milan at noaa.gov <mailto:Anna.Milan at noaa.gov>
>>>>>>>> <mailto:Anna.Milan at noaa.gov>, 303-497-5099 <tel:303-497-5099>
>>>>>>>> NOAA/NESDIS/NGDC
>>>>>>>>
>>>>>>>> http://www.ngdc.noaa.gov/metadata/emma
>>>>>>>> *~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~*
>>>>>>>>
>>>>
>>>>
>>>> --
>>>> *******************************************************
>>>> * Nan Galbraith Information Systems Specialist *
>>>> * Upper Ocean Processes Group Mail Stop 29 *
>>>> * Woods Hole Oceanographic Institution *
>>>> * Woods Hole, MA 02543(508) 289-2444 <tel:%28508%29%20289-2444> *
>>>> *******************************************************
>>>>
>>>>
>>>>
>>>
>>> John Graybeal
>>> jbgraybeal at mindspring.com <mailto:jbgraybeal at mindspring.com>
>>>
--
*******************************************************
* Nan Galbraith Information Systems Specialist *
* Upper Ocean Processes Group Mail Stop 29 *
* Woods Hole Oceanographic Institution *
* Woods Hole, MA 02543 (508) 289-2444 *
*******************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140522/1f5eec32/attachment-0001.html>
More information about the Esip-documentation
mailing list