[Esip-discovery] Standard metadata in collectioncasts

Sun Feb 24 18:11:50 EST 2013

Hi Eric,

Sorry for taking so long to get back to you.  It turns out that there isn't any accessible documentation about any of the services that we don't advertise, so I can't directly answer your question.  However, all the public API's are documented at nsidc.org/api, though the OpenSearch documents are in the process of being updated so may not be perfectly correct at the moment.  That being said, the normal route people are supposed to use is to do either subscribe to the NSIDC collection cast feed and notice that an ad for a collection might meet their needs and notice that it has an OpenSearch granule level query.  Then they would access the description document for that lower level service (the link is in the cast and OpenSearch return entry), which defines all the query parameters including things like the "ascending" / "descending" terms described below, to formulate the appropriate granule level OpenSearch queries.  If you know the instrument id, you can also do an OpenSearch query against that instrument to return a list of "granules" that meet the query.  (and of course all of this can be done by machines too, in fact our IceBridge portal does just that).

I note that an undocumented feature is that if instead of invoking the above services you invoke a service of the form http://nsidc.org/api/opensearch/1.1/dataset/<DATASETID>/measurement/ you will get a listing of the measurements (e.g., try http://nsidc.org/api/opensearch/1.1/dataset/NSIDC-0342/measurement/).

Hope that is at least moderately helpful,

Ruth

On Feb 19, 2013, at 2:32 PM, Eric Rozell <rozele at rpi.edu> wrote:

> Sorry, I meant that the collection might be some set of highly related files (e.g., same instrument), that has the same set of variables.  They probably would share the same OPeNDAP directory, or at least share some parent directory a few levels up.
> 
> On Feb 19, 2013, at 4:23 PM, Eric Rozell wrote:
> 
>> Sorry for the ambiguous references...
>> 
>> Yes, I do mean "variable" when I say "parameter",
>> and for dataset, I'm essentially talking about the "file" that you eventually run into if you traverse an OPeNDAP directory.
>> 
>> Basically, for me a "dataset" is something that you can make a DAS/DDS/DDX request against, and a collection would be a set of such files that share the same set of variables.
>> 
>> Hope this clarifies things a bit.
>> 
>> Ruth, could you point me to an example service which returns those variables?
>> 
>> Cheers,
>> -Eric
>> 
>> On Feb 19, 2013, at 4:12 PM, Lynnes, Christopher S. (GSFC-6102) wrote:
>> 
>>> 
>>> On Feb 19, 2013, at 4:06 PM, Ruth Duerr <rduerr at nsidc.org> wrote:
>>> 
>>>> Hi Chris,
>>>> 
>>>> It will be interesting to here what Eric says on this; but yes that is what I was thinking.  I do note that for our equivalent circumstance we even include a separate "variable" which we call Pass which has two directions "ascending" and "descending" so that folks can query for one or the other (or both) at the granule level.  
>>> 
>>> Mind you, in the AIRS case, it would be more useful if we called them "daytime" and "nighttime".
>>>> 
>>>> Ruth
>>>> 
>>>> On Feb 19, 2013, at 2:00 PM, "Lynnes, Christopher S. (GSFC-6102)" <christopher.s.lynnes at nasa.gov> wrote:
>>>> 
>>>>> Yes, language bedevils us here:  "parameter" and "dataset" are both highly ambiguous in this context. Sometimes people use "parameter" to equal "variable". Likewise, "dataset" sometimes means "variable", other times means "granule" and other times "collection". I think we need to use examples as much as possible.
>>>>> 
>>>>> In my language, I am actually trying to avoid the word "parameter" as a synonym for "variable" in the netCDF sense.  So in that case, it's the variables that get the "standard_name" attribute.  
>>>>> 
>>>>> Start with an easy one:  Say we have a data collection "AIRS/Aqua Level 3 Daily standard physical retrieval (AIRS+AMSU)", with an OPeNDAP hierarchy starting at http://acdisc.gsfc.nasa.gov/opendap/ncml/Aqua_AIRS_Level3/AIRX3STD.005/.  This has individual time slices, or granules, for each day, all of which have hte same sets of variables.
>>>>> 
>>>>> For instance, take the variables:
>>>>> RelHumid_AU46, which has CF standard name relative_humidity, and  RelHumid_DU162 which has the same standard name.
>>>>> 
>>>>> One is compiled from the ascending node of the satellite and the second is descending.  Similar variables are to be found in the daily product collection that does not use AMSU, as well as all of the monthly equivalents of these product collections.  Is that what you had in mind?
>>>>> 
>>>>> On Feb 19, 2013, at 3:42 PM, Ruth Duerr <rduerr at nsidc.org> wrote:
>>>>> 
>>>>>> Hi Eric,
>>>>>> 
>>>>>> I am not sure what your definition of dataset is but it sounds like the low level datasets within a file like HDF uses.  Not sure if that is true or not.  If so though, maybe we are using the term collection the same way as you are.  In that case, I should say that some NSIDC OpenSearch services already support what you are talking about (i.e., they return a list of parameters for the data set/collection). However I don't think we are using CF standard names, rather the GCMD parameter list, though I'd have to double check that.  I am also pretty sure that we actually aren't advertising these services as being publicly available (though they are).
>>>>>> 
>>>>>> I should also note that we want to keep the CollectionCast spec the same as the data set level OpenSearch spec for query results.
>>>>>> 
>>>>>> I guess all of this is to say that we would support such an optional list provided it was part of both the CollectionCast and OpenSearch results specs…
>>>>>> 
>>>>>> Ruth
>>>>>> 
>>>>>> On Feb 19, 2013, at 12:15 PM, Eric Rozell <rozele at rpi.edu> wrote:
>>>>>> 
>>>>>>> I'm currently working on a project  interested in harvesting the CF standard names in opendap metadata.
>>>>>>> 
>>>>>>> I thought this might be an interesting use case for collection casts.
>>>>>>> 
>>>>>>> I currently dont have a way of determining which opendap datasets belong to the same collection other than inspecting the URL paths.
>>>>>>> 
>>>>>>> What would be useful is an aggregate feed of the standard parameters provided in a particular collection.  Could we add an (optional) list of parameters from a standard vocabulary that are shared across all datasets from a collection?
>>>>>>> 
>>>>>>> Not sure if this has been previously discussed or not.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Eric
>>>>>>> _______________________________________________
>>>>>>> Esip-discovery mailing list
>>>>>>> Esip-discovery at lists.esipfed.org
>>>>>>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>>>>>> 
>>>>>> _______________________________________________
>>>>>> Esip-discovery mailing list
>>>>>> Esip-discovery at lists.esipfed.org
>>>>>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>>>>> 
>>>>> --
>>>>> Dr. Christopher Lynnes, NASA/GSFC, ph: 301-614-5185
>>>>> 
>>>>> 
>>>> 
>>> 
>>> --
>>> Dr. Christopher Lynnes, NASA/GSFC, ph: 301-614-5185
>>> 
>>> 
>>> 
>>> 
>> 
>> _______________________________________________
>> Esip-discovery mailing list
>> Esip-discovery at lists.esipfed.org
>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>> 
> 
> _______________________________________________
> Esip-discovery mailing list
> Esip-discovery at lists.esipfed.org
> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery