[Esip-discovery] Datacasting, collection casting -- these should be harmonized
Ruth Duerr
rduerr at nsidc.org
Mon Jan 28 12:49:12 EST 2013
Hi Steve,
I thought I'd take a couple of minutes to explain a bit more about collection casting and OpenSearch, since it seems that maybe the current version of the RFC isn't quite fitting the bill yet. I'm not sure if this helps or not; so you might let me know.
Collection casting is really about advertising the existence of data sets in a totally location independent and discoverable way. It is called collection casting rather than data set casting, simply because of the confusion of names with datacasting. Outside of NASA, collections are often the terms used for what I usually think of as a data set. The contents of a collection cast typically would be the spatial/temporal coverage of the data set, its name, a description and whatever set of links to additional information or services the provider deems necessary/useful. Those are indeed often links to services like granule level OpenSearch, OpenDAP access endpoint or OGC W*S services; but could also include any other services that work with that data set. For example, at NSIDC we include links to our service to get the complete metadata record (which by the way is available as ISO-19115 metadata as well in NASA DIF and FGDC CGDSM format). Equally reasonable would be to advertise human-aimed services such as Giovanni in the cast.
The format of the entries in a Collection Cast is intentionally the same as theDiscovery cluster's OpenSearch result format The goal is to ensure that both are as simple and compatible as possible. For that reason, we've started with ATOM, not RSS; there are already several RFC's that discuss use of registered MIME types to disambiguate link types, etc.
Is any of that helpful?
Ruth
On Jan 28, 2013, at 8:57 AM, "Steve Richard" <steve.richard at azgs.az.gov> wrote:
> There's a NASA RFC out for a RSS feed to publish metadata about satellite
> image granules as they come in. The key content seems to be an
> rss2:enclosure element with a link to a specific granule file (a kind of
> dataset). Metadata beyond the standard RSS elements is completely soft
> typed, therefore only interoperable within the confines of some community
> profile about the datacasting:customEltDef's.
>
> There's an ESIP RFC for an ATOM feed to publish metadata about collections.
> The intention, based on the RFC, appears to be to enable software clients to
> parse the feed content (atom:link elements) and set up interfaces for users
> to access subsets of a dataset described by the feed. The RFC adopts
> conventions for specifying geographic and temporal coverage (important). The
> interesting thing (IMHO) about the ESIP proposal is the idea of include
> links with opensearch descriptions for how to access subsets of the dataset,
> e.g. using openDAP type requests, or OGC filter for WFS or CSW type
> requests. The fact that there is an interface that allows URL-based subset
> requests seems to be what makes something a collection.
>
> Its unclear to me why the community would need two such similar
> specifications, other than to codify existing implementations. If the NASA
> Datacasting RFC is just describing an existing RSS implementation, perhaps
> it should be named something more like 'NASA satellite data RSS
> conventions'? If the NASA Datacasting proposal is meant to be forward
> looking, perhaps it should utilize some of the ISO metadata conventions that
> are being adopted by NASA
> (http://earthdata.nasa.gov/wiki/main/index.php/ISO_19115_Home), consider
> using ATOM instead of RSS, use registered MIME types?
>
> steve
>
>
>
>
>> -----Original Message-----
>> From: esip-discovery-bounces at lists.esipfed.org [mailto:esip-discovery-
>> bounces at lists.esipfed.org] On Behalf Of Lynnes, Christopher S. (GSFC-6102)
>> Sent: Monday, January 28, 2013 5:39 AM
>> To: Ruth Duerr
>> Cc: Bingham, Andrew W; datacasting at list.jpl.nasa.gov; esip-
>> discovery at lists.esipfed.org
>> Subject: Re: [Esip-discovery] RFC ready to go?
>>
>> Alright-y, I have (hopefully) disambiguated datacasting from collection
> casting in
>> the document, while leaving in a reference to the sibling DatacastingRSS.
>>
>> On Jan 28, 2013, at 1:29 AM, Ruth Duerr <rduerratnsidc at gmail.com> wrote:
>>
>>> Hi Chris,
>>>
>>> I finally did my homework from the ESIP meeting. I seriously think that
> there
>> are enough differences between the Discovery cluster's conventions for
>> OpenSearch/Collection Casting and Andy's Datacasting, that the two things
>> should be kept quite separate. So I disagree with Andy's suggestions
> below. I've
>> put a number of comments in the google doc but was unable to actually edit
> it.
>>>
>>> In any case, DataCasting has its own RFC and I think that is
> appropriate.
>>>
>>> As for what to call the document, I tend to agree with Andy that there
> are a lot
>> of technologies that legitimately fall under the heading of "discovery".
> The ESIP
>> Discovery cluster has only worked on OpenSearch/Collection Casting, so
>> perhaps the title should reflect that.
>>>
>>> Ruth
>>>
>>> On Jan 25, 2013, at 4:34 PM, "Bingham, Andrew W (3880)"
>> <andrew.w.bingham at jpl.nasa.gov> wrote:
>>>
>>>> Chris et al.,
>>>>
>>>> My apologies for not responding earlier - I'm not on the cluster email
> list and
>> received your recent email only via Nga.
>>>>
>>>> Below are my comments to your RFC.
>>>>
>>>> Andy
>>>>
>>>>
>>>>
>>>> 1. Replace Section 7.1.2 with an introduction to Datacasting. Below
> is my
>> proposed text - which is intended to clear-up confusion with the
> peer-reviewed
>> literature.
>>>>
>>>> 7.1.2 Introduction to Datacasting
>>>> Datacasting is a publish-subscribe method where a data provider
> publishes
>> the availability of new data granules (data files) via an XML feed and a
> data
>> consumer subscribes to the feed to learn about recently available data
> (Bingham
>> et al., 2009). The Datacasting Project
> (http://datacasting.jpl.nasa.gov), which
>> was funded under the NASA ACCESS Program (2005 and 2009), developed an
>> RSS-based specification for the XML feeds, which is termed DatacastingRSS.
> A
>> NASA RFC for DatacastingRSS is currently under consideration
>>
> (https://earthdata.nasa.gov/wiki/main/images/a/a6/Datacasting-Standard-RFC-
>> 20121108.pdf). In this specification, data files related to a data set
> are cast out
>> in a single feed. Linkages to the data set information is provided via
> elements
>> embedded in the channel tag; information related to the data file
> (spatial and
>> temporal extent, link to the data file, browse representation etc.) are
> provided
>> via elements embedded in the item tag. The DatacastingRSS specification
> also
>> allows for the definition and addition of "custom elements" that allows
> data
>> providers to add their own elements to the item tag. The default and
> custom
>> elements within the item tags, enables a Datacasting Feed Reader to filter
> lists
>> of items so data consumers can discover and utilize specific files.
>>>>
>>>> In this RFC, we have slightly modified the definition of the term
> "Datacasting"
>> - we consider Datacasting as a publish-subscribe method, but we extend its
> use
>> for announcing the availability of data sets, as well as data files. The
> Atom
>> specification proposed in this RFC therefore unifies the terminology for
>> publishing data sets and data files. However, the specification lacks the
> concept
>> of "custom elements" that exists under DatacastingRSS and therefore limits
> the
>> number of facets by which a user can discover files.
>>>>
>>>> References
>>>> Bingham A.W, S. McCleese, T. Stough, R. G. Deen, K. Hussey and N.
> Toole,
>> 2009, Earth Science Datacasting: Informed Pull and Information
> Integration,
>> IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 47, NO. 10,
>> OCTOBER 2009.
>> http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5191107.
>>>>
>>>>
>>>> 2. The title "ESIP Discovery Conventions" is too broad - I'd like to
> think that
>> DatacastingRSS is an ESIP Discovery Convention, but obviously does not fit
> into
>> your RFC!
>>>> A more accurate title would be : Atom-based Discovery Conventions.
>>>>
>>>> 3. Include tables illustrating the Atom specification for OpenSearch
> and
>> Datacasting respectively. See the DatacastingRSS RFC for an example.
>>>>
>>>>
>>>> ---
>>>> Andrew W Bingham
>>>> andrew.bingham at jpl.nasa.gov
>>>>
>>>> 818-354-1768 (office)
>>>> 818-687-0626 (mobile)
>>>>
>>>>
>>>> On 1/24/13 5:21 PM, "Quach, Nga T (388A)" <Nga.T.Chung at jpl.nasa.gov>
>> wrote:
>>>>
>>>>>
>>>>>
>>>>>
>>>>> Begin forwarded message:
>>>>>
>>>>>> From: "Lynnes, Christopher S. (GSFC-6102)"
>> <christopher.s.lynnes at nasa.gov>
>>>>>> Date: January 24, 2013, 12:31:54 PM PST
>>>>>> To: "<esip-discovery at lists.esipfed.org>" <esip-
>> discovery at lists.esipfed.org>
>>>>>> Subject: [Esip-discovery] RFC ready to go?
>>>>>>
>>>>>> In terms of modifications to the ESIP Discovery RFC, it looks like we
> have
>> reached a stabilization point. I am now going to import it into MS Word
> next
>> week, reformat a little and add table of contents. So if there is
> anything that still
>> needs changing, speak (or write) now or forever etc etc etc.
>>>>>>
>>>>>>
>> https://docs.google.com/document/d/1ymt5vsP2jw2YMVnc2B8uEI0BKTHcT4jS
>> NVQzruLIFPA/edit
>>>>>> --
>>>>>> Dr. Christopher Lynnes NASA/GSFC, Code 610.2 phone:
> 301-614-5185
>>>>>> "Perfection is achieved, not when there is nothing left to add, but
> when
>> there is nothing left to take away" -- A. de Saint-Exupery
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Esip-discovery mailing list
>>>>>> Esip-discovery at lists.esipfed.org
>>>>>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>>>> _______________________________________________
>>>> Esip-discovery mailing list
>>>> Esip-discovery at lists.esipfed.org
>>>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>>>
>>> _______________________________________________
>>> Esip-discovery mailing list
>>> Esip-discovery at lists.esipfed.org
>>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>>
>> --
>> Dr. Christopher Lynnes, NASA/GSFC, ph: 301-614-5185
>>
>>
>>
>> _______________________________________________
>> Esip-discovery mailing list
>> Esip-discovery at lists.esipfed.org
>> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
>
> _______________________________________________
> Esip-discovery mailing list
> Esip-discovery at lists.esipfed.org
> http://www.lists.esipfed.org/mailman/listinfo/esip-discovery
More information about the Esip-discovery
mailing list