[Esip-documentation] The next big thing

[Ted Habermann via Esip-documentation]

All,

Interesting discussion on evolution of standards... My feeling is that standards need to evolve over time, but that the gains in capabilities should be significant enough to justify the evolution. What Bob is saying is that the justification for these changes is not significant enough for him to re-tool. That is important information...

I think it is important to keep in mind that there is a much more significant change on the horizon which is the inclusion of groups in netCDF metadata conventions and HDF files that implement them. It is my opinion that the benefits of this change are very significant and clearly significant enough to motivate re-tooling (maybe not for Bob, but for many others). Of course, that is only my opinion and others could disagree. I initially outlined the case at http://wiki.esipfed.org/index.php/NetCDF,_HDF,_and_ISO_Metadata and will continue to elucidate it in the future.

When I wrote that page, 2.5 years ago, I was thinking that the Object Convention for Data Discovery (name change clarifies that this is a different thing) would include basically only concepts that were in ACDD but those concepts would be represented in objects (groups) that were informed by the ISO standards (which, of course, has groups). In other words, there would still be (potentially) some translation required between names of attributes in OCDD and ISO names and, more importantly, the transform from ISO to OCDD would be lossy.

My thinking on this has changed recently towards essentially using ISO UML models to directly drive the content in the OCDD. There is a direct mapping between ISO UML and XML, so this is essentially saying that there would be a direct relationship between ISO XML and HDF group/attribute structure. Each HDF metadata group would include two required attributes (role and objectType) that would correspond to the role and object types in the ISO UML. I will update the examples on the wiki page to reflect these changes. I suspect that this will also  involve adding namespace attributes to the file and also namespace prefixes to the attribute names, i.e. linkage will become cit:linkage and the cit namespace will be defined in a global attribute.

One advantage of this approach is that it will end up as a set of encoding rules for writing metadata compliant with any convention into an HDF file. For example, if you liked Ecology Metadata Language (EML) you could write EML metadata into an HDF file using a known set of rules and be able to extract that content using a set of inverse rules. Of course, you could also support multiple metadata conventions (say HDF-EOS and CF) in a single file. It also avoids creating yet another discovery convention (YADC).

BTW, it is important to keep in mind that the CF Central Committee has started (yet again) to discuss using groups in data files. This will certainly happen soon. The OCDD is really an attempt to bring the benefits of groups that are motivating that change into the metadata part of the file where they are (at least) equally important.

Anyway, this discussion has re-energized my desire to work on this new approach and I wanted to initiate the discussion here ASAP to get feedback and recruit good ideas from smart people...

Ted

[cid:32323496-C60B-49FF-8310-11CCF46BDC72]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140926/3f3947ed/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SignatureSm.png
Type: image/png
Size: 30402 bytes
Desc: SignatureSm.png
URL: <http://www.lists.esipfed.org/pipermail/esip-documentation/attachments/20140926/3f3947ed/attachment-0001.png>