[Esip-preserve] Fwd: ISO/TC 211: preservation

Wed Jul 16 10:31:53 EDT 2014

Some care will be needed in creating a standard for all geospatial data.
Here are a few caveats:

1.  Vertical coordinates are not simple.  See <
http://mtp.mjmahoney.net/www/notes/altitude/altitude.html>
which treats both the relationship between geometric altitude (usually
abbreviated z) and geopotential
altitude (Greek Phi) and the dependence of various formulae on the shape of
the Earth.  While lidars
and radars use time-of-flight to get fairly direct geometric altitudes,
radiosondes and passive IR sensors
reduce data to geopotential.  The important point is that to convert from
geometric altitude to geopotential
for a particular vertical profile, the data source needs to record the
temperature and humidity profile (meaning
values of temperature and humidity as a function of barometric pressure).

Meteorologists and atmospheric scientists use geopotential in their
forecasting models because
winds would flow along geopotential surfaces without changing potential
energy.

For oceanic fields, the relevant ancillary vertical profiles are
temperature and salinity.  In addition, oceanic models
(such as the one that MIT has) can use sigma coordinates for the vertical.
Sigma normalizes the pressure
fields so that this variable is 1 at the bottom of the column and 0 at the
top.  To convert to geopotential
or depth in the ocean, the data need to have the pressure at the bottom of
the column, an additional
piece of data to include.

Of course, if the primary interest is in just the solid surface of the
Earth, this may not be a problem
as long as the interested parties want geometric altitude.  However,
hydrologists will need geopotential
because water can naturally flow "uphill" if the geopotential is lower
further from the Earth's center.
GPS data does not give geopotential, so some care in documentation of
coordinates is in order.

2.  It would seem to be sensible to consult standard glossaries for terms
before finalizing the data
dictionary.  For example, NOAA's National Weather Service has an on-line
glossary at <http://w1.weather.gov/glossary/>
The American Meteorological Society has another at <
http://glossary.ametsoc.org/wiki/Main_Page>.
The World Meteorological Organization also maintains an on-line glossary
that appears to be a bit
fancier: <http://wmo.multicorpora.net/MultiTransWeb/Web.mvc>.  If you just
try a Google search
for WMO Glossary, the first URL gives a list of organizations.

3.  In dealing with basic data sources, it would be helpful to remember
that instruments and platforms
are not standardized.  Some instruments, such as radiosondes, are mass
produced.  In this
case, the vendor usually supplies manuals with operation and calibration
procedures as well as algorithms.
However, aircraft and satellite data management systems for research
measurements are usually one-off
systems.  Likewise, research instruments are one-off systems.  Such one-off
systems do not have a
standardized set of operational modes, formats for exceptions, and types of
housekeeping data.  Basically,
instruments and platforms would have to use the same physics and electronic
parts in the same assemblies
and designs if we were to try to get common exceptions and causes of
anomalous conditions.

Research teams may also introduce specialized data codes for observational
conditions.  For example,
radiosondes ascending through cold cloud layers may collect ice on the
sensing elements.  Observers
may differ in their opinions about how to detect icing and how to annotate
or edit the data record
to indicate the condition.  In satellite systems, the satellite may require
instruments to go into
"safe" mode when they are conducting orbital adjustments or start
tumbling.  Likewise, instruments
may have special calibration modes that have different sequences of data
collection.

4.  As a specific note on vocabulary, the term "resolution" in the data
dictionary may not be defined
in accord with the existing technical standards for the measurements.
Wikipedia's disambiguation
page on this term may be a helpful place to start <
http://en.wikipedia.org/wiki/Resolution>.  The page
on "image resolution" at <http://en.wikipedia.org/wiki/Image_resolution> is
probably not as well documented
as it should be.  The article at <
http://en.wikipedia.org/wiki/Accuracy_and_precision> is closer to the
technical standards.

It has been known for a long time that in many cases (including optical
imaging systems) that
there is a tradeoff between precision and resolution.  One of the classic
papers in this area
is Backus, G. and Gilbert, F., 1970: Uniqueness in the inversion of
inaccurate gross Earth data,
Phil. Trans. of the Roy. Soc., 266, No. 1173, pp. 123-192.  The authors use
the term "resolving
power" instead of resolution, but it is clear that one should expect that
the more local precision
(or reduction in uncertainty) one seek, the lower the resolving power of
the resulting data.

5.  In preserving data, it's clear that to use data sensibly, researchers
need to understand what
they're working with.  Because most Earth science data involve complex
physics and math,
there must be some documentation that provides the derivation of the
algorithms and the
models of the instrument physics.  Likewise, there needs to be some storage
of ancillary
data, such as physical constants (including the spherical harmonic
coefficients that represent
the shape of the geopotential) and calibration data.  It isn't clear
whether this documentation is
"metadata" or some other kind of material.  This material probably needs to
be worked into
web sites in a way that can be used by people with different backgrounds,
perhaps as tutorials.

Bruce R. Barkstrom

On Tue, Jul 15, 2014 at 4:00 PM, Ted Habermann via Esip-preserve <
esip-preserve at lists.esipfed.org> wrote:

>  Wolfgang,
>
>  The train ride worked out quite well. I was the only person in my cabin,
> so the 4-person cabin was great. I did actually get some sleep… Thanks so
> much for your help!
>
>  I am forwarding the preservation draft to the preservation cluster of
> the Earth Science Information Partners. This is a diverse group of
> scientists from NASA, NOAA, and other organizations that are interested in
> preservation. They are a great group that have thought about these problems
> quite a bit. I did note that you still seem to refer to OAIS as being
> related to satellite imagery. I suspect that most of these people think of
> it as being broader than that.
>
>  ESIP - Wolfgang is a friend from Germany that is writing a draft
> preservation proposal for ISO TC 211 building on the OAIS. I spoke with him
> during a recent ISO meeting about preservation and said I would connect him
> with you…
>
>  Thanks to all,
> Ted
>
>
> _______________________________________________
> Esip-preserve mailing list
> Esip-preserve at lists.esipfed.org
> http://www.lists.esipfed.org/mailman/listinfo/esip-preserve
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/pipermail/esip-preserve/attachments/20140716/93551728/attachment.html>