[ESIP-all] Guide on the Statement of Uncertainty

Mon Nov 5 14:04:51 EST 2012

A useful introduction to some of the vocabulary on Accuracy and Precision
looks to be the Wikipedia article:
http://en.wikipedia.org/wiki/Accuracy_and_precision

Another Wikipedia article on a similar topic: "Measurement uncertainty" is
http://en.wikipedia.org/wiki/Measurement_uncertainty

The "keeper of the official standards" in this area is the Joint Committee
for Guides in Metrology (JCGM).  For JCGM consult

http://www.bipm.org/en/committees/jc/jcgm/wg1.html

which is a site maintained by the Bureau International des Poids et Mesures
(or - in English instead of French - The International Bureau of Weights and
Measures).  You'll find a couple of US representatives, including one from
NIST on the list of members.

The fundamental reference document is the "Evaluation of measurement data
- Guide to the expression of uncertainty in measurement JCGM 100:2008.  You
can find the complete set of documents at

http://www.bipm.org/en/publications/guides/gum.html

The JCGM also maintains a website through ISO on this subject, at

http://www.iso.org/sites/JCGM/GUM-introduction.htm

Part of the material in this collection is a vocabulary identified as VIM.
See

JCGM 200:2008 International vocabulary of metrology — Basic and general

Of interest to ESIP members and data producers is JCGM 101 on "Propagation
of
distributions using a Monte Carlo method - which is also an ISO/IEC Guide
98-3-1.  Note also that the vocabulary for metrology is also available
through this site.

As well as I can make out, the JCGM has concentrated on verbal definitions
and not on casting their vocabulary into an XML schema.  On the other hand,
they've at least gotten an agreement between French standards bodies and
English ones (the GUM is published in both languages).

A key element in the GUM approach is the measurement model.  In terms of
Earth
science data products, each Product Generation Executable (or, if the data
producer is using an approach like a Graphic Workflow Engine, each of
the functions that reside in a module) provides a measurement model.
Documenting these models is complex because they can reside in
- an unpublished technical report
- source code
- scripts that govern execution
- as published papers
In addition, mathematical models invariably contain coefficients that can
also be documented as
- unpublished technical reports
- tables in technical reports or papers
- embedded in source code
- read in as data files during data production
- read in as scripts

The individual data producers seem to me to be dealing with such different
scenarios for data production that it may be inappropriate to try to develop
a "one size fits all" approach.  There is a vast difference between the
production
problems involved in fitting radiosonde data and satellite sounder data
into a weather
forecast, the high volume data production of climate data records with
multiple
versions of data product lines whose files are reprocessed about once a
year,
and exploratory data production done by producers using a GWE.  Revising
operational code - or climate data record code is VERY expensive and often
requires
large teams of people.

Thus, it isn't clear how these various "residues" of knowledge play
together.
In some ways, their documentation might be a provenance issue.  As a second,
it means that cross-referencing between data and the mathematical models
is going to be a key element for searches.  To make that a bit more
specific,
if you can't find out which set of coefficients an algorithm was using, you
will have a lot of trouble figuring out how uncertain the data are.  In
operational
data production, there are the usual pleasantries of having a team under
intense
pressure to get things working - while the team also has to document what
they
did.

There is also an interaction between the user task (and user background)
and the quality of knowledge required to evaluate uncertainty.  A high
school
student looking for on-line pictures or a citizen scientist helping to
reconstruct meteorological records from ship logs probably do not need
as much background knowledge and mathematical skill as, say someone trying
to reconstruct the uncertainty in radiosonde geopotential thicknesses.

I highly recommend the Ferson, et al. report identified in the Wikipedia
article on
Measurement Uncertainty (Ferson, S., Kreinovich, V., Hajagos, J.,
Oberkampf, W., and Ginzburg, L. 2007. "Experimental Uncertainty Estimation
and Statistics for Data Having Interval Uncertainty". SAND2007-0939.)

The international community (including ISO) behind this area has created
a new framework for dealing with uncertainty.  The GUM recommends that
the term "error" not be used in stating uncertainty.  It's not easy to
abandon the old ideas of "random error" and "bias" or "systematic error".
However, that's what the metrology community has done.

[Brent, if you want to embed these comments into the Info Quality ESIP web
pages, feel free to do so.  I am still fighting with changes to my Ubuntu
interface that happened in the last week.  I guess I'll figure it out with
a bit
more practice.]

Bruce B.

On Mon, Nov 5, 2012 at 11:18 AM, Brent Maddux <brentm at ssec.wisc.edu> wrote:

> Hi Everyone,
>
> The Webex details are below for anyone wishing to join the Information
> Quality Cluster Telecon today.  It will start at Noon Eastern time(8am
> PST), here is the agenda<http://wiki.esipfed.org/index.php/November_2012_IQ_Cluster_Meeting>
> .
>
> Thanks,
> Brent
>
> Nov 5 Information Quality Cluster Meeting Agenda
> 1. Cluster Spinup and Efforts
>
>    - Summer meeting review
>    - Intermediate- and long-term goals
>
> a. [ESDIS 2013 Priority List<https://earthdata.nasa.gov/wiki/main/index.php/The_ESDIS_2013_Priority_List>]
> b. Describing the need and importance of data/info Quality in the Data
> Science Life Cycle, do this through NASA c. Data Quality Resources and
> Definitions Clearinghouse
>
>    - Tangible output for the cluster
>
>  2. Winter Meeting
>
>    - Sessions ideas
>    - Collaboration areas with other committees or clusters
>
> Semantic Web? Preservation Committee?  3. Structure of the Cluster
>
>    - Chairs, Co-Chairs
>    - Long-term collaboration areas with other committees or clusters
>
> 4. Anything Else
>
>    - PLEASE ADD
>
>
>
> On Nov 1, 2012, at 2:45 PM, Erin Robinson wrote:
>
> Hi All -
>
> Here is the call-in info for next Monday, Nov. 5:
>
>
> -----------------------------------------------------------------------------------------------
> To Join the online portion of the Personal Conference meeting
>
> -----------------------------------------------------------------------------------------------
> 1. Go to https://esipfed.webex.com
> 2. Click the “Meeting Center” tab
> 3. Select the Info Quality
> If a password is required, enter the Meeting Password: 23133897
>
>
> -----------------------------------------------------------------------------------------------
> To start the audio portion of the Personal Conference meeting w/ phone
>
> -----------------------------------------------------------------------------------------------
> Call-in toll-free number (US/Canada): 1-877-668-4493
> Call-in toll number (US/Canada): +1-408-600-3600
> Call-in toll number (US/Canada)*: +1-571-918-6008
> Global call-in numbers:
> https://esipfed.webex.com/esipfed/globalcallin.php?serviceType=MC&ED=9910728&tollFree=1
> Toll-free dialing restrictions:
> http://www.webex.com/pdf/tollfree_restrictions.pdf
>
> Attendee access code: 231 338 97
>
> * Call-in toll number (US/Canada) should only be used if the primary
> number does not work.
>
> *Erin Robinson*
> Information and Virtual Community Director
> Foundation for Earth Science | 314.369.9954 | erinrobinson at esipfed.org
> www.esipfed.org
> [image: Facebook]<http://www.facebook.com/pages/Federation-of-Earth-Science-Information-Partners-ESIP/150015055044779>
>  [image: Twitter] <http://www.twitter.com/ESIPFed> [image: Vimeo]<http://www.vimeo.com/esipfed>
>
>
>
> _______________________________________________
> ESIP-all mailing list
> ESIP-all at lists.esipfed.org
> http://www.lists.esipfed.org/mailman/listinfo/esip-all
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.lists.esipfed.org/mailman/private/esip-all/attachments/20121105/ed5136cf/attachment.html>