[Esip-disasters] [ESIP-all] Request for info on "Trusted Data" concepts

Tue Jun 13 21:59:57 EDT 2017

Hi Karen,

I would agree with Peng.  For disasters, trusted data is more important than trusted repositories.  I do not even think that these trusted repositories can actually exist.  Any repository can be hacked fairly easily these days. Tragic example of centralization!

Trusted data for disasters is data the I can have access to 100% of the time, reliably and very fast.  Central repositories are single point failure and/or extremely expensive to maintain.  Not quite in the NASA budget for disasters.  It is actually silly to be required to retrieve data from one single location given the cloud.  I ought to be able to retrieve any dataset by title from potentially many different places and especially over a long weekend when the repo is down.

Turning to distributed repositories allow you to spread the cost and the risk AND improve the speed/availability. This is certainly more affordable to NASA and its international end-users/partners.

You get access to data faster.  It is cheaper to publish it.  Can we make it safer? (aka trusted).Answer is definitely yes.  

Data can be accessed by content rather than by location.  The access url is a cryptographic hash based on the file content.  You know immediately if the file has been tempered with or not.  If you can download it simultaneously and successfully from many repos, you certainly can trust the data.

The other aspect of trust is reproduceability of the actionable information.  This requires linked [meta]-data with the data to allow you to have full access to the data sources in a similar manner (access by content).  You ought to be able to reproduce the product if you have access to the sources and the recipe.

Bonus: All this technology is available today open source as a foundation for a multi-billion dollar emerging industry.

This is actually called the Permanent Web.  Isn’t it something you would trust?

Biggest problem is not technical.

Pat Cappelaere
pat at cappelaere.com

Cell: 410 340 4868
Skype: patrice_cappelaere
gmail: cappelaere at gmail.com

> On Jun 13, 2017, at 12:37 PM, Moe, Karen (GSFC-407.0)[EMERITUS] via Esip-disasters <esip-disasters at lists.esipfed.org> wrote:
> 
> Thank you for all of the good inputs! Peng has a good point about the difference of trusting datasets and data repositories. For my benefit, at least, I think it would be good to include an overview of ICSU WDS work with trusted repositories (that NOAA and USGS, probably among other US agencies have dealt with). 
> 
> Rorie, are there some introductory charts available to explain the high level purpose and motivation for the new WDS–DSA Core Trustworthy Data Repositories certification standard? 
> 
> I think our Disasters cluster could look at repositories and have a follow on discussion (ideally before our July meeting) to look at the differences between repositories and datasets. Our operational communities would likely tap both, so there’s a benefit to addressing both and understanding the key differences. I also hope to have a couple more examples of certification strategies as we move forward.
> 
> Thanks,
> Karen
> 
> From: Ge Peng - NOAA Affiliate <ge.peng at noaa.gov <mailto:ge.peng at noaa.gov>>
> Date: Tuesday, June 13, 2017 at 8:59 AM
> To: Rorie Edmunds <rorie.edmunds at icsu-wds.org <mailto:rorie.edmunds at icsu-wds.org>>, Karen Moe <karen.moe at nasa.gov <mailto:karen.moe at nasa.gov>>, Robert R Downs <rdowns at ciesin.columbia.edu <mailto:rdowns at ciesin.columbia.edu>>
> Cc: ESIP All <esip-all at lists.esipfed.org <mailto:esip-all at lists.esipfed.org>>, Ge Peng <ge.peng at noaa.gov <mailto:ge.peng at noaa.gov>>, Erin Robinson <erinrobinson at esipfed.org <mailto:erinrobinson at esipfed.org>>
> Subject: Re: [ESIP-all] Request for info on "Trusted Data" concepts
> 
>> Thanks a lot, Rorie and Bob, for the timely information on new WDS-DSA Core Trustworthy Data Repositories Requirements and ISO 16363:2012.
>>  
>> A certified trustworthy repository should support and ensure the quality of its overall data holdings with well-defined and managed processes and procedures.  However, currently, most data centers are not certified.
>>  
>> I would also like to draw attention to the difference between data repositories trustworthiness and dataset trustworthiness. They support each other but not the same thing (please see the slides 17 and 18 of this presentation: tinyurl.com/DSMMintro <http://tinyurl.com/DSMMintro>). I think that defining trustworthy datasets needs a holistic and integrated, both vertically and horizontally, approach.
>>  
>> I would like to mention the dataset maturity assessment reference frameworks that have been developed or under-development at NOAA. These assessment models can be used as a part of validating or certifying individual datasets by providing consistent and integratable quality information, combining with trustworthy repositories requirements and ISO 1363:2012 standards.
>>  
>> An overview of the current state of building a consistent framework for curation and presentation of Earth Science data quality can be found at: http://commons.esipfed.org/node/9625 <http://commons.esipfed.org/node/9625>
>>  
>> Hope it helps and please let me know if you have any question or need any additional information.
>>  
>> --- Peng
>> 
>> On Tue, Jun 13, 2017 at 2:20 AM, Rorie Edmunds via ESIP-all <esip-all at lists.esipfed.org <mailto:esip-all at lists.esipfed.org>> wrote:
>>> Dear Colleagues,
>>> 
>>> Since the ICSU World Data System and the new WDS–DSA Core Trustworthy Data Repositories Requirements (https://goo.gl/Z8BDeC <https://goo.gl/Z8BDeC>) have been mentioned, please allow me to respond:
>>> Karen, we would welcome any interactions you would like to have on this subject. Please feel free to get in contact if I can be of any assistance to you. And many thanks to Bill for pointing everyone towards our new, joint certification standard.
>>> Bob, this is a very important and interesting development. It is useful to learn and I will pass it on to the WDS–DSA Standards and Certification Board (https://goo.gl/vXlB4V <https://goo.gl/vXlB4V>).
>>> David, this is also useful information. ICSU-WDS has proposed a panel session at AGU on Certification and Trustworthiness (https://agu.confex.com/agu/fm17/preliminaryview.cgi/Session25933.html <https://agu.confex.com/agu/fm17/preliminaryview.cgi/Session25933.html>), which has now been combined with another traditional session on a similar topic, so I am sure that some synergies can be found.
>>> Best wishes,
>>> Rorie
>>> 
>>> ----
>>> 
>>> Rorie Edmunds, PhD | Programme Officer
>>> World Data System-International Programme Office
>>> c/o NICT, 4-2-1 Nukui-kitamachi, Koganei, Tokyo 184-8795, Japan
>>> Tel: +81-4-2327-6394
>>> rorie.edmunds at icsu-wds.org <mailto:rorie.edmunds at icsu-wds.org> | www.icsu-wds.org <http://www.icsu-wds.org/>
>>> 
>>> 
>>> _______________________________________________
>>> ESIP-all mailing list
>>> ESIP-all at lists.esipfed.org <mailto:ESIP-all at lists.esipfed.org>
>>> http://www.lists.esipfed.org/mailman/listinfo/esip-all <http://www.lists.esipfed.org/mailman/listinfo/esip-all>
>>> 
>> 
>> 
>> 
>> -- 
>> Ge Peng, PhD
>> Research Scholar
>> Cooperative Institute for Climate and Satellites - NC (CICS-NC)/NCSU and
>> NOAA’s National Centers for Environmental Information (NCEI)
>> Center for Weather and Climate (CWC)
>> 151 Patton Ave, Asheville, NC 28801
>> +1 828 257 3009 <>; ge.peng at noaa.gov <mailto:ge.peng at noaa.gov>
>> ORCID: http://orcid.org/0000-0002-1986-9115 <http://orcid.org/0000-0002-1986-9115>
>> Following CICS-NC on Facebook <http://www.facebook.com/cicsnc>
>> 
>> 
> _______________________________________________
> Esip-disasters mailing list
> Esip-disasters at lists.esipfed.org
> http://lists.deltaforce.net/mailman/listinfo/esip-disasters

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.deltaforce.net/pipermail/esip-disasters/attachments/20170613/dcc52e36/attachment-0001.html>