[Bessig] Data Rescue and the shutdown

[Ruth Duerr]

Well actually, the Boulder Group went their own way as their lead works for CloudBurst a big data company who was concerned about the availability of transportation data and other govm’t data sets.  They actually set up a multi-petabyte data store, copying whole EPA, etc. servers.  They had an interface to those data but the cloud vendor who donated it reneged after it reached the few petabyte size.  The data still exists but I don’t know how to access it any more.  

Ruth

> On Jan 29, 2019, at 11:21 AM, Cathy Smith <cathy.smith at noaa.gov> wrote:
> 
> All
> 
> The shutdown is over (for now at least) so we don't need the data now. But it would be good if one could get the data if there is another shutdown or if something else happens. To do that, the data would ideally need to be updated to near present and any data fixes applied. Also, the name of the archive/URL would need to be published. And of course it would have to be hosted on a non federal govt computer.
> 
> The specific types of data people I knew wanted were the NCAR Reanalysis (NCAR has this but not in the PSD format), Gridded SST data like the ERSST, OLR, gridded precipitation (CMAP, GPCP), Also station data like the GHCN-M at NCEI. But I'm sure other scientists at universities had other specific data needs.
> 
> Cathy
> 
> On 1/29/19 11:13 AM, Ruth Duerr wrote:
>> I will ask the Boulder data rescue folks as they rescued the most real data I think (whole servers worth).  I have a chat with Joan, their lead, later today and then on Friday.
>> 
>> Ruth
>> 
>>> On Jan 29, 2019, at 10:22 AM, Cathy Smith via Bessig <bessig at lists.esipfed.org> wrote:
>>> 
>>> All
>>> 
>>> After Trump became president, there were large 'data rescue' efforts that were occurring throughout the US. During the government shutdown I thought perhaps NOAA data could be obtained from the archives of those data rescue activities for use by university and other non federal scientists. But I didn't see any reference anywhere during the shutdown to the data that was rescued.
>>> 
>>> Does anyone know if that data is available somewhere? Are the groups who did the hackathons updating data files? It seems the shutdown would have been the perfect opportunity to have made available the rescued data.
>>> 
>>> Also, now that some government orgs have websites with less info than they used to have, is anyone making the webpages that were rescued available to the public?
>>> 
>>> Thanks,
>>> 
>>> Cathy Smith
>>> 
>>> -- 
>>> ----------------------------------------------
>>> NOAA/ESRL PSD and CU CIRES
>>> 303-497-6263
>>> https://www.esrl.noaa.gov/psd/people/cathy.smith/
>>> 
>>> Emails about data/webpages may get quicker responses from emailing
>>> esrl.psd.data at noaa.gov
>>> 
>>> _______________________________________________
>>> Bessig mailing list
>>> Bessig at lists.esipfed.org
>>> https://lists.esipfed.org/mailman/listinfo/bessig
> 
> -- 
> ----------------------------------------------
> NOAA/ESRL PSD and CU CIRES
> 303-497-6263
> https://www.esrl.noaa.gov/psd/people/cathy.smith/
> 
> Emails about data/webpages may get quicker responses from emailing
> esrl.psd.data at noaa.gov
>