<div dir="ltr"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style=""><font color="#1d1c1d" face="Slack-Lato, appleLogo, sans-serif"><span style="font-size:15px;font-variant-ligatures:common-ligatures"><br></span></font><span class="gmail-c-mrkdwn__br" style="font-size:15px;box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"><span class="gmail-c-emoji gmail-c-emoji__medium gmail-c-emoji--inline" style="box-sizing:inherit;overflow:visible;display:inline-flex;vertical-align:top;object-fit:contain;width:22px;height:auto"><img src="https://a.slack-edge.com/production-standard-emoji-assets/13.0/apple-medium/1f4c6@2x.png" alt=":calendar:" style="box-sizing: inherit; overflow: hidden; top: 10.9091px; object-fit: contain; height: 22px; width: 22px;"></span></i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"> Meeting Logistics! </i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"><span class="gmail-c-emoji gmail-c-emoji__medium gmail-c-emoji--inline" style="box-sizing:inherit;overflow:visible;display:inline-flex;vertical-align:top;object-fit:contain;width:22px;height:auto"><img src="https://a.slack-edge.com/production-standard-emoji-assets/13.0/apple-medium/1f5a5-fe0f@2x.png" alt=":desktop_computer:" style="box-sizing: inherit; overflow: hidden; top: 10.9091px; object-fit: contain; height: 22px; width: 22px;"></span></i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"> </i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"><span class="gmail-c-emoji gmail-c-emoji__medium gmail-c-emoji--inline" style="box-sizing:inherit;overflow:visible;display:inline-flex;vertical-align:top;object-fit:contain;width:22px;height:auto"><img src="https://a.slack-edge.com/production-standard-emoji-assets/13.0/apple-medium/1f440@2x.png" alt=":eyes:" style="box-sizing: inherit; overflow: hidden; top: 10.9091px; object-fit: contain; height: 22px; width: 22px;"></span></i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"> </i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"><span class="gmail-c-emoji gmail-c-emoji__medium gmail-c-emoji--inline" style="box-sizing:inherit;overflow:visible;display:inline-flex;vertical-align:top;object-fit:contain;width:22px;height:auto"><img src="https://a.slack-edge.com/production-standard-emoji-assets/13.0/apple-medium/1f4ac@2x.png" alt=":speech_balloon:" style="box-sizing: inherit; overflow: hidden; top: 10.9091px; object-fit: contain; height: 22px; width: 22px;"></span></i></b><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><i style="box-sizing:inherit"></i></b><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><br></span></div><div class="gmail_default" style=""><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Topic: </span><b style="font-size:15px;box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">fsspec's ReferenceFileSystem: A Virtual View of the Binary Chunks of any URLs on Another Storage Backend</b><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Monday May 24 26th, 10-11am PT / 1-2pm ET</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><a target="_blank" class="gmail-c-link" href="https://us02web.zoom.us/j/86535177705?pwd=ay9yVDJ6UzNiSGRMWTFxbkNXdEJXUT09" rel="noopener noreferrer" style="font-size:15px;box-sizing:inherit;text-decoration-line:none;font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">https://us02web.zoom.us/j/86535177705?pwd=ay9yVDJ6UzNiSGRMWTFxbkNXdEJXUT09</a><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Meeting ID: 865 3517 7705</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Passcode: 354962</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Find your local number: </span><a target="_blank" class="gmail-c-link" href="https://us02web.zoom.us/u/knxOPNBj5" rel="noopener noreferrer" style="font-size:15px;box-sizing:inherit;text-decoration-line:none;font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">https://us02web.zoom.us/u/knxOPNBj5</a></div><div class="gmail_default" style="font-size:small"><br><span class="gmail-c-mrkdwn__br" style="box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><b style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Abstract:</b><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">The size of datasets needed for scientific applications is such </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">that downloading files to a laptop is no longer feasible. Rather, we </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">must move compute to the data, and access data directly on cloud</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">storage services. This has led to new, specialized data formats, such as Zarr and </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Cloud-Optimised GeoTIFF that enable chunk-wise and parallel IO, </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">as championed, for example, by the Pangeo collaboration.</span><span class="gmail-c-mrkdwn__br" style="box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Many public agencies require their archival data to be stored in a recognised standard </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">format like HDF5. However, translating a dataset to a cloud-optimized format amounts to </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">making a copy of the original, and so uses up resources.</span><span class="gmail-c-mrkdwn__br" style="box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">ReferenceFileSystem is an fsspec implementation which gives a virtual </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">view onto the binary chunks of any URLs on another storage backend. Since HDF5 </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">(and other) data is stored internally as binary encoded chunks, if we can </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">map which chunk belongs where, we can present these binary chunks to Zarr and </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">read the original data as a Zarr dataset, thus eliminating any conversion step. </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Therefore, at the cost of a single scan of the </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">original data (where we store the metadata of the chunk offsets), we get </span><span style="color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">performant parallel IO on many binary formats in archival storage.</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-size:15px;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></div><div class="gmail_default" style=""><font color="#1d1c1d" face="Slack-Lato, appleLogo, sans-serif"><span style="font-size:15px;font-variant-ligatures:common-ligatures"><br></span></font><span class="gmail-c-mrkdwn__br" style="font-size:15px;box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Agenda:</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248);font-size:15px"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">* 20-30 minutes - Presentation</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248);font-size:15px"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">* 20-30 minutes - Discussion questions</span><br style="box-sizing:inherit;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248);font-size:15px"><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">* 10 minutes - Announcements: Working sessions, ESIP Summer Meeting Planning, Open Call for other announcements</span><span class="gmail-c-mrkdwn__br" style="font-size:15px;box-sizing:inherit;display:block;height:8px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)"></span><span style="font-size:15px;color:rgb(29,28,29);font-family:Slack-Lato,appleLogo,sans-serif;font-variant-ligatures:common-ligatures;background-color:rgb(248,248,248)">Looking forward to seeing y'all there!</span><br></div><div class="gmail_default" style="font-size:small"><br></div></div>