home.aspx

 
.

WORKING WITH DATA IN THE CLOUD

October 28, 2020
USA
SHARESHARESHARE
Data is everywhere, and data is bigger than it used to be. It is no longer practical to download all you need to do your job to your local machine and run your analysis there - download times are too long, and the data won’t fit in memory. These days, data is stored with one of the big cloud vendors or within an institutional data lake. To analyze it, you not only need ways to interact with various remote file storage systems, you also need storage formats that let you access only the data you need, not the whole dataset (bye bye json and excel). This presents an opportunity to process data faster, in parallel, and to distribute and catalog datasets for sharing among teams, rather than copying code and data.