A View from ORNL: Scientific Data Research Opportunities in the Big Data Age
| Autor: | Greg Eisenhauer, William F. Godoy, Manish Parashar, Ruonan Wang, Arthur B. Maccabe, Jong Choi, E. Suchyta, Scott Klasky, Lipeng Wan, Kshitij Mehta, David Pugmire, James Kress, Tahsin Kurc, Mark Kim, George Ostrouchov, Matthew Wolf, Mark Ainsworth, Chuck Atkins, Norbert Podhorszki, Berk Geveci, Qing Liu, Jeremy Logan |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
business.industry
Computer science Big data 020207 software engineering 02 engineering and technology 01 natural sciences Data science 010305 fluids & plasmas Data modeling Development plan Data visualization Workflow Software SPARK (programming language) 0103 physical sciences Scalability 0202 electrical engineering electronic engineering information engineering business computer computer.programming_language |
| Zdroj: | ICDCS |
| Popis: | One of the core issues across computer and computational science today is adapting to, managing, and learning from the influx of "Big Data". In the commercial space, this problem has led to a huge investment in new technologies and capabilities that are well adapted to dealing with the sorts of human-generated logs, videos, texts, and other large-data artifacts that are processed and resulted in an explosion of useful platforms and languages (Hadoop, Spark, Pandas, etc.). However, translating this work from the enterprise space to the computational science and HPC community has proven somewhat difficult, in part because of some of the fundamental differences in type and scale of data and timescales surrounding its generation and use. We describe a forward-looking research and development plan which centers around the concept of making Input/Output (I/O) intelligent for users in the scientific community, whether they are accessing scalable storage or performing in situ workflow tasks. Much of our work is based on our experience with the Adaptable I/O System (ADIOS 1.X), and our next generation version of the software ADIOS 2.X [1]. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|