The growing availability and power of computing resources, has led in recent years to the production of a huge amount of data. However, writing to disk results of numerical simulations is a serious bottleneck in high-performance computing. Moreover, sometimes data are stored without a specific organization or format, making them no portable neither accessible on other resources, severely limiting their use.
Thus, today issues are both the choice of the best way to write and to read data taking into account of the hardware resources available, like bandwidth and the filesystem's properties; and the choice of the better techniques to write and to read data in order to obtain portable and accessible data.
Aim of this course is to introduce the basis for the usage of the MPI-2 I/O library for binary data, and HDF5 one of the most used standard data formats, that allow for data writing and reading in an organized, efficiency and portable way
Some note about the hardware features affecting the I/O performance are also provided.
Managment of large scientific data. I/O: basic concepts. Parallel file I/O with MPI2. HDF5 library.
Students and researchers of any scientific field, interested in improving their knowledge in production and managment of scientific data.
Experience with Linux operating systems. Knowledge and experience of Fortran, C, or C++. Knowledge and experience of MPI