As CFD datasets get larger and unsteady computation becomes more common, file I/O is an increasing bottleneck. Methods that write post-processing extracts directly from solvers as they are running have proven to be a solution to this problem. Intelligent Light has long advocated for this approach and recently we were asked to participate in activities to advance the state of the art in this area.
In January, I participated in the DOE's "In Situ Data Management (ISDM)" workshop which focused on defining areas for research intended to ease the analysis and understanding of massive simulations enabled by exa-scale computing. A workshop report is forthcoming.
In February, I was invited to present at a mini-symposium at the SIAM CSE 2019 meeting aimed at avoiding the big data problem through in situ techniques.
If you would like to know more about In Situ Data Management and CFD Data Analytics or to schedule a meeting with Steve Legensky, send email to firstname.lastname@example.org.
Data from Juan D. Colmenares, Svetlana Poroseva, Yulia T. Peet, and Scott M. Murman. "Analysis of uncertainty sources in DNS of a turbulent mixing layer using Nek5000", 2018 Fluid Dynamics Conference, AIAA AVIATION Forum, (AIAA 2018-3226). Simulations were performed on the Pleiades Computer system at NASA Ames Research Center. Images created by Intelligent Light.