Last month at the PAR CFD 2015 meeting in Montreal, Canada, I presented a paper entitled "The Impact of In Situ Data Processing and Analytics upon Scaling of CFD Solvers and Workflows".
This work is based on research under our Department of Energy Phase II grant (Award Number DE-SC0007548) and also work under a DOE grant, through the Office of Advanced Scientific Computing Research (Award number DE-SC0012449).
The latter grant is an effort led by Wes Bethel at Lawrence Berkeley National Laboratory in collaboration with Argonne National Laboratory, Georgia Tech and Kitware. In this work we used the AVF-LESLIE code from Prof. Suresh Menon's lab at Georgia Tech, instrumenting it with VisIt/Libsim.
The goal is to determine the overhead associated with in situ processing in comparison to conventional file based volumetric post-processing at scale. For this paper, ‘at scale’ means on the order of 60,000 cores.
Next year we plan to be at 120,000 cores. To date, AVF-LESLIE has been instrumented with VisIt/Libsim and is now able to directly output FieldView XDB files using 40,000 cores and we're working on new in situ data analysis pipelines that can only be performed in situ.