Intelligent Light and FieldView

In Situ Data Management at DOE Workshop and SIAM CSE

​As CFD datasets get larger and unsteady computation becomes more common, file I/O is an increasing bottleneck. Methods that write post-processing extracts directly from solvers as they are running have proven to be a solution to this problem. Intelligent Light has long advocated for this approach and recently we were asked to participate in activities to advance the state of the art in this area.

In January, I participated in the DOE's "In Situ Data Management (ISDM)" workshop which focused on defining areas for research intended to ease the analysis and understanding of massive simulations enabled by exa-scale computing. A workshop report is forthcoming.

In February, I was invited to present at a mini-symposium at the SIAM CSE 2019 meeting aimed at avoiding the big data problem through in situ techniques.

If you would like to know more about In Situ Data Management and CFD Data Analytics or to schedule a meeting with Steve Legensky, send email to datascience@ilight.com.

Data from Juan D. Colmenares, Svetlana Poroseva, Yulia T. Peet, and Scott M. Murman. "Analysis of uncertainty sources in DNS of a turbulent mixing layer using Nek5000", 2018 Fluid Dynamics Conference, AIAA AVIATION Forum, (AIAA 2018-3226). Simulations were performed on the Pleiades Computer system at NASA Ames Research Center. Images created by Intelligent Light.


Continue reading
  0 Comments
0 Comments

EPISODE: A new user interface to automatically Extract, Reduce and Sort Large CFD Parametric Studies

Intelligent Light R&D continues to produce new tools for large scale CFDParametric Studies. EPISODE is a new large scale data management tool that enables an engineer to readily extract knowledge and insight from their large scale physics based simulations and experimental data. EPISODE provides tools that enable the user to create a relevant subset of their solution results via in-situ data extraction at regions of interest, further reduce the size of that data via proper orthogonal decomposition (POD), and then sort the parametric space of both the input and output solver parameters using self organizing maps (SOMs).

This project consists of new data extracts and compression methods based upon POD (proper orthogonal decomposition) and image compression methods such as JPEG. In addition, we're developing a new UI based upon self organizing maps  which will automatically sort a large number of simulations based upon the parametric inputs and outputs. The result is a set of colored maps that helps to determine the trends in the data.  The user will be able to click on different areas of the map which will then display the results in FieldView. The results may be directly from the CFD solver, reconstructed from the POD or it may be computed from a reduced order model (ROM) that was derived from the CFD and POD results.

The EPISODE project will address limitations of current data analysis tools by:

  • Performing data management and post-processing in-situ, without writing to storage and without direct engineer interaction.

  • Maintain ability to support high-frequency information for maximum temporal fidelity.

  • Handle both experimental and computational data together, supporting automated batch processing.

  • Deliver post-processing capabilities to rapidly collect engineering information such as mass flows through a passage, FFTs, etc.

Key project collaborators include:

  • Robert Haimes from MIT - Haimes' expertise will support the development of a scalable data extracts architecture and plug-in components.

  • Prof. Steven Gorrell from BYU - Professor Gorrell contributes CFD domain expertise in applications for gas turbines.

This work is sponsored by the Air Force Research Laboratory (AFRL) through a Phase II SBIR, Contract FA8650-14-C-2439, and TPOC Michael List.

Continue reading
  0 Comments
0 Comments

Parallel to the People - HPC benefits preview: FieldView 14

Smart CFD - FieldView 14 Will Address More Large Data and Parallel Computing Needs.  

FieildView on Hyperwall

Intelligent Light's Steve Legensky with NASA Ames' Tim Sandstrom stand in front of NASA's Hyperwall. FieldView 13 is already helping NASA exploit HPC enabled CFD.

Smart CFD means addressing today's challenges and preparing for tomorrow's.   Intelligent Light recognizes the pressing needs of the CFD community to manage ever-increasing amounts of data.  We are developing FieldView product and support services to help our customers exploit these demanding trends and deliver accurate, thorough, and timely results to their organizations.

These trends include:

  1. Large files and remote computing requiring effective workflow design and data management.

  2. Enabling broader use of HPC resources.

  3. Driving toward a future where in-situ post-processing becomes an everyday tool freeing users from the burdens of huge volumes of CFD files.


Multi-Windows in FieldView 14

The upcoming FieldView 14 release (available September, 2013) will build on the product's long heritage of supporting HPC users and helping them work productively with their HPC systems both local and remote.  FieldView 14 will deliver CFD post-processing benefits including:

  • New, free and low-cost FieldView tools for working with and sharing XDB files

  • The included high performance dataset reads enabled by FieldView Dataguide for all FieldView 14 users as part of the standard license.  Dataguide eliminates the need to rewrite/reformat your files and automatically reports function, min/max, and location data on read in.

  • "Parallel Power to the People" - As multi-core and parallel systems (SSI and clusters) are a standard part of today's CFD computing workflow, all FieldView licenses will support the use of up to 8 cores per dataset in the standard license

    • No cost upgrades to parallel for annual and supported licenses.

    • Current parallel licenses will receive a no-cost upgrade.

    • No increase in costs for support or renewal for existing users.

FieldView

Continue reading
  0 Comments
0 Comments

Intelligent Light Pioneers High Performance Cloud-Enabled CFD

Intelligent Light is enabling computational fluid dynamics on the cloud, announcing an agreement to make FieldView™, its market-leading CFD post-processing software, available on the cloud computing capability offered by R Systems, a leader in HPC resources on demand.

Providing immediate access to flexible computing capacity, the arrangement gives FieldView users the ability to scale up using parallel processing or scale out with concurrent batch processing to meet capacity needs during peak loads, special projects, or tight deadlines. FieldView’s client-server architecture enables data to remain on the cloud while interactive work is performed from the user’s desktop. In addition, any CFD users who compute on the R Systems cloud can access FieldView for post-processing.

"Everyone is talking about cloud computing, but very few people are talking about successfully using it for CFD simulation," says Steve Legensky, founder and general manager of Intelligent Light. "Our arrangement with R Systems securely and efficiently addresses the challenges that large, complicated datasets can pose on the cloud. Our intention is to enable engineers and researchers to use FieldView-based post-processing as an integral part of the infrastructure of a cloud-based CFD workflow."

"R Systems' high performance computational resources combined with Intelligent Light's FieldView post-processing software provides clients an innovative tool suite that allows for maximum productivity. We see FieldView as an key enabler for CFD users wishing to leverage cloud-based resources," states Brian Kucic, R Systems' business principal. "R Systems is pleased to partner with leading independent software vendors (ISVs) such as Intelligent Light to help increase widespread adoption of HPC resources."

Testing the process

In order to test the viability of CFD post-processing on the cloud, Intelligent Light launched a pilot study, selecting R Systems as the cloud provider. The study, a wind turbine aerodynamics problem with more than 40 cases, encompassed both steady cases for power generation and unsteady cases for wake propagation. The resulting 1.4 terabytes of data were post-processed by FieldView on the cloud in both parallel and concurrent batch modes using FieldView client-server operation. The data remained on the cloud machine in all cases and was remotely accessed from a laptop. With 77,000 core hours of computation, the results proved that cloud-enabled CFD is not only possible, but valuable in terms of time and cost savings.

Accurate 'pay as you go' answers

Organizations with their own HPC resources will find cloud computing useful during peak workloads or for testing out new tools and processes without impacting their production machines, and Legensky believes small and medium size engineering enterprises will also gain significant benefits from cloud-enabled CFD.

"With relatively inexpensive HPC cloud resources and the flexibility and capabilities of FieldView, users can get the exact answers they need on a 'pay as you go' basis," he explains. "We're removing the barriers of cost, infrastructure and specialization, and leveling the CFD playing field for all users."

Legensky notes that, while the high cost of HPC equipment and the significant time and costs related to CFD solvers tend to attract the majority of management attention, the need for robust, reliable post-processing should not be overlooked. In fact, a reliable remote post-processing capability should be considered a critical component of CFD in the cloud. "Post-processing is the critical time when decisions get made," he says. "It's really the most important time in engineering - raw data has become something actionable, something you can learn from. One of FieldView's greatest strengths is its ability to quickly get users from data to decisions."

Automation, batch keys to efficiency

The advent of HPC may mean that ever larger datasets can be run, but without a highly efficient, automated post-processing workflow, "you're exposing yourself to a data tsunami," Legensky says. "You can compute really big solutions in client-server or local modes, but if you have to read the whole file every time, or have too much data to handle, you're losing valuable time. Automation and data management are key."

Automating tasks such as repeating the same set of calculations for hundreds of design variations, or creating images and animations, means increased efficiency, better accuracy, and faster results. Complete backward compatibility ensures that automated routines can be extended for use with newer datasets and future versions of FieldView, thus protecting the investments made in developing automation routines.

FieldView's feature set allows users to easily bridge the gap from interactive to fully automated and reliable batch post-processing. Earlier this year Intelligent Light introduced an innovative batch-only licensing option called FieldView Batch Packs which enable the use of multiple instances of FieldView on an HPC server for concurrent processing at a fraction of the cost of standard FieldView licenses. Concurrent batch processing reduces turnaround time and enables high throughput for transient simulations, both key to streamlining the CFD workflow. Batch operation will be supported on the R Systems cloud service.

Reaping the benefits

The power of a highly efficient CFD workflow is readily apparent in Formula 1 at Red Bull Racing, where more than 80% of their aerodynamic design is driven through CFD and FieldView. With thousands of cores running concurrently, the team's compute requirements are massive, as are the resultant datasets. While in general post-processing can take twice the time of a single solver run, nearly 90% of Red Bull Racing's post-processing tasks are automated via batch processing. Every morning, engineers receive automatically generated FieldView PDF reports, which can run to several hundred pages and include hundreds of animations. By interacting with reports rather than software, the engineering team is free to focus on results, not the process itself. A detailed case study (PDF) is available at the Intelligent Light website.

Recent research conducted by Intelligent Light on simulating the aerodynamics of a bicycle wheel also illustrates the challenges presented by large, complex data and how FieldView’s automation tools help solve them. The study, which features unique customized visualizations, is available here.

Technology, Teamwork, Trust: About Intelligent Light
For more than 25 years, Intelligent Light has been solving the toughest engineering challenges faced by manufacturing and research organizations around the world. Architected for today's high performance computing environments, the company's flagship FieldView™ family of products combines true ease-of-use with the industry's most sophisticated CFD post-processing and large-data visualization capabilities. Intelligent Light's Applied Research Group conducts pure research on the cutting edge of CFD while customizing and delivering real-world solutions to customers in industries such as aerospace, automotive, general manufacturing and turbomachinery. Composed of leading experts in CFD, computer science, visualization and more, with a customer-focused, results-oriented philosophy, Intelligent Light drives CFD simulation for increased productivity, faster answers, deeper insight and maximum return on investment. Visit www.ilight.com for more information.

Fast, Flexible, Freedom: About R Systems
R Systems provides technical expertise and optimized HPC cluster resources to the commercial and academic research communities. The company's flexible service offerings and simplified business model provide the power of world class computing to minimize time, maximize effectiveness, and add value to researchers' HPC results. R Systems' objectives focus on providing fast and flexible HPC implementation, leaving users with the freedom to manage their core research objectives. Supported by expert technical staff, the custom system configurations allow researchers to avoid tedious systems management, making every research project a value-added experience. R Systems' agnostic approach to OS, hardware, and software ensures that a custom solution will match researcher usage requirements and expectations. This results in productive research throughput while saving time and reducing cost. Visit www.rsystemsinc.com for more information.

Continue reading
  0 Comments
0 Comments