Tecplot 360 and FieldView Add Support for VTK File Formats

With popular simulation codes like COMSOL[2], M-Star CFD[3], and Kestrel[4] exporting VTK file formats, the Tecplot 360 and FieldView teams have made the investment to add (and improve) support for these file types.

The VTK file formats are XML-based and come in many flavors[1]:

*.vtu – Unstructured Grid (can contain polyhedral cells)
*.vti – Image Data (structured where all cells are the same size. Cell faces are axis aligned)
*.vtp – Polyhedral Grids (can contain point data without cells – for particles)
*.vts – Structured Grids
*.vtr – Rectilinear Grid (structured where all cell faces are axis aligned)

Within these file formats there are a few different ways they can be written:

• ASCII – Easy to read and write, but highly inefficient. ASCII files are larger than binary files and slow to load.
• Binary Inline – Binary inline files mix the data and XML header information throughout the file.
• Binary Appended – This format organizes all the XML header information at the top of the file, while all the heavyweight data is at the bottom of the file.

The VTK file formats also support various forms of compression. Compression is great for reducing file size, but requires CPU-time to compress/decompress data when writing/reading the file. Whether or not you use compression will depend on whether file size or performance is more important. That said, you might see faster read/write times with compressed files. Since the disk is often the bottleneck, compression/decompression can be handled in parallel on the CPU. This experience will depend on disk speed, network bandwidth, and CPU.

If you’re going to write VTK data, consider using Binary Appended as this is the most efficient format for post-processors to read. Why? Tecplot 360 only needs to read the header information at the top of the file to understand what is going to be loaded. With Binary Inline, the header information is distributed throughout the file which requires scanning through the entire file to gather the header information.

If you have a parallel code – note that it’s common to write one VTK file per MPI-rank and then create a single Parallel VTK file (e.g. *.pvtu) that points to each individual file. While there’s nothing wrong with this on the surface – some file systems have inode[8] limitations or impose limits on the number of files allowed per user, which can make this method of saving files untenable.

Consider writing Tecplot binary or FVUNS instead:

You should also consider writing Tecplot’s PLT or SZPLT or FieldView’s FVUNS. The Tecplot binary file formats are also very efficient, especially the SZPLT file format[6]. Tecplot’s TecIO library offers a simple API for writing data and even includes an MPI version[7] you can write data easily from a parallel CFD code, and to a single file (no inode problems here). All that, and the Tecplot staff can provide support.

The FVUNS format (FieldView Unstructured) is documented in the FieldView documentation and will provide great performance if you use FieldView for post-processing.

References:

1. https://examples.vtk.org/site/VTKFileFormats/
2. https://www.comsol.com/
3. https://mstarcfd.com/
4. https://centers.hpc.mil/CREATE/CREATE-AV.html
5. https://tecplot.com/2018/01/02/3-reasons-post-process-comsol-results-tecplot-360/
6. https://tecplot.com/2018/09/19/comparison-of-tecplot-data-file-formats/
7. https://tecplot.com/2019/03/20/improving-tecio-mpis-parallel-output-performance/
8. https://www.admin-magazine.com/HPC/Articles/What-Is-an-Inode

Appendix

Performance tests were run to compare VTI, VTU, SZPLT, and PLT performance using Tecplot 360. For these tests we used data made available for the SciVis 2018 contest: https://sciviscontest2018.org/

Test 1: Structured Data, VTI vs SZPLT vs PLT

Load 100 timesteps of a 300x300x300 (27M points) asteroid strike simulation and plot a single iso-surface, rendering an image for each timestep. Data resided on a network drive and was loaded across the network, using Tecplot 360 in batch on a Windows 10 VM with 8-cores and 32Gb RAM.

VTI came out the clear winner here because the data file format is very efficient for this type of data. Not only are the XYZ coordinate values generated by rule (not stored in the file), but the solution data uses ZLib compression and is largely composed of integers which compress very well. Furthermore, the plot produced here only needs to load one scalar variable. The VTI file simply loads less data off disk than with the SZPLT and PLT files.

Why is the SZPLT file larger and slower to load than PLT in this case? The SZPLT format contains some additional header information, which makes the files slightly larger. When reading the data, SZL technology reads data in small blocks which is less time efficient than loading one large block. This is more memory efficient though – which can be seen in the memory profile below – you can see that the RAM requirement grows much more slowly with the SZPLT file. In all cases, the RAM limit on the machine is met and Tecplot 360, using Load-On-Demand technology, unloads unnecessary data. The real benefit of SZPLT format is evident with large unstructured grids.

Test 2: Unstructured Data, VTU vs SZPLT vs PLT

Load 1 timestep of an unstructured version of the asteroid strike data and plot a single iso-surface.  This data set is composed of 512 zones and contains a total of 271M elements.  Data resided on a network drive and were loaded across the network, using Tecplot 360 in batch on a Windows 10 Workstation with 32-logical-cores and 128Gb RAM.

In this case, SZPLT is the clear performance winner due to the SZL technology, which allows Tecplot 360 to only load the cells required for the generation of the iso-surface. Since the iso-surface passes through a small fraction of the 271M total cells, Tecplot 360 is able to load a small fraction of the full dataset to produce this image.

The VTU result is smaller on disk than the SZPLT file, due to the use of ZLib compression within the files. The SZPLT file is smaller than the PLT file because the SZPLT format uses a lossless compression of the cell connectivity while PLT does not.

It should be noted that the VTU result is actually composed of 512 separate VTU files, as this simulation was run in parallel on 512 cores. Tecplot’s TecIO-MPI library allows writing of a single SZPLT file from a solver running in parallel through the use of MPI-IO. Some people find dealing with a single file rather than one per core easier, and this helps avoid IT enforced file count limitations.