REVIEW 72 references
InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations
T0 review · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read InSituNet learns to map simulation, visualization, and viewpoint parameters to images, allowing users to explore new parameter settings of ensemble simulations without rerunning the simulations.
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
The model is a convolutional network that takes simulation, color-mapping, and viewpoint parameters as input and outputs a 256 by 256 image. During training it compares its output with ground-truth images in three ways: pixel differences in a feature space from a pretrained VGG network, and an adversarial discriminator that tries to tell synthetic images from real ones. The paper tests the model on a combustion simulation, a cosmology simulation, and an ocean simulation, reporting pixel and structural similarity scores on held-out parameter settings. It also uses the network's derivatives to show which parameters change the image the most.
Extended reading notes
Core claim
The load-bearing assertion is that a trained InSituNet maps (Psim, Pvis, Pview) to a visualization image I so well that users can explore unseen parameter settings: 'With the trained model, users can generate new images for different simulation parameters under various visualization settings' (Section 1, Equation 1). On the Nyx comparison, InSituNet beats both interpolation and GAN-VR on all four metrics (PSNR 28.47 vs 23.93 and 20.67; SSIM 0.803 vs 0.699 and 0.627; Table 6). If correct, this means a surrogate trained only on in situ images can answer what the visualization would look like across the sampled parameter ranges.
Load-bearing premise
The paper assumes the finite set of in situ images spans a parameter-to-image mapping smooth enough for a convolutional regressor to interpolate unseen combinations, and that 100 sampled viewpoints per ensemble member are sufficient (Section 4: 'taking 100 viewpoints for each ensemble member is sufficient to train InSituNet'). This assumption is load-bearing because arbitrary exploration is only tested on held-out points drawn from the same sampling distribution; the paper never tests extrapolation outside the parameter ranges or with visual mappings not enumerated in training. If the true mapping is not smooth in the chosen parameterization, the predicted images and the sensitivity curves in Section 7.5 could diverge from actual simulations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Assumptions & free parameters
free parameters (6)
- lambda (adversarial loss weight) =
0.01
- network width k =
32 for SmallPoolFire, 48 for Nyx, 48 for MPAS-Ocean
- training iterations =
125000
- viewpoint sample count =
100 per ensemble member
- learning rates alpha_R and alpha_D, beta1 =
5e-5, 2e-4, 0
- training ensemble runs =
3900, 400, 270 per dataset
assumptions (4)
- domain assumption The visualization images of an ensemble simulation vary smoothly enough with simulation, visual mapping, and view parameters for a convolutional regressor to interpolate unseen parameter combinations.
- ad hoc to paper Pretrained VGG-19 features, developed on natural images, are informative for comparing scientific visualization images.
- domain assumption Adversarial training with spectral normalization and TTUR converges to a useful regressor rather than mode collapse or instability.
- domain assumption The L1 norm of the generated image and its derivatives reflect scientifically meaningful parameter sensitivity.
Cite this review
Pith. "Pith review of InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations." pith.science (2026). https://pith.science/paper/CVXGRMGM
@misc{pith2026190800407,
author = {Pith},
title = {Pith review of: InSituNet: Deep Image Synthesis for Parameter Space Exploration of Ensemble Simulations},
year = {2026},
howpublished = {\url{https://pith.science/paper/CVXGRMGM}},
note = {Machine review of arXiv:1908.00407}
}
read the original abstract
We propose InSituNet, a deep learning based surrogate model to support parameter space exploration for ensemble simulations that are visualized in situ. In situ visualization, generating visualizations at simulation time, is becoming prevalent in handling large-scale simulations because of the I/O and storage constraints. However, in situ visualization approaches limit the flexibility of post-hoc exploration because the raw simulation data are no longer available. Although multiple image-based approaches have been proposed to mitigate this limitation, those approaches lack the ability to explore the simulation parameters. Our approach allows flexible exploration of parameter space for large-scale ensemble simulations by taking advantage of the recent advances in deep learning. Specifically, we design InSituNet as a convolutional regression model to learn the mapping from the simulation and visualization parameters to the visualization results. With the trained model, users can generate new images for different simulation parameters under various visualization settings, which enables in-depth analysis of the underlying ensemble simulations. We demonstrate the effectiveness of InSituNet in combustion, cosmology, and ocean simulations through quantitative and qualitative evaluations.
Figures
Figures from the paper (11 more)
Reference graph
Works this paper leans on
-
[1]
J. Ahrens, S. Jourdain, P. O’Leary, J. Patchett, D. H. Rogers, P. Fasel, A. Bauer, M. Petersen, F. Samsel, and B. Boeckel. In situ MPAS-Ocean image-based visualization. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Visualization & Data Analytics Showcase, 2014
work page 2014
-
[2]
J. Ahrens, S. Jourdain, P. O’Leary, J. Patchett, D. H. Rogers, and M. Pe- tersen. An image-based approach to extreme scale in situ visualization and analysis. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 424–434, 2014
work page 2014
- [3]
-
[4]
A. S. Almgren, J. B. Bell, M. J. Lijewski, Z. Luki ´c, and E. V . Andel. Nyx: A massively parallel AMR code for computational cosmology. The Astrophysical Journal, 765(1):39, 2013
work page 2013
-
[5]
A. C. Bauer, H. Abbasi, J. Ahrens, H. Childs, B. Geveci, S. Klasky, K. Moreland, P. O’Leary, V . Vishwanath, B. Whitlock, and E. W. Bethel. In situ methods, infrastructures, and applications on high performance computing platforms. Computer Graphics Forum, 35(3):577–597, 2016
work page 2016
- [6]
-
[7]
S. Bergner, M. Sedlmair, T. M ¨oller, S. N. Abdolyousefi, and A. Saad. ParaGlide: Interactive parameter space partitioning for computer simu- lations. IEEE Transactions on Visualization and Computer Graphics , 19(9):1499–1512, 2013
work page 2013
-
[8]
T. Biedert and C. Garth. Contour tree depth images for large data visual- ization. In Proceedings of Eurographics Symposium on Parallel Graphics and Visualization, pp. 77–86, 2015
work page 2015
Show all 72 references
-
[9]
Biswas, G
A. Biswas, G. Lin, X. Liu, and H.-W. Shen. Visualization of time-varying weather ensembles across multiple resolutions. IEEE Transactions on Visualization and Computer Graphics, 23(1):841–850, 2017
2017
-
[10]
A. Bock, A. Pembroke, M. L. Mays, L. Rastaetter, T. Ropinski, and A. Ynnerman. Visual verification of space weather ensemble simulations. In Proceedings of 2015 IEEE Scientific Visualization Conference , pp. 17–24, 2015
2015
-
[11]
Booshehrian, T
M. Booshehrian, T. M¨oller, R. M. Peterman, and T. Munzner. Vismon: Facilitating analysis of trade-offs, uncertainty, and sensitivity in fisheries management decision making. In Computer Graphics Forum, vol. 31, pp. 1235–1244. Wiley Online Library, 2012
2012
-
[12]
Bremer, G
P.-T. Bremer, G. H. Weber, J. Tierny, V . Pascucci, M. S. Day, and J. B. Bell. Interactive exploration and analysis of large-scale simulations using topology-based data segmentation. IEEE Transactions on Visualization and Computer Graphics, 17(9):1307–1324, 2011
2011
-
[13]
Brock, J
A. Brock, J. Donahue, and K. Simonyan. Large scale GAN training for high fidelity natural image synthesis. In Proceedings of International Conference on Learning Representations, 2019
2019
-
[14]
Bruckner and T
S. Bruckner and T. M¨oller. Result-driven exploration of simulation param- eter spaces for visual effects design. IEEE Transactions on Visualization and Computer Graphics, 16(6):1468–1476, 2010
2010
-
[15]
H. Chen, S. Zhang, W. Chen, H. Mei, J. Zhang, A. Mercer, R. Liang, and H. Qu. Uncertainty-aware multidimensional ensemble data visualiza- tion and exploration. IEEE Transactions on Visualization and Computer Graphics, 21(9):1072–1086, 2015
2015
-
[16]
Coffey, C.-L
D. Coffey, C.-L. Lin, A. G. Erdman, and D. F. Keefe. Design by dragging: An interface for creative forward and inverse design with simulation en- sembles. IEEE Transactions on Visualization and Computer Graphics, 19(12):2783–2791, 2013
2013
-
[17]
Di and F
S. Di and F. Cappello. Fast error-bounded lossy HPC data compression with SZ. In Proceedings of International Parallel and Distributed Pro- cessing Symposium, pp. 730–739, 2016
2016
-
[18]
C. Dong, C. C. Loy, K. He, and X. Tang. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2):295–307, 2016
2016
-
[19]
Dosovitskiy, J
A. Dosovitskiy, J. T. Springenberg, and T. Brox. Learning to generate chairs with convolutional neural networks. In Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1538–1546, 2015
2015
-
[20]
Dutta, C.-M
S. Dutta, C.-M. Chen, G. Heinlein, H.-W. Shen, and J.-P. Chen. In situ distribution guided analysis and visualization of transonic jet engine simulations. IEEE Transactions on Visualization and Computer Graphics, 23(1):811–820, 2017
2017
-
[21]
Fernandes, S
O. Fernandes, S. Frey, F. Sadlo, and T. Ertl. Space-time volumetric depth images for in-situ visualization. In Proceedings of 2014 IEEE Symposium on Large Data Analysis and Visualization, pp. 59–65, 2014
2014
-
[22]
S. Frey, F. Sadlo, and T. Ertl. Explorable volumetric depth images from ray- casting. In Proceedings of 2013 XXVI Conference on Graphics, Patterns and Images, pp. 123–130, 2013
2013
-
[23]
L. A. Gatys, A. S. Ecker, and M. Bethge. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576, 2015
2015 arXiv
-
[24]
L. A. Gatys, A. S. Ecker, and M. Bethge. Texture synthesis using convolu- tional neural networks. In Proceedings of Advances in Neural Information Processing Systems, pp. 262–270, 2015
2015
-
[25]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio. Generative adversarial nets. In Proceedings of Advances in Neural Information Processing Systems, pp. 2672–2680, 2014
2014
-
[26]
J. Han, J. Tao, and C. Wang. FlowNet: A deep learning framework for clus- tering and selection of streamlines and stream surfaces.IEEE Transactions on Visualization and Computer Graphics, 2018, Early Access
2018
-
[27]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778, 2016
2016
-
[28]
Heusel, H
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of Advances in Neural Information Processing Systems, pp. 6626–6637, 2017
2017
-
[29]
Hochreiter and J
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997
1997
-
[30]
F. Hong, J. Zhang, and X. Yuan. Access pattern learning with long short- term memory for parallel particle tracing. In Proceedings of 2018 IEEE Pacific Visualization Symposium, pp. 76–85, 2018
2018
-
[31]
X. Hou, L. Shen, K. Sun, and G. Qiu. Deep feature consistent varia- tional autoencoder. In Proceedings of 2017 IEEE Winter Conference on Applications of Computer Vision, pp. 1133–1141, 2017
2017
-
[32]
Johnson, A
J. Johnson, A. Alahi, and F.-F. Li. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of European Conference on Computer Vision, pp. 694–711, 2016
2016
-
[33]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of International Conference on Learning Representations, 2015
2015
-
[34]
Kurach, M
K. Kurach, M. Lucic, X. Zhai, M. Michalski, and S. Gelly. The GAN landscape: Losses, architectures, regularization, and normalization. arXiv preprint arXiv:1807.04720, 2018
2018 arXiv
-
[35]
Lakshminarasimhan, N
S. Lakshminarasimhan, N. Shah, S. Ethier, S.-H. Ku, C. S. Chang, S. Klasky, R. Latham, R. Ross, and N. F. Samatova. ISABELA for effec- tive in situ compression of scientific data. Concurrency and Computation: Practice and Experience, 25(4):524–540, 2013
2013
-
[36]
Ledig, L
C. Ledig, L. Theis, F. Husz´ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. Photo-realistic sin- gle image super-resolution using a generative adversarial network. In Proceedings of 2017 IEEE Conference on Computer Vision and Pat...
2017
-
[37]
Luˇci´c, K
M. Luˇci´c, K. Kurach, M. Michalski, S. Gelly, and O. Bousquet. Are GANs created equal? A large-scale study. In Proceedings of Advances in Neural Information Processing Systems, pp. 700–709, 2018
2018
-
[38]
K.-L. Ma. In situ visualization at extreme scale: Challenges and op- portunities. IEEE Computer Graphics and Applications , 29(6):14–19, 2009
2009
-
[39]
Matkovi´c, D
K. Matkovi´c, D. Graˇcanin, B. Klarin, and H. Hauser. Interactive visual analysis of complex scientific data as families of data surfaces. IEEE Transactions on Visualization and Computer Graphics, 15(6):1351–1358, 2009
2009
-
[40]
Mirza and S
M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014
2014 arXiv
-
[41]
Miyato, T
T. Miyato, T. Kataoka, M. Koyama, and Y . Yoshida. Spectral normalization for generative adversarial networks. In Proceedings of International Conference on Learning Representations, 2018
2018
-
[42]
Miyato and M
T. Miyato and M. Koyama. cGANs with projection discriminator. In Proceedings of International Conference on Learning Representations , 2018
2018
-
[43]
Nair and G
V . Nair and G. E. Hinton. Rectified linear units improve restricted boltz- 10 © 2019 IEEE. This is the author’s version of the article that has been published in IEEE Transactions on Visualization and Computer Graphics. The final version of this record is available at: 10.1109/...
2019
-
[44]
Obermaier, K
H. Obermaier, K. Bensema, and K. I. Joy. Visual trends analysis in time- varying ensembles. IEEE Transactions on Visualization and Computer Graphics, 22(10):2331–2342, 2016
2016
-
[45]
Orban, D
D. Orban, D. F. Keefe, A. Biswas, J. Ahrens, and D. Rogers. Drag and track: A direct manipulation interface for contextualizing data instances within a continuous parameter space. IEEE Transactions on Visualization and Computer Graphics, 25(1):256–266, 2019
2019
-
[46]
Pathak, P
D. Pathak, P. Kr¨ahenb¨uhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544, 2016
2016
-
[47]
Piringer, W
H. Piringer, W. Berger, and J. Krasser. Hypermoval: Interactive visual validation of regression models for real-time simulation. In Computer Graphics Forum, vol. 29, pp. 983–992. Wiley Online Library, 2010
2010
-
[48]
J. Poco, A. Dasgupta, Y . Wei, W. Hargrove, C. R. Schwalm, D. N. Huntzinger, R. Cook, E. Bertini, and C. T. Silva. Visual reconciliation of alternative similarity spaces in climate modeling. IEEE Transactions on Visualization and Computer Graphics, 20(12):1923–1932, 2014
1923
-
[49]
Potter, A
K. Potter, A. Wilson, P.-T. Bremer, D. Williams, C. Doutriaux, V . Pas- cucci, and C. R. Johnson. Ensemble-vis: A framework for the statistical visualization of ensemble data. In 2009 IEEE International Conference on Data Mining Workshops, pp. 233–240. IEEE, 2009
2009
-
[50]
S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee. Gener- ative adversarial text to image synthesis. In Proceedings of International Conference on Machine Learning, pp. 1060–1069, 2016
2016
-
[51]
Ringler, M
T. Ringler, M. Petersen, R. L. Higdon, D. Jacobsen, P. W. Jones, and M. Maltrud. A multi-resolution approach to global ocean modeling.Ocean Modelling, 69:211–232, 2013
2013
-
[52]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In D. E. Rumelhart, J. L. McClel- land, and CORPORATE PDP Research Group, eds., Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, ...
1986
-
[53]
Samsel, M
F. Samsel, M. Petersen, G. Abram, T. L. Turton, D. Rogers, and J. Ahrens. Visualization of ocean currents and eddies in a high-resolution global ocean-climate model. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis...
2015
-
[54]
A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120, 2013
2013 arXiv
-
[55]
Sedlmair, C
M. Sedlmair, C. Heinzl, S. Bruckner, H. Piringer, and T. M¨oller. Visual parameter space analysis: A conceptual framework. IEEE Transactions on Visualization and Computer Graphics, 20(12):2161–2170, 2014
2014
-
[56]
D. Shepard. A two-dimensional interpolation function for irregularly- spaced data. In Proceedings of the 1968 23rd ACM National Conference, pp. 517–524, 1968
1968
-
[57]
Simonyan and A
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of International Conference on Learning Representations, 2015
2015
-
[58]
Splechtna, K
R. Splechtna, K. Matkovi´c, D. Graˇcanin, M. Jelovi´c, and H. Hauser. Inter- active visual steering of hierarchical simulation ensembles. InProceedings of 2015 IEEE Conference on Visual Analytics Science and Technology, pp. 89–96, 2015
2015
-
[59]
Tikhonova, C
A. Tikhonova, C. D. Correa, and K.-L. Ma. Explorable images for visu- alizing volume data. In Proceedings of 2010 IEEE Pacific Visualization Symposium, pp. 177–184, 2010
2010
-
[60]
Tikhonova, C
A. Tikhonova, C. D. Correa, and K.-L. Ma. An exploratory technique for coherent visualization of time-varying volume data. Computer Graphics Forum, 29(3):783–792, 2010
2010
-
[61]
Tikhonova, C
A. Tikhonova, C. D. Correa, and K.-L. Ma. Visualization by proxy: A novel framework for deferred interaction with volume data. IEEE Transactions on Visualization and Computer Graphics, 16(6):1551–1559, 2010
2010
-
[62]
Torsney-Weir, A
T. Torsney-Weir, A. Saad, T. M¨oller, H.-C. Hege, B. Weber, J.-M. Ver- bavatz, and S. Bergner. Tuner: Principled parameter finding for image segmentation algorithms using visual response surface exploration. IEEE Transactions on Visualization and Computer Graphics, 17(12):1892–...
1901
-
[63]
J. Wang, L. Gou, H. Yang, and H.-W. Shen. Ganviz: A visual analytics approach to understand the adversarial game. IEEE Transactions on Visualization and Computer Graphics, 24(6):1905–1917, 2018
1905
-
[64]
J. Wang, S. Hazarika, C. Li, and H.-W. Shen. Visualization and visual analysis of ensemble data: A survey. IEEE Transactions on Visualization and Computer Graphics, 2018, Early Access
2018
-
[65]
J. Wang, X. Liu, H.-W. Shen, and G. Lin. Multi-resolution climate en- semble parameter analysis with nested parallel coordinates plots. IEEE Transactions on Visualization and Computer Graphics, 23(1):81–90, 2017
2017
-
[66]
H. G. Weller, G. Tabor, H. Jasak, and C. Fureby. A tensorial approach to computational continuum mechanics using object-oriented techniques. Computers in Physics, 12(6):620–631, 1998
1998
-
[67]
C. Xie, W. Xu, and K. Mueller. A visual analytics framework for the detection of anomalous call stack trees in high performance computing applications. IEEE Transactions on Visualization and Computer Graphics, 25(1):215–224, 2019
2019
-
[68]
J. Xie, L. Xu, and E. Chen. Image denoising and inpainting with deep neural networks. In Proceedings of Advances in Neural Information Processing Systems, pp. 341–349, 2012
2012
-
[69]
Zhang, I
H. Zhang, I. J. Goodfellow, D. N. Metaxas, and A. Odena. Self-attention generative adversarial networks. arXiv preprint arXiv:1805.08318, 2018
2018 arXiv
-
[70]
Zhang, W
K. Zhang, W. Zuo, Y . Chen, D. Meng, and L. Zhang. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017
2017
-
[71]
Y . Zhou, Z. Zhu, X. Bai, D. Lischinski, D. Cohen-Or, and H. Huang. Non- stationary texture synthesis by adversarial expansion. ACM Transactions on Graphics, 37(4):49:1–49:13, 2018
2018
-
[72]
J.-Y . Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of 2017 International Conference on Computer Vision, pp. 2242–2251, 2017. 11
2017
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.