Pith. sign in

REVIEW 2 major objections 3 minor 67 references

Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits

T0 review · 2 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multi-agent reinforcement learning beats gradient-based optimization on all eight photonic inverse-design tasks.

desk verdict A solid open-source benchmark and a genuine BPPO win over gradient on the simulated tasks, but the abstract overclaims and the gradient baseline is not given a fair, sample-matched run. read the letter →

arxiv 2506.18627 v1 pith:FCMJOTPI submitted 2025-06-23 cs.LG cs.AI

Yannik Mahlau , Maximilian Schier , Christoph Reinders , Frederik Schubert , Marco Bügling , Bodo Rosenhahn This is my paper
classification cs.LGcs.AI
keywords PhotonicIntegratedCircuitsMARLDiscreteOptimizationOpticalComputinginversedesignbanditFDTDsimulationproximalpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that inverse design of photonic integrated circuits can be reformulated as a bandit problem over thousands of binary voxel choices, and that multi-agent reinforcement learning solves this problem more effectively than the gradient-based optimization that has been the standard approach. Two algorithms, BAC (bandit actor-critic) and BPPO (bandit proximal policy optimization), share a single policy network across agents that each decide whether one voxel holds material or air, with each agent conditioned on a positional encoding of its location. On eight simulated tasks covering corner bends, fiber couplers, and scalar-vector multiplications in both 2D silicon and 3D polymer, the paper reports that the best of the two RL algorithms outperforms gradient-based optimization on every task using only a few thousand simulation samples. If true, this would make RL a practical alternative for discovering photonic components that are hard to reach by local search.

What carries the argument

The central object is a bandit environment in which the joint action is the full binary design vector $a \in \{0,1\}^N$ for $N$ voxels, the payoff $R(a)$ is an FDTD electromagnetic simulation of the resulting component, and performance is measured only by the best payoff seen within the simulation budget. BPPO (bandit proximal policy optimization) removes the critic and estimates each action's advantage as the payoff of the sampled joint action minus the old policy's expected payoff, using clipped importance ratios to keep updates stable; BAC (bandit actor-critic) trains a central critic on the joint action and backpropagates through a straight-through estimator of the discrete sample, masking 95% of agents' gradients and periodically reinitializing the policy to keep exploring.

What would settle it

Run gradient-based optimization on the same eight environments with the same number of simulation queries per task and per-task learning-rate tuning; if it matches or exceeds BPPO's best reported performance on any task, the paper's claim that multi-agent RL outperforms the previous state of the art would fail.

Watch

Extended reading notes

Core claim

The central claim is that the discrete, high-dimensional inverse-design problem for photonic integrated circuits—choose material or air at thousands of voxels—can be solved by a bandit-style multi-agent RL formulation, and that this formulation outperforms the previous state of the art, gradient-based optimization with a straight-through estimator. The authors argue that gradient-based methods get stuck in local optima and degrade in 3D polymer tasks where the mapping from continuous latent parameters to physically valid designs introduces gradient errors, while the stochastic policies of BAC and BPPO explore more broadly. In the reported experiments, BPPO beats the gradient baseline on all eight tasks and BAC on seven of eight; the best design in every environment comes from one of the two RL methods. The paper also reports that the RL-found designs are robust, retaining better performance than error-free gradient-descent designs even when ten percent of voxels are randomly flipped.

Load-bearing premise

The load-bearing premise is that the gradient-based baseline was given a fair comparison: it ran fewer optimization steps than the other methods because gradient computation takes time, and its learning rate was tuned on a single environment rather than per task.

Editorial extensions

If this is right

  • Photonic inverse design can be cast as a bandit problem with thousands of binary voxel decisions, so RL algorithms can optimize components using only about ten thousand simulation samples.
  • BPPO outperforms the gradient-based baseline on all eight tasks, while BAC wins on seven of eight; every task's best result comes from one of the two RL methods.
  • The RL-optimized designs are robust: randomly flipping ten percent of the voxels still leaves BPPO and BAC designs outperforming the error-free gradient-descent designs.
  • The open-source environment provides a benchmark for further sample-efficient RL research in photonics and beyond.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same shared-policy bandit decomposition could apply to any expensive black-box optimization over a large binary grid, such as metamaterial or metasurface design, not just photonic circuits.
  • Chaining the optimized scalar-vector multipliers should yield matrix multiplications and, with nonlinear materials, programmable optical neural networks; the paper notes the nonlinear extension as future work.
  • Re-running the gradient baseline with an equal number of simulation queries and per-task learning-rate tuning would isolate how much of the reported margin comes from exploration rather than from the baseline's limited budget.
  • Because the robustness test uses random voxel flips, testing against correlated fabrication errors would show whether the stochastic-policy advantage survives real manufacturing tolerances.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 3 minor

Summary. The paper formulates photonic integrated circuit (PIC) inverse design as a discrete, bandit-style optimization problem over binary voxel choices, and presents two multi-agent reinforcement learning algorithms, BAC and BPPO, which share a policy network across thousands of voxel agents conditioned on positional encodings. The authors introduce an FDTD-based environment covering corner, coupler, and scalar-vector multiplication components in both silicon (2D) and polymer (3D) forms, and compare BAC and BPPO against random search, DUCT, gradient-based optimization, evolutionary algorithms, and independent Q-learning. The central claim is that the proposed RL methods outperform previous state-of-the-art gradient-based optimization on all eight design tasks while using only a few thousand environment samples.

Significance. If the comparative claim is established, this work would represent a meaningful step toward sample-efficient, discrete, gradient-free inverse design for photonic components, especially for 3D polymer structures where gradient-based methods struggle. The paper has several strengths: hyperparameters for the RL methods are tuned on a separate, fast Game-of-Life environment rather than on the test tasks; rewards are computed with an external FDTD simulator (FDTDX), providing an independent evaluation signal; and the code is released openly, which aids reproducibility and could make the environment a useful benchmark. The algorithmic novelty of casting the design task as a many-agent bandit with a shared policy and positional encoding is plausible and worth investigating further.

major comments (2)
  1. [Section 1 and Table 1] The claim that 'both algorithms significantly outperform gradient-based optimization' is contradicted by the data in Table 1. On the Si-Coupler environment, BAC achieves 17.8 ± 0.8 while gradient descent reaches 41.5 ± 2.1, so BAC is substantially worse than the gradient baseline. The abstract also states that 'our algorithms' outperform previous state-of-the-art gradient-based optimization. Since the evidence only supports this statement for BPPO across all eight tasks, the claim needs to be narrowed or the discrepancy explained and addressed for BAC on Si-Coupler.
  2. [Figure 7 caption and Appendix A.4] The comparison against the gradient-based baseline is not sample-matched. The Figure 7 caption states that 'Gradient descent performed fewer steps than the other algorithms because gradient computation required time,' and the x-axis is stretched to align all curves. Table 1 reports final performance values, but if gradient descent was stopped before exhausting the same environment-sample budget given to the RL methods, the reported numbers are not its performance under the advertised budget. The paper does not report the number of gradient steps or FDTD solves per environment. Additionally, Appendix A.4 shows that the gradient baseline's only hyperparameter, the peak learning rate, was tuned on the silicon coupler environment alone, not per task. Because the headline claim is explicitly comparative, the validity of the comparison is load-bearing; the manuscript should either provide sample-matched runs for gradient descent, report exact step counts and show convergence behavior within those steps, or clearly qualify the 'outperform' claim to acknowledge the budget asymmetry and the single-task hyperparameter tuning.
minor comments (3)
  1. [Section 4] There is a typo: 'wether' should be 'whether' in the sentence about the designer's decision to place material or air.
  2. [Figure 7] The caption says the x-axis for gradient descent is stretched to allow comparison, but this makes direct visual comparison of progress rates difficult. Please indicate the actual number of steps used or plot both axes on the same scale with a secondary axis for gradient descent.
  3. [Section 5 and Table 1] The paper refers to a 'few thousand environment samples' and a budget T, but it does not explicitly state in the main text that all algorithms were given the same total budget of 10,000 environment queries. Please state the budget explicitly and confirm that the numbers in Table 1 are best reward within that budget for every algorithm.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the empirical RL results are measured against external FDTD rewards and independent baselines, and the paper's self-citations are contextual rather than load-bearing.

full rationale

The paper's central claim is empirical: BAC and BPPO outperform gradient-based optimization on eight FDTD-simulated design tasks. The reward function R(a) is computed by an external electromagnetic simulation, not by the algorithms' own objectives, so there is no sense in which the reported performances are defined in terms of the quantities they purport to demonstrate. The BPPO and BAC objectives optimize a policy or critic against collected payoff data; no predicted quantity is defined in terms of the quantity it claims to predict. Hyperparameters for the RL methods were tuned on a separate Game of Life environment, not on the target PIC tasks, which removes the main fitted-input concern. The gradient baseline's learning rate was swept on the silicon coupler environment and then reused elsewhere; that is a fairness or generalizability concern, not circularity. The paper's self-citations — Mahlau et al. (2025) for simulation speed, Schubert et al. (2025) for the gradient-based baseline and the FDTDX simulator — are contextual: the gradient baseline is an independently implemented optimization algorithm, and comparing against one's own previously published baseline is standard practice rather than a derivation that reduces to its own inputs. The Figure 7 caption explicitly notes that gradient descent performed fewer steps because gradient computation required time; this weakens the comparative claim empirically but does not make the claim equivalent to its inputs by construction. No uniqueness theorem, ansatz-by-citation, or renaming pattern is present. Therefore, no significant circularity is found.

Assumptions & free parameters 13 free parameters · 4 assumptions · 0 invented entities

The central claims rest on FDTD reward accuracy, fabrication-constraint modeling, and the expressiveness of per-voxel Bernoulli policies. No physical constants are fitted; the fitted quantities are algorithm hyperparameters tuned on a separate Game of Life environment, plus one learning rate sweep for the gradient baseline.

free parameters (13)
  • BPPO epsilon clip = 0.4978
    Tuned via Optuna on the Game of Life test environment; controls the PPO trust region and affects optimization behavior.
  • BPPO entropy coefficient = 0.005759
    Tuned on Game of Life; balances exploration.
  • BPPO gradient updates per rollout = 66
    Number of policy update steps between environment rollouts; tuned on Game of Life.
  • BPPO learning rate = 1e-4
    Tuned via Optuna on Game of Life.
  • BAC actor learning rate = 1e-3
    Learning rate for the BAC actor; tuned via Optuna on Game of Life.
  • BAC critic learning rate = 1e-4
    Learning rate for the BAC critic; tuned via Optuna on Game of Life.
  • BAC policy gradient mask fraction m = 0.95
    Fraction of agent gradients masked per policy step to preserve stochasticity; chosen by the authors.
  • BAC critic gradient steps U = 128
    Critic gradient steps per environment sample; chosen via Optuna.
  • BAC policy gradient steps G = 1024
    Policy gradient steps per environment sample; chosen via Optuna.
  • Gradient baseline peak learning rate = 0.01
    Selected by a sweep only on the silicon coupler environment, not per task; affects the fairness of the headline comparison.
  • DUCT exploration constant c = 0.2145
    Tuned via Optuna on Game of Life; balances exploration and exploitation in the DUCT baseline.
  • DUCT noise mean = 0.3242
    Gaussian noise added to the DUCT exploration term; tuned via Optuna on Game of Life.
  • Positional encoding bands b = 8
    Number of sine/cosine bands in the agent observation; chosen via Optuna on Game of Life.
assumptions (4)
  • domain assumption Maxwell's equations discretized with FDTD yield accurate predictions of light propagation for the proposed designs.
    Section 2 and Appendix C use FDTD with Yee grid, PML, and CFL-stable time steps; all rewards are simulation outputs and no fabricated measurements validate them.
  • domain assumption The fabrication constraints for silicon (2D uniform extrusion, 80 nm feature size) and polymer (500 nm voxels, no floating material, no enclosed cavities) are correctly captured by the binary grid and the mapping from actions to valid designs.
    Sections 4.1, 4.3 and Appendix C describe the constraint mapping; if the mapping diverges from real physical constraints, optimized designs may be unbuildable.
  • ad hoc to paper An independent Bernoulli policy per voxel with shared parameters and positional encoding is an expressive enough search distribution to represent high-performing designs.
    The entire MARL approach relies on this decomposition; the paper provides empirical evidence but no theoretical guarantee that this policy class contains good designs.
  • domain assumption The scalar reward (transmission fraction or 1 minus MSE) is a sufficient objective for component quality.
    Section 4.3 defines the rewards; effects like phase coherence, bandwidth, and fabrication tolerance are not part of the objective but may matter in practice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits." pith.science (2026). https://pith.science/paper/FCMJOTPI

@misc{pith2026250618627,
  author       = {Pith},
  title        = {Pith review of: Multi-Agent Reinforcement Learning for Inverse Design in Photonic Integrated Circuits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FCMJOTPI}},
  note         = {Machine review of arXiv:2506.18627}
}
read the original abstract

Inverse design of photonic integrated circuits (PICs) has traditionally relied on gradientbased optimization. However, this approach is prone to end up in local minima, which results in suboptimal design functionality. As interest in PICs increases due to their potential for addressing modern hardware demands through optical computing, more adaptive optimization algorithms are needed. We present a reinforcement learning (RL) environment as well as multi-agent RL algorithms for the design of PICs. By discretizing the design space into a grid, we formulate the design task as an optimization problem with thousands of binary variables. We consider multiple two- and three-dimensional design tasks that represent PIC components for an optical computing system. By decomposing the design space into thousands of individual agents, our algorithms are able to optimize designs with only a few thousand environment samples. They outperform previous state-of-the-art gradient-based optimization in both twoand three-dimensional design tasks. Our work may also serve as a benchmark for further exploration of sample-efficient RL for inverse design in photonics.

Figures

Figures reproduced from arXiv: 2506.18627 by the authors.

Figure 1
Figure 1. Design task of a linear operation on a photonic integrated circuit. In (a), 65% of the incom [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Optimized designs for the three different environments using either 2D silicon (purple) [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Comparison of BPPO, BAC and gradient descent regarding design variance and robustness [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Performance of BAC and BPPO with and without a structural prior through the positional [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]
Figure 5
Figure 5. Figure 5: Example designs for the game of life environment. In (a), a random design and in (b) a [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Influence of the learning rate hyperparameter on the performance of gradient-based opti [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Learning curves for algorithms tested in our work in the different environments. Gradient [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]
Figure 8
Figure 8. Figure 8: Simulation Scenes for the different environments presented in our work. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

67 extracted references · 54 canonical work pages

  1. [1]

    Optuna: A next-generation hyperparameter optimization framework

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp.\ 2623--2631, 2019

    2019

  2. [2]

    Sgd generalizes better than gd (and regularization doesn’t help)

    Idan Amir, Tomer Koren, and Roi Livni. Sgd generalizes better than gd (and regularization doesn’t help). In Conference on Learning Theory, pp.\ 63--92. PMLR, 2021

    work page 2021

  3. [3]

    Inverse design of nanophotonic devices with structural integrity

    Yannick Augenstein and Carsten Rockstuhl. Inverse design of nanophotonic devices with structural integrity. ACS Photonics, 7 0 (8): 0 2190--2196, 2020. doi:10.1021/acsphotonics.0c00699

    work page doi:10.1021/acsphotonics.0c00699 2020

  4. [4]

    Universal design of waveguide bends in silicon-on-insulator photonics platform

    Meisam Bahadori, Mahdi Nikdast, Qixiang Cheng, and Keren Bergman. Universal design of waveguide bends in silicon-on-insulator photonics platform. J. Lightwave Technol., 37 0 (13): 0 3044--3054, Jul 2019

    work page 2019

  5. [5]

    Estimating or propagating gradients through stochastic neurons for conditional computation

    Yoshua Bengio, Nicholas L \'e onard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013

    arXiv 2013

  6. [6]

    Crossq: Batch normalization in deep reinforcement learning for greater sample efficiency and simplicity

    Aditya Bhatt, Daniel Palenicek, Boris Belousov, Max Argus, Artemij Amiranashvili, Thomas Brox, and Jan Peters. Crossq: Batch normalization in deep reinforcement learning for greater sample efficiency and simplicity. In International Conference on Learning Representations, 2024

    work page 2024

  7. [7]

    Bogdanov, Sergey Makarov, and Yuri Kivshar

    Andrey A. Bogdanov, Sergey Makarov, and Yuri Kivshar. New frontiers in nonlinear nanophotonics. Nanophotonics, 13 0 (18): 0 3175--3179, 2024. doi:doi:10.1515/nanoph-2024-0396

    work page doi:10.1515/nanoph-2024-0396 2024

  8. [8]

    JAX : Composable transformations of Python + NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne , and Qiao Zhang. JAX : Composable transformations of Python + NumPy programs, 2018

    work page 2018

Show all 67 references
  1. [9]

    A universal approach to nanophotonic inverse design through reinforcement learning

    Marco Butz, Alexander Leifhelm, Marlon Becker, Benjamin Risse, and Carsten Schuck. A universal approach to nanophotonic inverse design through reinforcement learning. In CLEO 2023, pp.\ STh4G.3. Optica Publishing Group, 2023. doi:10.1364/CLEO_SI.2023.STh4G.3

    2023 doi

  2. [10]

    Yang, S.B

    Ming Cai, Hyunwoo Park, Jackie Yang, Youseok Suh, Jun Chen, Yandong Gao, Lunwei Chang, John Zhu, S C Song, Jihong Choi, Gary Chen, Bo Yu, Xiao-Yong Wang, Vincent Huang, Gudoor Reddy, Nagaraj Kelageri, David Kidd, Paul Penzes, Wayne Chung, S.H. Yang, S.B. Lee, B.Z. Tien, Giri N...

    2019

  3. [11]

    Randomized ensembled double q-learning: Learning fast without a model

    Xinyue Chen, Che Wang, Zijian Zhou, and Keith Ross. Randomized ensembled double q-learning: Learning fast without a model. In 9th International Conference on Learning Representations, ICLR 2021, 2021

    2021

  4. [12]

    Soft actor-critic for discrete action settings

    Petros Christodoulou. Soft actor-critic for discrete action settings. arXiv preprint arXiv:1910.07207, 2019

    1910 arXiv

  5. [13]

    Friedrichs, and Hans Lewy

    Richard Courant, K. Friedrichs, and Hans Lewy. \"U ber die partiellen differenzengleichungen der mathematischen physik. Mathematische Annalen, 100: 0 32--74, 1928

    1928

  6. [14]

    Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

    Christian Schroeder De Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip HS Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the starcraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020

    2011 arXiv

  7. [15]

    Trends in ai inference energy consumption: Beyond the performance-vs-parameter laws of deep learning

    Radosvet Desislavov, Fernando Martínez-Plumed, and José Hernández-Orallo. Trends in ai inference energy consumption: Beyond the performance-vs-parameter laws of deep learning. Sustainable Computing: Informatics and Systems, 38: 0 100857, 2023. ISSN 2210-5379

    2023

  8. [16]

    Inverse-designed diamond photonics

    Constantin Dory, Dries Vercruysse, Ki Youl Yang, Neil V Sapra, Alison E Rugar, Shuo Sun, Daniil M Lukin, Alexander Y Piggott, Jingyuan L Zhang, Marina Radulaski, et al. Inverse-designed diamond photonics. Nature communications, 10 0 (1): 0 3309, 2019

    2019

  9. [17]

    Incorporating Nesterov Momentum into Adam

    Timothy Dozat. Incorporating Nesterov Momentum into Adam . In Proceedings of the 4th International Conference on Learning Representations, pp.\ 1--4, 2016

    2016

  10. [18]

    Tidy3d: hardware-accelerated electromagnetic solver for fast simulations at scale

    Flexcompute . Tidy3d: hardware-accelerated electromagnetic solver for fast simulations at scale. https://www.flexcompute.com/download-whitepaper/, 2022

    2022

  11. [19]

    Counterfactual multi-agent policy gradients

    Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    2018

  12. [20]

    Pygad: An intuitive genetic algorithm python library

    Ahmed Fawzy Gad. Pygad: An intuitive genetic algorithm python library. Multimedia tools and applications, 83 0 (20): 0 58029--58042, 2024

    2024

  13. [21]

    Mathematical games

    Martin Gardner. Mathematical games. Scientific american, 222 0 (6): 0 132--140, 1970

    1970

  14. [22]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. PMLR, 2018

    2018

  15. [23]

    The next generation of deep learning hardware: Analog computing

    Wilfried Haensch, Tayfun Gokmen, and Ruchir Puri. The next generation of deep learning hardware: Analog computing. Proceedings of the IEEE, 107 0 (1): 0 108--122, 2019. doi:10.1109/JPROC.2018.2871057

    2019

  16. [24]

    Metal-assisted chemical etching of silicon and nanotechnology applications

    Hee Han, Zhipeng Huang, and Woo Lee. Metal-assisted chemical etching of silicon and nanotechnology applications. Nano Today, 9 0 (3): 0 271--304, 2014. ISSN 1748-0132

    2014

  17. [25]

    Rance, Gustavo F

    Qin Hu, Graham A. Rance, Gustavo F. Trindade, David Pervan, Long Jiang, Aleksandra Foerster, Lyudmila Turyanska, Christopher Tuck, Derek J. Irvine, Richard Hague, and Ricky D. Wildman. The influence of printing parameters on multi-material two-photon polymerisation based micro...

    2022

  18. [26]

    Forward-mode differentiation of maxwell’s equations

    Tyler W Hughes, Ian AD Williamson, Momchil Minkov, and Shanhui Fan. Forward-mode differentiation of maxwell’s equations. ACS Photonics, 6 0 (11): 0 3010--3016, 2019

    2019

  19. [27]

    Focused ion beam machining of silicon

    N.P Hung, Y.Q Fu, and M.Y Ali. Focused ion beam machining of silicon. Journal of Materials Processing Technology, 127 0 (2): 0 256--260, 2002. ISSN 0924-0136

    2002

  20. [28]

    A reinforcement learning method for optical thin-film design

    Anqing Jiang and Osamu Yoshie. A reinforcement learning method for optical thin-film design. IEICE Transactions on Electronics, 105 0 (2): 0 95--101, 2022

    2022

  21. [29]

    Otf gym: A set of reinforcement learning environment of layered optical thin film inverse design

    Anqing Jiang, Liangyao Chen, and Osamu Yoshie. Otf gym: A set of reinforcement learning environment of layered optical thin film inverse design. In CLEO: Science and Innovations, pp.\ SM1Q--7. Optica Publishing Group, 2021

    2021

  22. [30]

    Evolutionary Algorithms, pp.\ 49--71

    Yaochu Jin. Evolutionary Algorithms, pp.\ 49--71. Physica-Verlag HD, Heidelberg, 2003. ISBN 978-3-7908-1771-3

    2003

  23. [31]

    Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media

    Kane Yee . Numerical solution of initial boundary value problems involving maxwell's equations in isotropic media. IEEE Transactions on Antennas and Propagation, 14 0 (3): 0 302--307, May 1966

    1966

  24. [32]

    Optical computing: Status and perspectives

    Nikolay L Kazanskiy, Muhammad A Butt, and Svetlana N Khonina. Optical computing: Status and perspectives. Nanomaterials, 12 0 (13): 0 2171, 2022

    2022

  25. [33]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015

    2015

  26. [34]

    Deep reinforcement learning empowers automated inverse design and optimization of photonic crystals for nanoscale laser cavities

    Renjie Li, Ceyao Zhang, Wentao Xie, Yuanhao Gong, Feilong Ding, Hui Dai, Zihan Chen, Feng Yin, and Zhaoyu Zhang. Deep reinforcement learning empowers automated inverse design and optimization of photonic crystals for nanoscale laser cavities. Nanophotonics, 12 0 (2): 0 319--33...

    2023 doi

  27. [35]

    SGDR: stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings , 2017

    2017

  28. [36]

    Merging automatic differentiation and the adjoint method for photonic inverse design

    Alexander Luce, Rasoul Alaee, Fabian Knorr, and Florian Marquardt. Merging automatic differentiation and the adjoint method for photonic inverse design. Machine Learning: Science and Technology, 5 0 (2): 0 025076, 2024

    2024

  29. [37]

    Mastering zero-shot interactions in cooperative and competitive simultaneous games

    Yannik Mahlau, Frederik Schubert, and Bodo Rosenhahn. Mastering zero-shot interactions in cooperative and competitive simultaneous games. In Proceedings of the 41st International Conference on Machine Learning (ICML), 07 2024

    2024

  30. [38]

    A flexible framework for large-scale fdtd simulations: open-source inverse design for 3d nanostructures

    Yannik Mahlau, Frederik Schubert, Konrad Bethmann, Reinhard Caspary, Antonio Cal \`a Lesina, Marco Munderloh, J \"o rn Ostermann, and Bodo Rosenhahn. A flexible framework for large-scale fdtd simulations: open-source inverse design for 3d nanostructures. In Photonic and Phonon...

    2025

  31. [39]

    Multifunctional 2.5d metastructures enabled by adjoint optimization

    Mahdad Mansouree, Hyounghan Kwon, Ehsan Arbabi, Andrew McClung, Andrei Faraon, and Amir Arbabi. Multifunctional 2.5d metastructures enabled by adjoint optimization. Optica, 7 0 (1): 0 77--84, Jan 2020. doi:10.1364/OPTICA.374787

    2020 doi

  32. [40]

    Limits on fundamental limits to computation

    Igor L Markov. Limits on fundamental limits to computation. Nature, 512 0 (7513): 0 147--154, 2014

    2014

  33. [41]

    James Clerk Maxwell. Viii. a dynamical theory of the electromagnetic field. Philosophical Transactions of the Royal Society of London, 155: 0 459--512, 1865

  34. [42]

    Peter L. McMahon. The physics of optical computing. Nature Reviews Physics, 5 0 (12): 0 717–734, October 2023

    2023

  35. [43]

    ma-n 400 and ma-n 1400 - negative tone photoresists

    micro resist technology GmbH . ma-n 400 and ma-n 1400 - negative tone photoresists. https://www.microresist.com, 2025

    2025

  36. [44]

    Piggott, Weiliang Jin, Jelena Vuckovi \'c , and Alejandro W

    Sean Molesky, Zin Lin, Alexander Y. Piggott, Weiliang Jin, Jelena Vuckovi \'c , and Alejandro W. Rodriguez. Inverse design in nanophotonics. Nature Photonics, 12 0 (11): 0 659--670, November 2018. Publisher Copyright: Springer Nature Limited 2018

    2018

  37. [45]

    The primacy bias in deep reinforcement learning

    Evgenii Nikishin, Max Schwarzer, Pierluca D’Oro, Pierre-Luc Bacon, and Aaron Courville. The primacy bias in deep reinforcement learning. In International conference on machine learning, pp.\ 16828--16847. PMLR, 2022

    2022

  38. [46]

    Two-photon polymerization: Fundamentals, materials, and chemical modification strategies

    Seán O'Halloran, Abhay Pandit, Andreas Heise, and Andrew Kellett. Two-photon polymerization: Fundamentals, materials, and chemical modification strategies. Advanced Science, 10 0 (7): 0 2204072, 2023

    2023

  39. [47]

    Jung, Juho Park, Dongjin Seo, Yongha Kim, Chanhyung Park, Chan Y

    Chaejin Park, Sanmun Kim, Anthony W. Jung, Juho Park, Dongjin Seo, Yongha Kim, Chanhyung Park, Chan Y. Park, and Min Seok Jang. Sample-efficient inverse design of freeform nanophotonic devices with physics-informed reinforcement learning. Nanophotonics, 13 0 (8): 0 1483--1492,...

    2024 doi

  40. [48]

    Alan Roden and Stephen D

    J. Alan Roden and Stephen D. Gedney. Convolution pml (cpml): An efficient fdtd implementation of the cfs–pml for arbitrary media. Microwave and Optical Technology Letters, 27 0 (5): 0 334--339, 2000

    2000

  41. [49]

    Jaxmarl: Multi-agent rl environments in jax

    Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Gardar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, et al. Jaxmarl: Multi-agent rl environments in jax. arXiv preprint arXiv:2311.10090, 2023

    2023 arXiv

  42. [50]

    Learned fourier bases for deep set feature extractors in automotive reinforcement learning

    Maximilian Schier, Christoph Reinders, and Bodo Rosenhahn. Learned fourier bases for deep set feature extractors in automotive reinforcement learning. In 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), pp.\ 931--938. IEEE, 2023

    2023

  43. [51]

    Explainable reinforcement learning via dynamic mixture policies

    Maximilian Schier, Frederik Schubert, and Bodo Rosenhahn. Explainable reinforcement learning via dynamic mixture policies. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pp.\ To be published, 2025

    2025

  44. [52]

    Quantized inverse design for photonic integrated circuits

    Frederik Schubert, Yannik Mahlau, Konrad Bethmann, Fabian Hartmann, Reinhard Caspary, Marco Munderloh, J \"o rn Ostermann, and Bodo Rosenhahn. Quantized inverse design for photonic integrated circuits. ACS Omega, 2025

    2025

  45. [53]

    Proximal policy optimization algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  46. [54]

    Park, and Min Seok Jang

    Dongjin Seo, Daniel Wontae Nam, Juho Park, Chan Y. Park, and Min Seok Jang. Structural optimization of a one-dimensional freeform metagrating deflector via deep reinforcement learning. ACS Photonics, 9 0 (2): 0 452--458, 2022. doi:10.1021/acsphotonics.1c00839

    2022 doi

  47. [55]

    Deep transfer reinforcement learning in nanophotonics: A multi-objective inverse design approach

    Abdullah Bin Shams, Abdur Rahman Akib, and Stewart Aitchison. Deep transfer reinforcement learning in nanophotonics: A multi-objective inverse design approach. 2024 Conference on Lasers and Electro-Optics (CLEO), pp.\ 1--2, 2024

    2024

  48. [56]

    Snyder and J

    A .W. Snyder and J. Love. Optical Waveguide Theory. Springer, 1 edition, 1983. ISBN 0412099500

    1983

  49. [57]

    Allen Taflove and Susan C. Hagness. Computational electrodynamics: the finite-difference time-domain method. Artech House, Norwood, 3rd edition, 2005

    2005

  50. [58]

    Mandy J. W. Tak, Marc Lanctot, and Mark H. M. Winands. Monte carlo tree search variants for simultaneous move games. In 2014 IEEE Conference on Computational Intelligence and Games, pp.\ 1--8, 2014. doi:10.1109/CIG.2014.6932889

    2014

  51. [59]

    Multi-agent reinforcement learning: Independent vs

    Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In Proceedings of the tenth international conference on machine learning, pp.\ 330--337, 1993

    1993

  52. [60]

    Time reversal differentiation of fdtd for photonic inverse design

    Rui Jie Tang, Soon Wei Daniel Lim, Marcus Ossiander, Xinghui Yin, and Federico Capasso. Time reversal differentiation of fdtd for photonic inverse design. ACS Photonics, 2023

    2023

  53. [61]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  54. [62]

    Munchausen reinforcement learning

    Nino Vieillard, Olivier Pietquin, and Matthieu Geist. Munchausen reinforcement learning. Advances in Neural Information Processing Systems, 33: 0 4235--4246, 2020

    2020

  55. [63]

    Reinforcement learning for photonic component design

    Donald Witt, Jeff Young, and Lukas Chrostowski. Reinforcement learning for photonic component design. APL Photonics, 8 0 (10), 2023

    2023

  56. [64]

    Overcoming the spectral bias of neural value approximation

    Ge Yang, Anurag Ajay, and Pulkit Agrawal. Overcoming the spectral bias of neural value approximation. In International Conference on Learning Representations, 2022

    2022

  57. [65]

    The surprising effectiveness of ppo in cooperative multi-agent games

    Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, U...

    2022

  58. [66]

    Inverse design of high-q topological corner states nanocavities based on deep reinforcement learning

    Zhendi Yu and Ran Hao. Inverse design of high-q topological corner states nanocavities based on deep reinforcement learning. Optics Communications, 577: 0 131402, 2025. ISSN 0030-4018. doi:https://doi.org/10.1016/j.optcom.2024.131402

    2025

  59. [67]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.