Light in the dark forest -- I. An efficient optimal estimator for 3D Lyman-alpha forest power spectrum

The highly anisotropic nature of the Lyman-alpha (Ly$\alpha$) forest data introduces a complex survey window function that complicates the measurement of the three-dimensional power spectrum ($P_{\mathrm{3D}}$). In this paper, we present the first fully optimal estimator for $P_{\mathrm{3D}}$, which exactly deconvolves the survey window function and marginalizes contaminated modes that distort the power spectrum. Our approach adapts optimal estimator techniques developed for the 2D cosmic microwave background data to the 3D case. To achieve computational feasibility, we employ the conjugate gradient method and implement the P$^3$M formalism to handle large-scale and small-scale operations separately and efficiently. We validate our estimator using Monte Carlo mocks and Gaussian simulations, demonstrating its accuracy and computational efficiency. We confirm that mode marginalization eliminates distortions arising from quasar continuum errors and delivers robust power spectrum estimation, though it also inflates errors at large scales. This first implementation works in the flat sky case; we discuss the remaining steps needed to generalize to the curved sky. This formalism offers a foundation for the Ly$\alpha$ forest $P_{\mathrm{3D}}$ measurements and a new path toward cosmological constraints from the Ly$\alpha$ forest data.