The Simons Observatory: Overview of data acquisition, control, monitoring, and computer infrastructure

The Simons Observatory (SO) is an upcoming polarized cosmic microwave background (CMB) survey experiment with three small-aperture telescopes and one large-aperture telescope that will observe from the Atacama Desert in Chile. In total, SO will field over 60,000 transition-edge sensor (TES) bolometers in six spectral bands centered between 27 and 280 GHz to achieve the sensitivity necessary to measure or constrain numerous cosmological parameters, including the tensor-to-scalar ratio, effective number of relativistic species, and sum of the neutrino masses. The SO scientific goals require coordination and control of the hardware distributed among the four telescopes on site. To meet this need, we have designed and built an open-sourced platform for distributed system management, called the Observatory Control System (ocs). This control system interfaces with all subsystems including the telescope control units, the microwave multiplexing readout electronics, and the cryogenic thermometry. We have also developed a system for live monitoring of housekeeping data and alerting, both of which are critical for remote observation. We take advantage of existing open source projects, such as crossbar for RPC and PubSub, twisted for asynchronous events, grafana for online remote monitoring, and docker for containerization. We provide an overview of the SO software and computer infrastructure, including the integration of SO-developed code with open source resources and lessons learned while testing at SO labs developing hardware systems as we prepare for deployment.