Binary neutron star mergers in massive scalar-tensor theory: Quasi-equilibrium states and dynamical enhancement of the scalarization

We study quasi-equilibrium sequences of binary neutron stars in the framework of Damour-Esposito-Farese-type scalar-tensor theory of gravity with a massive scalar field, paying particular attention to the case where neutron stars are already spontaneously scalarized at distant orbits, i.e., in the high coupling constant case. Although scalar effects are largely quenched when the separation $a$ is $\gtrsim 3$--$6$ times of the Compton length-scale that is defined by the scalar mass, we show that the interaction between the scalar fields of the two neutron stars generates a scalar cloud surrounding the binary at the price of orbital energy when $a \lesssim 3$--$6$ times of the Compton length-scale. This enables us to constrain the scalar mass $m_\phi$ from gravitational-wave observations of binary neutron star mergers by inspecting the dephasing due to such phenomenon. In particular, the event GW170817 is suggestive of a constraint of $m_\phi \gtrsim 10^{-11}$ eV and the coupling strength should be mild if the neutron stars in this system were spontaneously scalarized.