ALMA detection of CO rotational line emission in red supergiant stars of the massive young star cluster RSGC1 -- Determination of a new mass-loss rate prescription for red supergiants

[Abridged] Aim: We aim to derive a new mass-loss rate prescription for RSGs that is not afflicted with some uncertainties inherent in preceding studies. Methods: We have observed CO rotational line emission towards a sample of RSGs in the open cluster RSGC1 that all are of similar initial mass. The ALMA CO(2-1) line detections allow to retrieve the gas mass-loss rates (Mdot_CO). In contrast to mass-loss rates derived from the analysis of dust spectral features (Mdot_SED), the data allow a direct determination of the wind velocity and no uncertain dust-to-gas correction factor is needed. Results: Five RSGs in RSGC1 have been detected in CO(2-1). The retrieved Mdot_CO values are systematically lower than Mdot_SED. Although only five RSGs in RSGC1 have been detected, the data allow to propose a new mass-loss rate relation for M-type red supergiants that is dependent on luminosity and initial mass. The new mass-loss rate relation is based on the new Mdot_CO values for the RSGs in RSGC1 and on prior Mdot_SED values for RSGs in 4 clusters, including RSGC1. The new Mdot-prescription yields a good prediction for the mass-loss rate of some well-known Galactic RSGs that are observed in multiple CO rotational lines, including alpha Ori, mu Cep and VX Sgr. However, there are indications that a stronger, potentially eruptive, mass-loss process - different than captured by our new mass-loss rate prescription - is occurring during some fraction of the RSG lifetime. Implementing a lower mass-loss rate in evolution codes for massive stars has important consequences for the nature of their end-state. A reduction of the RSG mass-loss rate implies that quiescent RSG mass loss is not enough to strip a single star's hydrogen-rich envelope. Upon core-collapse such single stars would explode as RSG.