Conservative cosmology: combining data with allowance for unknown systematics

When combining data sets to perform parameter inference, the results will be unreliable if there are unknown systematics in data or models. Here we introduce a flexible methodology, BACCUS: BAyesian Conservative Constraints and Unknown Systematics, which deals in a conservative way with the problem of data combination, for any degree of tension between experiments. We introduce hyperparameters that describe a bias in each model parameter for each class of experiments. A conservative posterior for the model parameters is then obtained by marginalization both over these unknown shifts and over the width of their prior. We contrast this approach with an existing hyperparameter method in which each individual likelihood is scaled, comparing the performance of each approach and their combination in application to some idealized models. Using only these rescaling hyperparameters is not a suitable approach for the current observational situation, in which internal null tests of the errors are passed, and yet different experiments prefer models that are in poor agreement. The possible existence of large shift systematics cannot be constrained with a small number of data sets, leading to extended tails on the conservative posterior distributions. We illustrate our method with the case of the $H_0$ tension between results from the cosmic distance ladder and physical measurements that rely on the standard cosmological model.