Multiple D3-instantons and mock modular forms I

We study D3-instanton corrections to the hypermultiplet moduli space in type IIB string theory compactified on a Calabi-Yau threefold. In a previous work, consistency of D3-instantons with S-duality was established at first order in the instanton expansion, using the modular properties of the M5-brane elliptic genus. We extend this analysis to the two-instanton level, where wall-crossing phenomena start playing a role. We focus on the contact potential, an analogue of the Kahler potential which must transform as a modular form under S-duality. We show that it can be expressed in terms of a suitable modification of the partition function of D4-D2-D0 BPS black holes, constructed out of the generating function of MSW invariants (the latter coincide with Donaldson-Thomas invariants in a particular chamber). Modular invariance of the contact potential then requires that, in case where the D3-brane wraps a reducible divisor, the generating function of MSW invariants must transform as a vector-valued mock modular form, with a specific modular completion built from the MSW invariants of the constituents. Physically, this gives a powerful constraint on the degeneracies of BPS black holes. Mathematically, our result gives a universal prediction for the modular properties of Donaldson-Thomas invariants of pure two-dimensional sheaves.