Spiral versus modulated collinear phases in the quantum ANNNH model

Motivated by the discovery of spiral and modulated collinear phases in several magnetic materials, we investigate the magnetic properties of Heisenberg spin S=1/2 antiferromagnets in 2 and 3 dimensions, with frustration arising from 2nd-neighbor couplings in one axial direction (the ANNNH model). Our results clearly demonstrate the presence of an incommensurate spiral phase at T=0 in 2 dimensions, extending to finite temperatures in 3 dimensions. The crossover between N\'eel and spiral order occurs at a value of the frustration parameter considerably above the classical value 0.25, a sign of substantial quantum fluctuations. We also investigate a possible modulated collinear phase with a wavelength of 4 lattice spacings, and find that it has substantially higher energy and hence is not realized in the model.