Self-induced oscillations in an optomechanical system

We have explored the nonlinear dynamics of an optomechanical system consisting of an illuminated Fabry-Perot cavity, one of whose end-mirrors is attached to a vibrating cantilever. Such a system can experience negative light-induced damping and enter a regime of self-induced oscillations. We present a systematic experimental and theoretical study of the ensuing attractor diagram describing the nonlinear dynamics, in an experimental setup where the oscillation amplitude becomes large, and the mirror motion is influenced by several optical modes. A theory has been developed that yields detailed quantitative agreement with experimental results. This includes the observation of a regime where two mechanical modes of the cantilever are excited simultaneously.