Quantum optomechanics describes utilizing optics to precisely manipulate and read out the motional degrees of freedom of a mechanical oscillator. If the mechanical oscillator is very weakly coupled to its environment, then the optomechanical interaction can be used to control the state of the mechanical oscillator in a quantum way. Applying these systems to advanced sensing techniques has inaugurated experiments on dark matter searches, gravitational wave detection, quantum gravitational phenomena tests, and sensing beyond the standard quantum limit. I am motivated by asking: ”What is the largest and most tangible object to reveal purely quantum phenomena?” In addition, I seek to use mechanics to explore quantum-enhanced applications.

In this thesis, I describe my work toward preparing quantum states of mechanical motion in a cavity optomechanical system. The system is a Fabry-Perot cavity that is filled with superfluid helium. A density wave of the helium serves as the mechanical resonator, whose effective mass is ~1 ng. The radiation pressure of the light is used as a gentle quantum “drumstick” to control the motion of the helium, while the helium, in exchange, imprints information about its motion on the emitted light. For such a large object, a myriad of different factors conspire to mask quantum effects. However, I can circumvent some of the obstacles by leveraging the material properties of superfluid helium and by using single-photon counting techniques.

In the experiment, I manipulated and characterized the state of the mechanics through optomechanical coupling and by performing photon counting measurements on the scattered light. I measured this mechanical resonator’s second/third/fourth-order coherence functions while it was in a thermal state with less than three phonons. In addition, I drove this mechanical resonator to a nearly coherent state. The state had around two phonons’ worth of fluctuations while its amplitude corresponded to 4X10^4 phonons. More striking quantum effects are related to states that are excluded by classical theories. Following the DLCZ protocol, I conditionally prepared non-classical photon-phonon entangled states. Their photon-phonon coherences violated a classical bound set by Cauchy-Schwarz inequality with a four-sigma significance.

I will also discuss our next steps using an even larger cavity to observe more macroscopic and more striking quantum features. Such a system shows prospects for dark matter detection, gravitational wave detection, and testing non-standard modified quantum theory.

Thesis advisor: Jack Harris (jack.harris@yale.edu)