Abstract
The investigation of materials that exhibit quantum phase transition provides valuable insights into fundamental problems in physics. We present neutron scattering under pressure in a triangular-lattice antiferromagnet that has a quantum disorder in the low-pressure phase and a noncollinear structure in the high-pressure phase. The neutron spectrum continuously evolves through critical pressure; a single mode in the disordered state becomes soft with the pressure and it splits into gapless and gapped modes in the ordered phase. Extended spin-wave theory reveals that the longitudinal and transverse fluctuations of spins are hybridized in the modes because of noncollinearity, and previously unidentified magnetic excitations are formed. We report a new hybridization of the phase and amplitude fluctuations of the order parameter near a quantum critical point in a spontaneously symmetry-broken state.
