Abstract
The van der Waals compound, MnBi2Te4, is the first intrinsic magnetic topological insulator, providing a materials platform for exploring exotic quantum phenomena such as the axion insulator state and the quantum anomalous Hall effect. However, intrinsic structural imperfections lead to bulk conductivity, and the roles of magnetic defects are still unknown. With higher concentrations of the same types of magnetic defects, the isostructural compound MnSb2Te4 is a better model system for a systematic investigation of the connections among magnetism, topology, and lattice defects. In this work, the impact of antisite defects on the magnetism and electronic structure is studied in MnSb2Te4. Mn-Sb site mixing leads to complex magnetic structures and tunes the interlayer magnetic coupling between antiferromagnetic and ferromagnetic. The detailed nonstoichiometry and site mixing of MnSb2Te4 crystals depend on the growth parameters, which can lead to ≈40% of Mn sites occupied by Sb and ≈15% of Sb sites by Mn in as-grown crystals. Single-crystal neutron diffraction and electron microscopy studies show nearly random distribution of the antisite defects. Band structure calculations suggest that the Mn-Sb site mixing favors a ferromagnetic interlayer coupling, consistent with experimental observation, but is detrimental to the band inversion required for a nontrivial topology. Our results suggest a long-range magnetic order of Mn ions sitting on Bi sites in MnBi2Te4. The effects of site mixing should be considered in all layered heterostructures that consist of alternating magnetic and topological layers, including the entire family of MnTe(Bi2Te3)n, its Sb analogs, and their solid solution.
