Broadband quantum memories hold great promise as multiplexing elements in future photonic quantum information protocols. Alkali-vapor Raman memories combine high-bandwidth storage, on-demand readout, and operation at room temperature without collisional fluorescence noise. However, previous implementations have required large control pulse energies and have suffered from four-wave-mixing noise. Here, we present a Raman memory where the storage interaction is enhanced by a low-finesse birefringent cavity tuned into simultaneous resonance with the signal and control fields, dramatically reducing the energy required to drive the memory. By engineering antiresonance for the anti-Stokes field, we also suppress the four-wave-mixing noise and report the lowest unconditional noise floor yet achieved in a Raman-type warm vapor memory, (15±2)×10−3photons per pulse, with a total efficiency of (9.5±0.5)%.