Many results in the quantum metrology literature use the Cramér–Rao bound and the Fisher information to compare different quantum estimation strategies. However, there are several assumptions that go into the construction of these tools, and these limitations are sometimes not taken into account. While a strategy that utilizes this method can considerably simplify the problem and is valid asymptotically, to have a rigorous and fair comparison we need to adopt a more general approach. In this work we use a methodology based on Bayesian inference to understand what happens when the Cramér–Rao bound is not valid. In particular we quantify the impact of these restrictions on the overall performance of a wide range of schemes including those commonly employed for the estimation of optical phases. We calculate the number of observations and the minimum prior knowledge that are needed such that the Cramér–Rao bound is a valid approximation. Since these requirements are state-dependent, the usual conclusions that can be drawn from the standard methods do not always hold when the analysis is more carefully performed. These results have important implications for the analysis of theory and experiments in quantum metrology.