We study the genealogical structure of a population with stochastically fluctuating size. If such fluctuations, after suitable rescaling, can be approximated by a nice continuous-time process, we prove weak convergence in the Skorokhod topology of the scaled ancestral process to a stochastic time change of Kingman's coalescent, the time change being given by an additive functional of the limiting backward size process.