ALT The latest breakthroughs in time-varying photonics are fueling novel, to the best of our knowledge, thermal emission phenomena, e.g., showing that the dynamic amplification of quantum vacuum fluctuations, induced by the time modulation of material properties, enables a mechanism to surpass the blackbody spectrum. So far, this issue has only been investigated under the assumption of non-dispersive time modulations. In this work, we identify the existence of a non-physical diverging behavior in the time-modulated emission spectra at high frequencies and prove that it is actually attributed to the simplistic assumption of a non-dispersive response of the time modulation associated with memory-less systems. Accordingly, we upgrade the theoretical formalism by introducing a dispersive response function, showing that it leads to a high-frequency cutoff, thereby eliminating the divergence and hence allowing for the proper computation of the emission spectra of time-modulated materials.