Every day one would remove [the Ashes from] the altar at keri`ath ha-geber or about that time, either before or after. But on the Day of Atonement at midnight, and on the feasts at the first watch, and before the cockcrow approached the Temple Court was full of Israelites.
(Purchase a printed Mishnah)