Following the arrow of time, one second at a time