He starts his loop back in the desert, but that's because that's when he first thinks that his quest will succed (sic), but things changed before, everything changed really, it's just The Tower throws him back to that point.