Even time travelers have to have internal causality primacy. While a time traveler skips around an outward appearance of causality as if time were a destination, internally there’s always got to be a before and after. That’s what previous means to a time traveler – not “before” in the context for everyone else, but before in terms of the time traveler. Maybe you run into issues of apparent paradox outwardly, but if someone can change their internal timeline you’re open to all the classic tropes of time travel nonsense (removing yourself from a timeline before you were born by actions you take in life, for instance.)

Really, time travelers, logically, shouldn’t be called travelers at all. It would probably be more correct to assert that they stay in one place and move the universe around them until it looks like so point in the past or waits statically and intransient until some point in a future opens up.

Consider :
What does ‘previous’ mean if you are capable of time travel?

The weird (non-Dalek) x-ray like flash of the Master’s internals makes me think the answer is something like “The Doctor has gone and fucked things up really, really, really, badly with what he did in Water of Mars.”