It would be interesting to know if it can get into a looping state after which it just repeats previous moves forever. I guess that would be an “end” of sorts.
> Because there are a finite number of states, it will always loop.
No it wouldn't. The fact that it will pass through every state infinite times doesn't mean it will do it in the same order each time. Each digit of pi has only 10 possible states, but it never loops.