Right now I have a debug system that allows the player to change it's observed character from the one you're controlling to another player. This moves the camera to their perspective.
Now, if I want to make this a proper feature for players to use I probably want to make that system be server authority by having the server tell the player what character to observe. Seems straight forward.
But that doesn't fix the problem entirely. What's stopping a player from saying “ignore what the server says, spectate this enemy player" through memory manipulation? I guess if a malicious player is in deep enough to know how to force this setting in memory, they can probably do a bunch of nasty stuff like move the camera around the map anyway so maybe worrying about this is moot anyway?