The only option for changing the prospect manager on the Prospect summary constituent tab of a constituent record is to edit and not replace a prospect manager which provides no history of the change.
Tracking changes to Prospects and Prospect managers is important historical information.