Legally speaking, yes.
But there's a few practical issues to consider. Are you physically going to swap seats or is one of the pilot comfortable to fly as PIC in the RHS? And what happens if the first P1 puts the aircraft in a situation (marginal weather, low on fuel, unusual attitude, you name it) that is unacceptable to the second P1, just before the switchover point?
Being PIC isn't about manipulating the controls. It's about being in command of the flight and bringing it to its safe conclusion.