Perfectly OK, as long as you have full view of the instruments and access to the controls.
It can be a bit disorienting first time or two, so worth doing it under supervision until you feel comfortable.
I know a few instructors who have got so used to it that they choose to fly in the RH seat all the time, even when not instructing.
Edit to add - very occasionally the POH stipulates the captain's seating position, or solo seating position.