Some constructive googling and I came up with David Horne on 023 92324553 of
Fort Widley Riding School, and the charge is £25 for the day. Six miles to the centre of Portsmouth. It's about as close as you can get before you hit the congested area of Cosham which is on/S of the A27/A3