You can drive there yes but come early as the parking is sometimes cramped. Best thing to do is get the bus or train to southampton then taxi or there is a bus that goes near dibden manor if you have time and want to save money. I took the taxi last year and was only £10 from Southampton, if uber is there it maybe cheaper now. Or even better, fly to southampton if possible?