It is not that difficult. The detractors are usually the DAY/VFR only flyers with a map and stop watch.
I manage to amass a lot of hours each year doing just that. I claim expenses for the flight in the form of milage and landing fees just the same as I would claim for driving and parking.
VFR you will be very limited but even then I managed 80% of my trips. With an IR I manage 95% of my planned trips and at worst have to delay a few days before doing them.
You would need a properly equipped aircraft with GOOD avionics not the flying school hacks that nothing works in. You need to be very current and confident in your ability (not over confident!). The sort of level I refer to is getting in the aircraft and flying anywhere with same sense of ease as doing the trip by car. There are many of us on here who use our aicraft in this way. IO, CJBoy, Timothy etc.