It depends on your job. If your presence is desirable rather than mandatory, then it could work as long as you have the strength of character to say no when conditions dictate. However if you HAVE to be there, then you WILL kill yourself. For low houred pilots, and probably most others as well, pressonitus is a big killer, if not the biggest killer.
Whereas I now fly for a living, with the appropriate reduction in minima (and even then I never assume that I am going to get there), in my previous career I occasionally used a light aircraft to travel to meetings. However I did my planning the day before with particular attention to the weather. If it was no good I went by car/train.
Please be very carefull.