I've just been down to Sitia LGST in Crete (again...) and Libya is just down the road from there.
I planned a visit in 2007 but gave up after seeing the hassle. All that "local sponsorship" crap. Having been down to N Africa a few times (not GA) I am sure one could just land and pull out a bundle of 1000 $ and all would be "smoothed" in the traditional African way... Same for Luxor - another trip I planned.
No doubt it can be done, and is probably easier now than in 2010 (not least due to the people who have done it, being accessible for their hints and tips) but you have to ask yourself whether it is worth the hassle.
Crete is practically N Africa and is far easier. Sitia is a GA heaven; a port of entry and has avgas.
Not wishing to deny others the adventure
There are also 3 or 4 agents who will arrange overflight/landing permissions and visas. I think most people end up using these agents. I was quoted £70/person for the permits.