I've been there once, but to other places in Africa a few more times.
I like Africa and I usually go out and do things and don't feel threatened.
However, in Lagos, I did stay in the hotel for the reasons above. I didn't take my computer or camera and used the in room safe for other stuff. It all worked ok, and I had a fairly pleasant trip. I have heard some horror stories, but our hotel has been refurbished and so most of the horror stories were taken away in that. The water stinks of chlorine in our hotel, which is better than the brown stuff you used to get! Be careful what you eat and drink and don't expect fast service or particulary polite people, but at the same time, don't worry overly about it and be streetwise. There are worse places you could go. The biggest problem is boredom if you are there for a few days.