I have had a similar problem and it turned out to be one receiving ISP that would not accept the inbound from me and that blocked the whole process, some of the 'freebee' ISPs can be a bit difficult.
Couldn't send a 4MB attachment last night to a single address using 'reply' so typed in the email address manually, hey presto, message gone!
You may have just one crooked address holding things up.