You're right about the Victorian, "us and them" attitude, held by many Brits and some of our American bretheren, but it can sometimes be too easy to stereotype.
After the initial "She's an Ashawo, he'll be bankrupt in a few months" stage passed and we subsequently got married I didn't have a problem company wise with my Nigerian wife, now of many years.
Funnily enough the enduring prejudice often seemed to come from older generation Nigerians. They assumed we met in a bar, she must have been there for "business", (both wrong), and she got evil looks and sometimes insults from that quarter.