Don't forget, you can fail for not going when the conditions are acceptable. So what are the limits? Well it's a VFR flight, you are in a single and probably inside Class C or D airspace. So if the weather shows that it's within the legal and aircraft limits (say for the flight plus one hour), you go. End of.
PM