The NTSB has said safety rules restrict such balloons to be aloft only when visibility is at least one mile, and pilots are supposed to stay clear of clouds.
That's interesting, since basic VFR is 1000/3 ... is the lesser visibility restriction due to the slow horizontal speed of a balloon? (Anybody know?)