As I mentioned earlier, procedures (RT and flight paths) and lookout (or lack of it) are the key issues here. Places can have very busy VFR traffic safely operating but it requires discipline and awareness.
It only needs one person to cuff it or go off-piste and you have a problem.