I think a few people are making things up here.
Yes, you are allowed to fly directly at the crowd.
Watch any red arrows display, any F16, any Typhoon. They will all at some point head towards crowd centre.
If you were to see the size of the area that any jet display covers in plan view, you would realise that it is utterly impossible to not overfly houses and roads in any display over the UK.
Just as a point of interest, you might be interested to know that displaying over the water is more dangerous for the pilots. You might not care, but it can be very tricky for the pilot to judge depending on sea state.