Not being there I can't comment on whether the flight was illegal or not. Those who were there and want to pursue things further will need to provide the authorities with credible evidence which will allow a prosecution.
But flight below 500' is not necessarily illegal as some often assume. You would need to do some trigonometry involving the aircrafts height and horizontal distance from the 'object' to ensure his Minimum Separation Distance (hypotenuse) was less than 500'. How far was this aircraft laterally from any object, structure, or person ??
climbing fairly sharply, in a zoom climb, not a small height adjustment, to about 100 feet below me! Then proceeded to cross my path left to right, so apparently wasn't paying any attention to lookout, as I was not far away in an aircraft with fairly visible colour scheme, skylined.
This is an assumption on your part of course. Perhaps he saw you and was content to pass 100' below you keeping you in sight. Hence 'see and avoid' was complied with. Under VFR there is no minimum separation requirement. Some pilots are happy with very small separation (gliders in thermals for example), whilst others seem to get twitchy if 'airliner type' separations are not adhered to.
As always with these emotive threads, there is more grey than black and white.