SS,
I'm not an expert but I cannot think of any rule of the air which is dependent on radio, at least in the UK.
In your case, because you had radio, you at least knew what the other party, who was obviously breaking the rules, was planning to do. Even then, you understandably found it difficult to decide what action to take in the short time you had available. At least you were on the ground.
I spent the first four years of my flying life without radio and on several occasions have been faced with others similarly putting me in a difficult situation without any opportunity to clarify what we were doing over the radio.
For example, on my QXC in a DH82, I was happily proceeding downwind at Shoreham, heading towards the sea. I was overtaken by a light twin who was obviously happy to fly a circuit which took him well out to sea. No way was I going to follow him out there at circuit height. So, what should I have done?
P1