Until recently I may have said the horse rider had a point. However last week I drove past two of them (on a horse) at 45 in a 60 zone. Both of the dears were waving at me to slow down at which point I thought f*** it if you can't control a horse when cars are doing the speed limit, don't ride it on the public highway.
And ultimately if you can't control the horse don't sit on it, don't blame someone else for your inability. The problem is, most horse riders don't seem willing to accept responsibility for their actions, weather it's a car driving by or a chinook, ultimately big loud things will come near you if you don't think you'll stay on the horse, get a new hobby.