I've not raced, but I spent a while helping out with RAeC air racing.
Aeroplanes are handicapped by maximum level flight speed, the course defined (x number of laps), and take-offs staggered. The idea is that if every aeroplane flies the course perfectly, then they'll all cross the finish line at the same moment - which inevitably won't happen as pilots misjudge wind, don't preserve energy as best they can in the turn, lose time because they didn't quite navigate right to the turning point, gain or lose on climbs, attract a few too many dead flies, etc.
All good fun, and open to any aeroplane which does more than 100mph in level flight - which is even a fair number of microlights and motorgliders these days so pretty all-inclusive (well, apart from the sheer number of hours that keen racers end up flying).
I have no problem seeing the appeal of it it and will probably do the course and enter a few races myself one of these years.
It's reasonable from a spectator viewpoint, as it's all relatively low level and the final dash for the finish line can be quite entertaining. The environment is a little political (what form of competition isn't), but generally pretty social and good natured.
G