Yes they do. (If you kept them informed about your change of address of course.)
They didn't in my case, and large numbers of others. I recall that at the first 5-year point, which was about 2005, nobody got a reminder. Maybe they started doing them on subsequent expiry periods but they didn't send one out to the earliest JAA license holders like myself.
My renewal was done with a FE in 2009 which was some 8 years after I got the license. I may have sent a payment to the CAA then but have no recollection of it. I will email them...