I used to use a 1/8th thick Steel strip, approx 4" x 3/4", bent to about 30 degrees with a hole in the end to place on the rivet head - the right size (e.g 1/8" for 1/8", etc - or metric for the new boys) it was rounded and smooth around the edges so as not to scratch.
Lay it on the top of round/uni heads and drill away - centers almost automatically without filing the flats - tends to wear after a bit but you got used to it. Can't remember many mishaps - but then, we were removing skins all the time!
Ex-431MU AEF trick