I looked into doing this last year, and the same conversation recently cropped up again - so I maybe flying that way myself before the summer.
IIRC you're not allowed to fly from Spain into Gib direct, all to do with this fued that makes the Spanish think it's theirs. You'd have to go via somewhere, else, ie Tangere's or somewhere in Morocco (airfield near Casablanca anyone?). Flying into Africa there maybe some club rule with hiring that may prevent that. I also remember Gib has special charts detailing the wind & turbulence which can be expected - They're available from the RAF. I'm 99.9% certain VFR is allowed.
I don't have any experience of A.C.d.M myself.