Well....youre right !! the Capt is the only person who can grant a F/Deck visit....but most of them rely on the opinions of f/a's as to the suitability of the person for such a visit (ie their appearance, I.D., whether they appear to be intoxicated etc), they rely heavily on f/a's perception of the individual in order to make their decision regarding visits. Maybe this flight attendant felt you werent suitable for some reason !! Maybe she/ he was too busy and just plain forgot ( we are human, and yes do make mistakes occasionally - albeit very rarely).