The structure of the flight crew licensing law in the ANO looks like this:
Art 20 requires an aircraft to have a flight crew numbering at least the minimum required by its C of A.
Art 21(1) requires the members of the flight crew to be licensed, except in certain circumstances decribed in (2).
Art 22 gives the CAA the power to issue (and renew etc.) UK/JAA licences in accordance with Schedule 8 which set out the privileges.
None of it refers to who may operate the controls. If you were to take the interpretation that operating the controls would make you a member of the flight crew, then you could only do so with an appropriate licence. But Art 130 defines flight crew as:
‘Flight crew’ in relation to an aircraft means those members of the crew of the aircraft who respectively undertake to act as pilot, flight navigator, flight engineer and flight radiotelephony operator of the aircraft;
which lends this interpretation little support.
Art 41(1) may also be relevant, as it requires one pilot to remain "at the controls" throughout the flight. However 41(3) clearly implies that a PNF is considered to be "at the controls".