In my opinion, if I was in the RHS and the Captain announced that he was disconnecting the FD to have a go at flying by hand my only response would be 'let me know when it's my go!'.
Clearly the majority of training has to be focused on SOP's as I would imagine 98% of sectors wouldn't offer any opportunity that would force seat of the pants flying however ignoring the 2% is foolhardy in my opinion. I personally wouldn't be happy with myself if I had any doubt in my abilities to bring the plane back to ground (in one piece obviously) if the computers failed. As a PAX you just have to hope that the people in the pointy end feel the same way. Such is life though, just one big risk!