PIC is ultimately responsible. You can't be ultimately responsible sitting in the jumpseat
How can s/he be ultimately responsible if in the bunk? 447 has shown accidents can happen at any time, as you say "flying can unravel in seconds". The Qantas A330 Learmonth upset could conceivably have occurred with the PIC taking rest if the flight demanded additional crew.