That's why. If they can't be sure, they're not going to antagonise him into doing something rash by taking away his toys. The pax were comfortable, they had food and drink. No one was getting hurt. Softly, softly.
But if the Police aren't sure why are they waiting there 4 hours?? Either get everyone off or get out of the way onto a remote bay, then take as long as you like.
Honestly if there is a credible bomb threat then it really should be a precautionary disembarkation.
If it's none of the above then really you have a hijack situation as the suspect is calling the shots. If that's the case then all bets are off and you do whatever it takes, including sitting there for hours.