As for the federal criminal charges, they're likely just a necessary means to hold him so that they can do a complete investigation. While the statements in the affidavit are all true, everyone realizes (at least to this point) that the guy probably lacked criminal intent. However, without actual charges being filed, he could legally walk out of the hospital and disappear and no one could stop him.
In UK people can be detained under the Mental Health Act - is there nothing similar in the USA?