The Bureau of Engraving and Printing like to be known as such and have since they were formed during the Civil War.
Don't forget that the Smithsonian has a number of small valuable objects (including the Hope diamond) and I assume where such items are in transit to other museums, armed guards would be carried. Many of the other agencies are mystifying - why would a chicken inspector/ schools inspector need to be armed at all, let alone when in transit. It would seem to be sensible that most of the above agencies should travel disarmed, and if their duty requires them to be armed in flight they should require special permits or even escort by other agencies. However in the current case it would appear that the agent followed proceedure in notifying the crew that he was armed, and his duties require that he be armed at all times (if the above list of agencies was to be severely cut, the Secret Service would probably be the last to be disarmed).
Note that the above list includes several sub-agencies, ie both the Secret Service and BEP come under the Department of the Treasury and in the majority of agencies listed, very few of the personnel would be required to be armed at all (I believe that regulations actually prohibit members of the Federal Prison Service travelling armed, even on active duty)