I believe the limits are more on components, as was stated the airframe can be built to withstand much more than a human occupant. At least one of the main nav components has been tested to 8 g's.
There are far more civilian/agency unarmed uses for UAV's than armed, and I hope that we can get past this issue....
CAA UK is much at the front lines on UAV regs with the recent CAP 722.
(nothing useful will come out from the FAA until 2020, if then)