I'm sure Barclays have assessed the security risks involved.
To start with most phones are locked, and perhaps the app enforces this as a policy if a PIN/security code/fingerprint lock is not already enabled.
The app is also tied to your phone, so your account cannot be accessed from another phone unless you go through the same "huge" approval procedure for that device - which, as you discovered, requires several authentication factors.
I expect that further authentication will be required if you want to move money to an account that is not yours.
Finally, there may be additional covert security measures in place that I'm not going to speculate on in a public forum.
I would expect that Barclays will act honourably in the event of fraud resulting from genuine, normal use of the app.