I was told this is the reason why Airbus doesn't recommend using APU bleed during takeoff (rather packs off instead), because it's not ensured crew wouldn't forget to turn it off in climb.
(Never understood why [APU...OFF] isn't part of the after takeoff / climb checklist)