Hi Heston...
Er, that's not how they are used in microlights and other permit aircraft. Usually the "handheld" is mounted on the panel and fed from the aircraft's power supply.
Well that may solve the battery going flat bit, but if you want to hear the loudspeaker maybe you need the headset adaptor, and/or the remote microphone (HM 112 for the Icom.)
Then you will most likely want to use an external aerial, and ditch the rubber duck. If your idea of mounting the radio is to use the belt clip into the plane's map pocket, then I hope you don't encounter too much turbulence...
We used to carry one on board our glider, I seem to remember the PTT was wired onto the stick, but gliders are considerably quieter than powered aircraft, and not usually contacting ATC facilities.