To put numbers on this:
Typical emergency O2 tank pressure is 1850 psi. From
rho = P / (R*T)
follows the density of the pressurized O2 is about 168 kg/m^3, less than a fifth that of water. Add about 30kg of steel for an assumed 40cm diameter by 1m length oxygen cylinder and the combined density of the O2 tank is about 450kg/m^3, or about half the density of water (and even less than that for salt water).
Thus, an O2 tank floating in 30 degree water will float, regardless of whether it is full or empty of pressurized O2.
Cheers
Doc