You're not the only one it's ambiguous to.
If you're VFR leaving a Class 'D' then you're subject to a formal clearance ("Cleared to leave CAS On track xyz not above ???? ft).
With this in mind, when you were asked for your cruising level a better response to you would have been "On leaving CAS report reaching 3000ft" or similar. That way the leaving clearance still stands, plus you're not being given a climb clearance (which you can't technically be given in Class 'G' anyway).
If there was going to be any ambiguity then you should have just been answered with "Roger" until you were outside CAS and then the climb should have been mentioned - not before.