MakeItHappenCaptain, I agree that Vx is normally the climb that you are looking to establish for obstacle clearance.
However, in the takeoff scenario, if you reference the POH as you say, a number of POH's will actually advise you to maintain TOSS until obstacles are cleared on take off for a performance/short field takeoff scenario. Makes sense really.
Take a bigger single like the 750XL for example. Zero flap Vx is 85 kts. All take offs require a 20 degree flap config. For a performance take off with 20 degree flap it is recommended to initially climb at 74 kts (this is the max weight TOSS and it refers you to the Performance section for TOSSs to use at lower weights - it can be as low as 59 KIAS at 4500lb) until clear of obstacles. You don't want to be retracting flap and accelerating to Vx 85 kts and plowing into trees until you have climbed above the obstacles in the initial climb phase.
A quick look at something smaller like a Warrior II POH expresses similar concepts for obstacle clearance on all take offs. Take the 25 degree (two notches) flap take off for example. Piper says: "40-52kts (depending on weight) back pressure to rotate to climb attitude. Accelerate to and maintain 44 to 57 kts (TOSS, Piper calls it '50 ft barrier' speed, depending on weight) until obstacle clearance is achieved and climb out at 79 kts (Vy). Retract flap slowly."
The reality is, most of the time we are not faced with a scenario where we are wanting to squeeze the last drop of performance out of our machine so we take off, clean up and cliimb at Vx if we feel like it for a best angle climb. But according to what your manufacturer recommends, if you want the best take off performance you will fly it by the book speeds and instructions. At the very least, certainly common sense should tell you that to change the take off configuration (eg raising flap) before safely clearing obstacles will only lead to tears (or some tense pucker moments if you are really lucky)!