This is a fair point - also important to remember that the Starship and Super Heavy vehicles they used for this test flight actually weren't all that refined. TPS full of gaps, etc. The newer TPS solutions on the newest Starships look infinitely better.
Models and simulations can only go so far before they need to be validated. But there will certainly be a lot more pressure on SpaceX now to nail the second OFT. If we don't see a perfect performance from the booster, there'll be many more heads turned. Certainly since Raptor should be performing a lot better than it is, though that could certainly be down to concrete flying everywhere and Stage 0 having some hiccups.
Fool me one, shame on you...