It's very unlikely you'd skip the GFPT. If you did, a dual flight would be required after every three hours solo.
The main disadvantage as already mentioned is you cannot carry friends/family when flying cross-country.
Not doing a PPL doesn't make it a harder CPL flight test, the ATO still needs to tick his boxes no matter your history.