haughtney1, your thinking is sound. Your error is a misunderstanding of priorities as applied by ATC.
All things being equal, arrival delay is distributed evenly amongst all aircraft. This delay figure becomes fairly stable about 40 minutes out from dest based on estimate for the feeder fix. "Popup" aircraft have the same priority as long haulers, and will take your position in the sequence if it means delay is spread evenly.
It's analogous to a deli queue in the supermarket. You don't get your ticket until you get to the counter - even though you phoned the manager from your Audi when you left home.