Rivets, there are very few reasons to remove the stone(s) but leave the GB behind -- mostly when it is really inflamed and too complicated to remove. Best to just have it out and move along.
IMHO a solitary small stone is a bit of a dilemma. Big solitary stones aren't often an acute problem. It's the small ones that can migrate out and clog things up rather suddenly. On the other hand, that big one was once small, so who knows?
I think if my livelihood depended on my GB behaving, in this scenario, I'd have it out. Radgirl's points are well taken.