My Fitbit doesn't stay in contact with the app all day, I turn off the bluetooth link on the Fitbit and on the phone except when synching/downloading the Fitbit stats every few days. The Fitbit does all of the calculation of calories itself, based on the measured activity levels, pulse rates, paces and some personal body stats (height, age and weight), the value is passed to the app only when synching takes place. Tracking my walking with a Garmin GPS shows that the Fitbit is not far out in calculating the distance walked from paces etc despite it not being fed a value for my pace length.
A quick check of the phone shows that forcing the Fitbit app to stop only stopped one of the three bits of the Fitbit suite running. It seems that neither of these remaining parts is responsible for the excess battery use seen at the weekend. The phone is still ticking over using about 1% of the battery per hour when I'm not fiddling with it.