- If you don't think you'll be fit for the duration of the flight - don't go.
- If you can't carry fuel for an alternate - don't go.
- Logistics of what might happen to the next day's operations are for Ops to worry about, not you, though alerting them of the possibility is sound practice.
- Logisticts of the doctor's next duty is for the medico ops people to worry about.
- The cost of the flight is not your concern, from an operational (will I or won't I) perspective.
Short of the possiblity of fatigue, I didn't see anything in your original post that said, "don't go". Sure you may not have gotten in and may have ended up at an alternate. But you may have completed the retrieval quite successfully.
By all means inform Ops folk that the probability of success, because of x, y & z isn't good. Your responsibilty is to decide if you can get from A to B/C and then to A/Alternate safely. If so, go, if not, don't go.