Popping over to France for lunch - pretty easy, though a little paperwork is required (flight plan, General Aviation Report, and usually 24hrs notice to customs where you want to land); plus flying over the sea can be tricky if there is no clear horizon so you might want to do some basic instrument flying first.
If you are renting from a club, it's quite likely they'll want an instructor to go with you the first time you do it; although this will put the cost up a bit, it's a good idea and will reassure you, plus he/she will make sure you have done the paperwork right the first time.
On the plus side, you can reclaim the duty paid on the fuel you 'exported' so that will help with the cost.
Tim