On AF if boarding starts during refuelling there is a warning to NOT fasten seat belts. I guess that this is so you can run if it catches fire.
Otherwise my advice is to always fasten them as soon as you sit down so you don't forget. If you can't unfasten a belt you should either practice or give up flying.