I know this is like my finger wagging re opening overhead lockers during an evac but, if your seatbelt is not fastened, you are not risking injury only to yourself but also to the small child upon whom you fall, probably at more than 1g.
Ex RAF and civ pilot, I ALWAYS have my seatbelt fastened - except, of course, when out of my seat.
Remember fasten the belt OVER blanket etc then CC won't have to wake you up to check
Once again apologies if I seem to be preaching.