I don't know how common it is, but don't worry, it's certainly not sinister! Especially on the first flight of the day for that crew / aircraft (later flights, the crew will of course stay on board for the short turnaround). Look at it this way, the crew have to get on board at some point and they certainly won't depart until they have completed the walkround inspection and all the required paperwork in the flight deck (which usually shouldn't take too long).
As a pilot friend of mine pointed out, most pilots have families and all are just as afraid of dying as any nervous passengers. The difference is, the crew know how safe it is, and how well trained they are, and how many backup systems the aircraft has, and what turbulence feels like, etc etc. Although I guess this won't necessarily make you any less nervous when you next fly!