Rudestuff
From my experience it does not matter if you haven't logged start and finish times. As long as you have the other details for each flight your log book is acceptable to the CAA
If they want to they can find start and finish times by contacting the airfield but they are very unlikely to do that