Open for everything but the Cruise and Descent IMHO. Yes the speed gain is small but it is still a gain maybe 3-5kts depending on your Aircraft, if you're flying for an hour that could mean about 1minute off the flight, not much I know but could mean the difference between a 0.95 flight and a 1.00 flight in the Techlog, do that same flight a few hundred times, say over 400hours that would equate to around 20hrs at an operating cost of say $400/hr and that's $8000 you cost the company just by not operating a simple lever. If you have a fleet of 3-4 Aircraft or more it all starts to add up and that could be your wage or your mates wage for the year! It could mean the difference between the Boss being flush enough to spend some money on extra maintenance or fancy toys like a decent GPS, JPI Gauges, an Upgrade to IO-550s etc...etc...
Of course this very much depends on the Aircraft and Climate you're in, I've had some Singles that you just cannot close the Cowls as they'll start to get real hot, real fast, so don't just do it and expect something, do it and make sure everything is working as expected.
Incidentally, it could save your Exhaust one day, I had a guy who had the Exhaust become disconnected and the only thing that stopped it from completely dropping out the bottom of the Aircraft was the fact he had his Cowl Flaps closed!! the sound it created scared the crap out of him, haha.