It depends on how you are doing it. The DATE in Excel is actually a number - years since 1/1/1900 (which is "1" in this context) with lots of decimals to take care of months, weeks, days, hours, etc. The "date" number format then converts this to the date we recognise.
If you subtract one date from another, the answer will (unless you reformat it) appear as a third date - and look odd.
You should be able to add those "differences", and divide them by the ROWS() count, and so on. My logbook certainly does that with no bother.
If all else fails, put the spreadsheet on a website and PM me the URL, and I'll take a look at it.
None of this works for dates before 1/1/1900. Excel will display them correctly, but won't recognise them as a date and won't process them in the same way. It can be done, but you need to extract day, month, and year and do your own calculations from those.