MS Office 2003, I believe is not supported much any more. Definitely not sold.
I have two options on my PC, since I'm using a dual boot Windows / Ubuntu Linux PC.
I have purchased a version of Office 07 called the Home and Student edition. It allows installs on three PCs, has only Word, Excel and PowerPoint with OneNote included. The cost is substantially lesser than the full version which most of us don't need for home use (MS Access being one such program). Can't use it on an Office (-ial) machine , though.
I have been using Libre Office / Open Office successfully on my Linux install. Before you stop reading ahead
, there is a Windows installer too.
Apache OpenOffice - Download tested and stable builds and
Home » LibreOffice 
It (that is, either one of these office suites) works very well with all MS Office files.
There may be some formatting changes in files where there is a lot of formatting when opening the document files in alternative systems (like we used to end up doing in the Military !! - paras and sub paras , with sub-sub paras). But if you do all your work mostly with simple files, Libre office or Open Office is great. What I do is I export the file directly as a pdf, so the formatting remains 'locked in'.
As the name suggests it is legally free and Open source. To save on disk space, you should install only the components that you need. Writer (for Word files), Calc (for Excel Files) and Impress (for Power Point files). If you need English UK, then there is a separate file to be additionally downloaded in addition to the main install. One big advantage of the Open office or Libre office suite is that the presentation / drop down menu list is the same as the Office 2003. No confusing ribbon interface here like the '07 and '10 have. Definitely worth a try, as most Linux users and many Windows users will no doubt vouch for. There are many large organisations, Banks even that use OpenOffice to save on costs.
Thunderbird is a mail client that works very well and can be easily configured.
Thunderbird - Software made to make email easier You can therefore do away with your need for the Outlook mail client.
Regarding reinstalling from the CD...I've heard of folks having similar problems with a pre-installed Windows versions too. There is no proper OS installation CD, and the restore CD supplied with the box doesn't work well enough for a clean format and install - only for a restore. I therefore buy the hardware assembled or pre-built, without an OS, that is, DOS only. I then purchase a Licensed CD installer for the OS / Office. This makes it much easier to configure the Computer, especially since I partition the disk and install at home. Windows of course, is unavoiadable for stuff like mobile phone backups.
Hope you do manage to sort out the issue in a way that suits you, anyway.
Regards