Well yes it depends on whether you need to go back to some older version.
Often, in a business/professional context, you do something on a PC, and then you need to open the item again (say, a document, a PCB design, a schematic, a graphic design, etc) 5 or 10 years later. It may have got corrupted 3 years ago in a PC crash, and gradually the corrupted version has propagated onto all your backups.
That's why one has to have a backup strategy like I described. Then you go back and back until you get the uncorrupted version.
Same with accounting software, but for a diff reason. You may need to retrieve a particular year-end data set, in case of a Revenue enquiry/investigation/employee fraud/whatever. HMRC are b*stards these days; they can go back 6 years but if they find something 5 years back they can go back another 6 years and if you don't have the supporting data they can make an assessment and you have nothing to defend yourself with.
Obviously if everything you do is irrelevant the moment you do it, a USB hard drive etc will be just fine.