However, I'm not relieved of my responsbility to provide an alerting service, and in the en-route IFR environment, failing to positively identify the aircraft reduces the quality of this service.
Yes it is nice to watch the safe progress on the radar. However, there is a difference between identifying the aircraft in an SSR only environment and watching the progress of an aircraft through your sector on the radar. i.e. you may not have identified the aircraft but if the SSR response disappears for a period of time you might be inclined to call the aircraft and check what's up.