Reply "Traffic copied, negative visual contact, looking" or words to that effect. If he's 2 miles away and moving at similar speed then you will probably have turned final before you meet him.
Continue the approach keeping a good lookout in all directions, remembering that there may be other traffic that even ATC doesn't know about. Turn final as normal, but bear in mind that this will put you at 90 degrees to the alleged traffic and thus make it harder to see. Turning away from the circuit may make matters worse if the other traffic has seen you and is assuming you will turn final, or if there is other traffic around. Don't forget the usual checks, and remember you're in landing config (ie nearer the stall & the ground) so flying the aircraft becomes even more crucial. If you feel your landing has been compromised, or just that your concentration isn't there, consider going around.
It is very disconcerting, a similar thing happened to me on my QXC. I was landing at Newcastle, having never been there before. I was given a right base join for the westerly runway, as I apporached from the North. As I was half way along base, I spotted traffic at the same height reciprocal heading. As it happened ATC knew about him, and he was on a left base, but I had missed that on the radio. Gave me an anxious few seconds though.