A few points worth noting:
A target which is moving across your field of vision, no matter how close he is, is not on a genuine collision course. A target on a collision course will appear stationary to you. Very hard to spot, and this greatly undermines the see-and-avoid principle which is one of the treasured cornerstones of GA.
You can't assume who the target is talking to. In Class G, he doesn't have to talk to anybody, and frankly talking to anybody who doesn't provide a radar
information service, or a radar
advisory service, is a total and complete waste of time for any traffic avoidance purposes. "There are seven aircraft known in your area".... yeah right that's really useful (not). The reality of UK airspace management (basically, no regular radar services outside controlled airspace) is that many experienced pilots fly in Class G with just a listening watch, on London Info or perhaps some nearby unit (in case one has to make a mayday call).
I sometimes see other planes on a course that will take us fairly close. Once I have established that we are going to miss by a good few hundred yards (which
looks quite close) or more, I am happy to just sit there flying a dead straight track on the autopilot (with a finger on the disconnect button) while watching the other plane. Often I see the other plane doing nothing and then in the last few seconds waking up and doing an avoidance maneuver which seems quite pointless. No doubt, he thinks I was head down and fiddling with my GPS, just like we read about in all the magazines
But the vast majority of targets are never seen. If that really worries you, mandatory transponders and TCAS are one way to have a go at addressing that... the other way is to get an IR and fly in controlled airspace as much as possible (and hope that nobody infringes it).