MikeX: It's more complex than a simple port scan.
The GRC test (and it's a good site) probes the device at the end of the public IP address that your browser reports. This is likely to be your router, not your computer. Routers generally have incoming firewalls blocking this kind of probe, and also translate the IP address(es) of your computer(s) from one range to another (I'm oversimplifying here), so there is no direct incoming path from the internet to your computer. This protects your LAN & computers from certain vulnerabilities, & is a valuable 1st line of defence.
However this does not protect against risks arising from faults in software on your computer. Your browser could make an outgoing request (to a website for example), & if the site uses certain techniques that exploit certain errors on certain systems, you can get a situation where your browser can be made to execute other people's code. If you are running an OS that the malicious code doesn't expect (e.g. OS X when the attack assumes Windows), then you'll merely get a browser crash or error (again, I'm oversimplifying). Attacks use lots of social engineering techniques (such as asking you to install a 'special codec' so you can see the latest pictures from Sheep Worrier's Monthly), or whatever. Or they try to exploit known vulnerabilities. As soon as MS (or Mozilla with Firefox) for example issue a patch, then people will try to reverse-engineer the appropriate attack by analysing the fix, and so go for those people who are tardy applying the patch. It's a Darwinian race. I don't intend this to be a Windows-bashing session; it just reads that way, BTW.
Microsoft in the past developed horrendous technology known as ActiveX, which in the innocent prehistory days of the internet, allowed anyone to write executable extensions to IE. This is the one of the main reasons why (to this day) people are suspicious of IE. Thankfully MS have shown some sense in recent times.
If the holes in the cheese line up, then you might be in a position to allow this 3rd-party code to run natively on your computer, outside the browser. This is particularly serious if you are running in an account that has full admin rights, because then that code does too. If you are running a restricted account, then (barring privilege escalations, which I'm not going to bore you with) that code can only do restricted things.
In my case, I generally use Firefox under OS X or Linux in a non-admin account, with Firefox not installed system-wide: it (1) only exists in my limited account, and (2) can't touch anything in the system that's outside my own account (this is enforced by the OS). So I'm confident that even if I get an infection, the worst thing it can do is damage to that single user account. I do the important stuff such as banking running OS X in a fairly locked-down configuration.
I do use Windows at times, but I'm rather more careful what I do with it, since it's inherently less secure in real situations than the alternatives, and also more actively targeted. Whatever you think of its usability, it's a poor OS in that people generally end up running in admin accounts routinely (careless software developers often write apps that won't install in non-admin accounts for one thing, although again it's a bad OS that allows them to do this).