Ping simply measures the round-trip time of a small packet of data between your computer and the target you are pinging. It has absolutely nothing at all to do with your download speed.
Think of it as a measure of the distance between you and the target, rather than a measure of how fat the pipe is.
Do your ping results indicate any packet loss? Try pinging your own router to see if the problem lies inside your own property. It might be that someone has set up a nearby wifi router on the same channel and it is causing interference (I have assumed that you are using wifi). Is anyone else using your network? Are you hosting a botnet?
Once you know the problem is not at home, you might also want to run a traceroute (or tracert on Windows). This will show you all the hops that exist between you and the target, and the ping for each of them. It will be obvious if one of them is very slow - though there is not a whole lot you can do about it.