Great news! Keep at it. I remember when I first started my PPL and totally hated the radio, although enjoyed the flying! For me radio telephony was the hardest part and I remember thinking I would never be able to take it in and learn the dark art. Talking to tower, talking to radar, flying, checks, changing frequency, noting the pressure settings, transponder frequencies, flying, working out where we were, holding the correct altitude, holding correct heading, checking, aaarrggh! My head used to be battered after each lesson!
Its just one of those things and it will all come together. Using the radio is about confidence and partly knowing what the controller might say to you. Once you get the hang you will start enjoying it.