Keep going
Your feelings sound similar to mine when I first started flying (and when I first stepped into a racing car) - the immediate task looks daunting if not impossible. I carried on because I knew it was what I wanted. When I started my instrument rating the thought of being in cloud or between layers of cloud left me cold and during the training I wondered how I'd manage to crack it. I wanted to fly in the mountains so went on to get a mountain rating and during that training I wondered if I could really safely land on the altisurfaces that we went to - but eventually it comes.
If you want it then carry on because you could end up regretting not following your wishes later in life.