Being from a different background and being quite far in age from my fellow classmates, I failed to properly integrate
I was an academic before, physicist
I feel you may have already identified an important part of the problem - aviation is by nature, a social activity and my experience of pilots who do not integrate with classmates (regardless of age disparity) and later, other members of their crew (team) is not positive. You mention that you soloed at 9 hours (which is good even by integrated standards), so you are clearly able, but that you are also undertaking theoretical knowledge on alternate days (which, although the original intent of integrated training at the inception of the JAA, is unusual in the EASA world unless you are preparing for retakes? (at least in my experience)) and that your location is remote; the reference to 'traffic patterns' and 'uncontrolled airfields' suggests that you are not in the UK (US perhaps?). May we know where you are as your training environment sounds as if it is having a highly negative impact on your progress.
Your comments concerning your loss of confidence are something that in the first instance, you should be discussing with your instructor - unless you feel that your instructor is not approachable or disinterested in which case the Chief Instructor or HoT should be consulted.
Good luck with your decision - I would think long and hard before transferring to modular though as if you seek airline employment, an uninterrupted training history with no indicators of possible psychological obstacles would be highly desirable following implementation of the Pilots' Psychological Evaluation that will soon be required prior to entering service.