Captain on the left because, once aircraft got large enough to have two crew side-by-side, controls (eg throttles) tended to be installed down the centreline of the aircraft to be operated by either, and as most people are right-handed, and, certainly in the earlier days, the captain did most of the work (!) it was natural to sit to operate them with your stronger hand.
Once this was established, all the stuff of approaching the terminal in a right-handed curve (because the aircraft was being operated from the left seat and it made it easier to judge clearances), and then handling passengers through the left hand side, follows on, along with all the rules about which side to pass each other, which suit left hand seating.
There have been some past exceptions. Some of the early purchasers of the DC3 (eg American Airlines) had the aircraft custom manufactured with the main door on the right, because it suited their own way of doing ground arrangements. But these exceptions died away.
There are many other similar situations around. The tradition to drive on the roads either on the left or the right in various countries goes back to various different types of horse-drawn vehicle which became common in different countries, and had different ways in which the driver sat and whipped the horses, but which had them doing the stronger action with their right hand (some countries sat on the wagon, others rode on one of the horses), and the it was then convenient for the vehicles to pass either on the left or the right to suit. Once this was established, motor vehicles once invented and introduced to the roads just followed on the local tradition (60% of the world population drive on the right, 40% on the left)
In the UK, which pioneered escalators and the tradition of standing on the right and allowing overtakers to pass on the left (the opposite of UK roads) this once again came from allowing the less stable, who were more likely to just stand, to hold the handrail with their stronger right hand.