Aspect Ratio on Aircraft Performance
 

I understand the aspect ratio is the wing-span squared divided by the wing-area. Would I be correct in describing aspect ratio as being one of the factors that determine how much lift you can squeeze out of a given amount of wing-area?

Only for very small aspect ratios (like on the F-104) there is an appreciable effect on maximum lift, as the lift distribution becomes very close to the elliptic one. For typical aspect ratios used on transport aircraft this effect is neglectable and the lift distribution is somewhat closer to the wing planform, producing a little more lift on the outer wing.
The major effect is anyway just the slope of the lift over AOA curve, the lower your aspect ratio the higher you have to take the aircraft nose to get the same lift coefficient.

The main factor is that the induced drag gets significantly smaller with higher aspect ratios making the wing more effective.

Structural implications of aspect ratio
 

Aspect ratio also influences wing loads. High aspect ratio wings have longer span with shorter cord lengths than low aspect ratio wings. Longer span means higher bending moments. Shorter cord means less cross section of the wing. There are practical limits from a structural strength perspective that place upper bounds on aspect ratio. Glider designs are able to achieve very high aspect ratios by keeping weight very low. Aircraft designed to carry a significant amount of payload located in the fuselage must find a design solution with a smaller aspect ratio.

Indeed, gliders carry most of their load (for high speed high performance use) in the wing, not the fuselage. Of course the fuselage holds the crew (one or two pilots) and quite often an engine, however the water tanks for higher wing loading are inside the wings and can outweigh the crew weight considerably. Sadly the weight limit is set by regulatory and probably competition standards instead of technical possibilities, therefore it is much lower than possible nowadays, 850kg MTOW currently over here, possible are weights of well over a ton with wing spans of around 30 meters and aspect ratios around 40.

Volume

I would have always thought that for a given wing-loading a higher aspect ratio would always produce a higher peak L/D ratio than a lower aspect-ratio.

Induced drag = drag due to vortex formation on the tips?

Aspect Ratio
 

Well simply put

Low Aspect Ratio => Greater Induced drag, higher stalling AOA and Attitude, Lesser lift increment for unit increase in AOA, Control issues at Higher AOAs.

Higher Aspect Ratios => Low Stalling AOAs and attitudes, lesser Induced drag because you are operating at a lower AOA for the same lift coefficient, Peak Lift is more and so is the stall more pronounced.

A detailed understanding is available in Aerodynamics for Naval Aviators a must read!! ask any military aviator