Try:
"I Always Wanted to Fly: America's Cold War Airmen" by Col (ret) Wolfgang Samuel. It is a compilation of various guys' stories, drags sometimes in the "I grew on in Kansas and played baseball" memories, but is very good in the flying parts.
"Spyflights of the Cold War" by Paul Lashmer. A good, quick read.
and, a bit dated, but still good,
"Mayday" by Michael Beschloss. Much about the political situation of the time, but good details about the CIA program, the U-2 development,
the SA-2s, etc.