a rhomboid wing like the F-23 is larger than the wing of the F-104
Somewhat in direct proportion to their gross weights. Remember the F-104 was called "missile with a man in it" - terrific zoom climb because of the hotrod engine, but hardly an efficient cruise vehicle. Although, the F-104A when upgraded to the late-60s J79-19 would indeed "supercruise" - the first operational fighter to do so.
In either case, F-104 or F-23, efficiency (in terms of fuel burn) is hardly #1 priority, given their missions.