Don't mix the two. Flight under Instrument Flight Rules (IFR) is not the same as flight by reference to instruments (IF). Some regions/Authorities or employers are more interested in one, some are more interested in the other. Most log books have an IF column + a user definable column. If your log book doesn't have a blank column then rename an unused column. If you're in a region (or think you might eventually need IFR logged time) then use the spare column for that and log IF in the IF column.
It's not unusual to log little or no IF on any given flight. I've had any number of *IFR* flights with zero IF.