I just can't believe that anyone flying with an Ipad on their knee isn't going to spend a bad amount of time heads down
It all depends on what app you are running.
If you are running a straight moving map app which shows the actual VFR chart, then no user interaction is required.
I think half the myths come from anti-GPS traditionalists, and half from people who saw somebody fumble around with the knobs trying to make the thing work. No way would I want to use a modern sophisticated handheld (e.g. a G695) without reading the manual and having a good play with it.