With regard to pilots doing this work (we must not forget many investigators are engineers or scientists):
Generally, you need a good professional flying background, some management or training experience, perhaps a smattering of engineering skills. Then the skills necessary for the job at hand - good communication skills, broad awareness, (a strong stomach) etc.
Some UK investigators do fly, albeit only a few days a month. This is vital to keep their experience current and keep them 'in touch'.
This flying will be with airlines or other operators, along similar lines to the arrangements that CAA staff sometimes enjoy to fly for airlines to keep current.
No harm in it - quite the reverse.