I think you should be OK, but as others have said, it may well be necessary to get somebody of very high standing as a referee. I have a few gaps in my history (I changed jobs a few times and had several periods of gardening leave, travelling etc.) but a friend of mine was a “gap referee” and he was able to tell them what I was doing. He holds a responsible position and the fact that I have known him for 15 years helped.
Maybe in your case, if you have multiple referees who are willing to cover the period, it may give them a warmer feeling that you were where you said you were. Somebody who is themselves subject to security clearance would be an ideal referee.