I'm also finding Kayak is thrashing anything Matrix can throw up, often at a cheaper price and more flexible routing - and I've been playing with Matrix for a few days now.
Well I've tried Kayak a few times and found it generally lousy. I guess it depends what routes you are searching for.
Kayak actually has ITA as its backend, so you should be able to match what you see on Kayak to ITA, if not be able to beat it because you won't be relying on Kayak's generic flight search implementation.