Once the hele is in the water (rotors stopped) it is a (not very good) boat, so you are into naval architecture. I suggest the best way to look at it is in stages:
Static Stability
Dynamic Stability due to wave motion
Other wave effects (breaking waves)
Wind Loading
Although you may just choose to look at lateral stability, the complete picture would need to consider longitudinal too. However, the numbers would probably work so that the hele would capsize laterally before longitudinally. What would be sensible to consider might be affected by whether and how likely it would be that a drogue got deployed, and how it was deployed. This could be type and even operation specific, depending upon whether and where a drogue was deployed.
For static stability, you need to consider the position of the C of G and the centre of flotation. Statically, the hele will capsize once the C of G acts vertically outside the centre of flotation, giving a positive upsetting force/arm. The basics are against you because typically the C of G is high and the centre of flotation low, which is (one of the reasons) why heles are not good boats.
Dynamic stability is related to how the wave moves and changes the slope of the water under the hele "hull". It could act with or against static stability. The worst case is, of course, when the two upsetting forces/arms act in unison.
Wind loading is a rather different animal. It need not act in the same direction as forces generated due to the sea surface.
You also need to consider the effect of breaking waves as well, if you want to get a complete picture. They could provide a very significant extra upsetting force, as a transient addition to the other forces.
My suggestion would be to look at static stability first. If you can get a forecast of sea surface gradient (which in principle is possible), you would get a first order estimate. There are lots of variables though, and you need to understand how they all might change.
Although there are other ways of dealing with it, the static stability problem is capable of being calculated using some basic naval architecture and readily extractable physical data from the aircraft.