Pretty standard for the sort of route and conditions you were flying. You should expect the winds over the Grampians to be signifcantly stronger than forecast - apart from the mountaining forecast (sometimes!). Essentially the airflow goes over the top and is 'squeezed; and so accelerated. It also forms waves and they males constant altitude difficult.
If you want to fly at constant altitude you will need to change power and direction frequently. Each wave will give you lift and then sink, if you are flying obliquely to the wind both can last for a significant period.
I rarely ever fly at constant height in this area because of it - go with the height gains and accept the losses. Occasionally the losses can be pretty big - decide on whether the loss will take you down towards the rotor on the leeside of the high ground - if it looks like it will - change course and power early - under all circumstances do not get into the rotor area - with 20 knots very bad things can hapen there. Otherwise just enjoy it.
With a little practice you can optimise the life/sink ratio and get some free performance. I seem to remember some good basic information on the Transport Canada website on basic mountain flying.