Not a lawyer by any way shape or form. But been through the process myself but in a significantly more amicable way.
One thing my solicitor banged in to my head from day 1 is I do not have the automatic right to see my children. End of.
However my children do have the automatic right to see me if they so wished. No one could deny then that right (unless they were at risk from harm by coming to me, which in most cases they are not).
That frame of mind helped me a lot when dealing with my ex. Your situation is clearly more complex and I never ended up physically in court as everything was agreed mutually.
But perhaps you could explore the angle that the mother is denying your child THEIR right to see you.