Height is money to spend
Robbo
You cannot have too much height should you have an engine failure (hope is that you never will). Your height cannot be a problem unless your too low. The circuit is not the purpose. The purpose is to get the aeroplane safely into a field. An engine failure should it ever happen will do so at any height.
The late Frank Morgan told me a story of an FIC student of his that was having a problem flying a PFL, yet alone teach it. Instead of commencing at a normal teaching height of 2-2500ft, he took the student instructor to 5000ft to give more time. Frank then closed the throttle and told the very nervous instructor to be to take his time and teach the complete process to him at a steady pace. The student promptly dived to 2,500ft before commencing the lesson which he screwed up in a rush yet again.
Always be pleased to have height for it gives you distance, time and more options. To have chosen a field under the wing is perfect. You will without doubt be able to land in it. Fly an extra orbit overhead the field if far too high. The 'high' or 'low key' points are to give you guidance which aren't always hit, in fact they usually are not. You may have to turn inside or widen out, that is what they are for - guidance.
You should review the taught technique with your instructor and then ask if you could be signed out to practice the PFL solo. Then experiment around different fields from different heights playing with the wind to understand its effect on you. Practising PFLs solo can be important as with a normal circuit. Practising solo will re-enforce your decision making which from your own description is your weak point.
Whatever, trim the aeroplane well and keep things simple. Don't be too hard on yourself. The major problem with the PFL is that the conditions will never be the same twice - accept that.