Hmmm....
Lots of assumptions required but here's my (ATCO) interpretation.
The airfield to which you were flying enjoys the protection of a Class D Control Zone (CTR). To enter the zone you NEED a clearance, if only along the lines of "Enter the CTR via XYZ, not above 1500ft, VFR".
You
must not enter the zone without a positive clearance. Assumptions don't count. If unsure you must ask if you really do have a clearance to enter.
CTR entry points are often at the boundaries of the Zone, and it's good practice to call at least 5 mins (or 15nm) before you get there.
You also said:
I understand that the tower doesn't get too excited when someone is entering the zone unexpected while the tower is busy with an ILS approach
Not true. Ever. They will.
Okay. So what's to be done? I suggest you treat this as a learning experience. Go and visit the tower. How? Give them a ring and explain. They'll be happy to see you and more than happy to explain why entering any CTR without a clearance is a Very Bad Idea. (They won't bite you either!)
Legal actions? Not usually, as long as you
have learned from the experience, just a letter from CAA SRG "offering appropriate advice".
Hope this helps
Rgds BEX