Stik,
Yes, you can.
I went there last autumn without a transponder. I rang Jersey ATC beforehand to discuss it and found them to be very helpful. They asked me to come in via the St Germain VRP to minimise the time I'd be in their zone.
When I got there on the day I got a SVFR clearance straight away (not above 1,000' I seem to recall) from there to the south east corner of Jersey and then along the coast to the airport.
All very straightforward as it turned out. Great facilities at the Jersey Aero Club too, who do all the GA handling.