See
GID33 for details.
In summary, to keep the licence valid by experience, you need a minimum of 12 hours and one flight with an instructor in the last 12 months of the 24 month validity period. Alternatively, you can do a skills check with an examiner instead.
To carry passengers, you need 3 take-offs and landings in the last 90 days.
These are legal minimum requirements. Most pilots will need to do more flying than the bare minimum to maintain their flying at an acceptable standard -- I'd suggest that if you can't do 20 hours a year you should consider carefully whether it's worth it.