Just removing the obvious file/folder or program.exe file is usually not enough. Most of those types of nasties embed themselves where you can't get to so you need a program that knows how to extract them with minimal disruption. That's why I also suggested MalwareBytes Anti-Malware. I know it detects and deletes on command PUP "infections". I've had them. One thing to be careful of though is that there can be a PUP show up in the analysis but it is part of a program or something that is legitimate.
With hindsight I'd say never go in and delete a folder or file as such in the hope of eliminating these things, always do a scan first. Otherwise it is akin to the proverbial using a sledgehammer to crack a walnut. It may do the job, but the collateral damage is vast.