Yes, but if the autopilot is in, by the time you have got the QRH out, found the page etc the trim could be full deflection. I have tried this in the sim and would not like it to happen for real. You can control the aircraft as normal, but no autopilot and a hell of a push or pull is required. So much that my colleague had to help and I couldn't control the forces for more than about 10 mins without a break, arm wobble was a problem too.
If I saw it, I would switch them off regardless of whether it was a memory item (normally I am 100% SOP and it is a memory item in my company). If I were mistaken I could always switch them on again. Not to be confused with the 'stab trim' message though which is different.