Yes, egress can be a problem, and a big factor in drownings. but if people make it out, or are pulled out, already having a life jacket on could make a huge difference. If you're injured, it's much more certain to pull a toggle and be floating, than to start to don the lifejacket and fail. Otherwise a passenger who cannot help themselves is much more likely to survive if a fellow passenger simply pulls the toggle on their life jacket, and they float upright on their own.
There will be passengers who never get it - those who unbuckle as soon as the seat belt light is turned off, or try to take their life's belongings as carry on. But education can make a difference for most passengers. A lot of what needs to be taught could be preflight on the internet. Many things are presented that way now, and it works well for most. Even my underwater egress course had an internet precourse!