That is one of those questions with no yes or no answer - it depends - and the dependance ultimately comes down to experience.
At one extreme well spaced CBs that are clearly visibile and avoidable are not going to present a problem.
Clearly, at the other extreme embedded CBs, without a stormscope are more than I am willing to take on.
Of course as is usually the case with the weather there is every variation in between.
There are a number of good books that deal with met for the VFR pilot which are well worth a read and include solid discussion about flying around CB activity and how to avoid the worst of the associated problems. I would certainly invest in one of the better ones before the summer season is properly underway!