If you have separate tickets then you will have to enter the UK, pick up your baggage, take the train to T3 and start all over again.
With a through ticket you would probably be given both boarding passes at BRU and be expected to use the airside transit options. If you choose to exit landside and use the train between T5 and T3 you won't need to check in again in T3 simply present your boarding pass at T3 security. If you don't get a boadring pass for LHRMCO at BRU that might cause a few scratched heads when you checkin in T3 but I don't see anything insoluble.
In the end I see no reason why you are barred from doing it. The only possible problem might be if the time taken means you miss the flight from T3; if they discover you've been landside they might be difficult but I think that's unlikely.