I presume you are travelling with Icelandair ... and this is not only their 'home' base but the nerve centre of their 'hub' operation between Europe and the US.
Your transfer will be straightforward, as will your baggage transfer between aircraft, although you may have to pass through a security control.
This is their speciality and they get it right.