The country in which your medical records are held (usually where you have the first medical) is the state that will administer your licence. It is the quirks of that national authority that need to be met.
Some EASA states will accept the theoretical knowledge exams from other states so check which are accepted by your chosen Authority.
Neither the Medical or Theory MUST be completed before starting flying but you must have the medical before going solo and all the theory must be completed prior to doing the final exam. I’ve found most people do the theory exams as they progress through the flying as a little practical experience can make the theory easier to digest.
Once you have a medical there is no real need to go back to that country as all subsequent Medicals can be done in any EASA state.