Open a new email in Outlook. Click the ¶ button if you have one on the standard toolbar to toggle the little devils on or off.
You can also click Tools > Options and in the View tab uncheck any and all that are in the Formatting Marks section when composing a new email. This menu is only available in the editing window.
Good Luck!