In my research/learning of Office 365, I came across this great article outlining the features available/unavailable in Office 365 when you create a public-facing website. Covers most of what I had already experienced myself and some things I hadn't, so it's a nice guide to read through:
Of note are some of the "hidden" bits and pieces which still appear to be
available - although I'm not sure where this places a site from a support
perspective if you do alter some of these "hidden" settings.