And in fact this is all Outlook allows, as far as I can tell. And I basically never want to see images in email in-line...

Time to write that mail client again, I guess. (I fake this by using AdBlock to block images in my online mail apps, which carefully wrap mail-embedded images in easy-to-regexp urls.)