We want to provide anonymous access to the contents of a library through URLs (like image references for the site), but we don't want a user to have library view/browse capability. The site collection is a team site.
With Publishing Sites in SharePoint 2010 there is a Feature called LockDown which disables automatically access to EditForm.aspx and AllItems.aspx for Anonymous.
This combined with the use of a Content Query Web part or a customized Data View (or a custom Web part - if you are in mood to go for the full-run) on a Page, should avoid anonymous to be able to get to views but still download Documents.
To transform the site into a Publishing site simply enable Publishing Infrastructure at the Site Collection level. You could have another Site be the actual Publishing Web to expose data in a Sub-Site - which still could remain a Team Site.