The users are automatically logged on, if they bear a valid Kerberos-Token and the site fullfills some requirements (Non-FQDNs work most times). This will only happen to users part of the same domain (so users outside of your domain will be asked for credentials).
To remove this behavior (which comes from Windows-Integrated with NTLM or Kerberos) just deactivate Windows-Integrated authentiction. I'd recommend basic auth, since the FBA comes with other pitfalls.
For clearification:
The fact if an application is claims-aware does not decide if you will be promted for your credentials or not. This only depends on the choosen authentication method (which may be "Windows Integrated" (=Kerberos or NTLM), Basic or Form-Based). The built in logic in your Browser will decide if it will prompt or not.
Form-Based will always be promted (if you don't save your credentials in a cookie or whatever).
Basic will always be promted (you can store that in your browser).
NTLM (=Negotiate) might promt you for credentials in some circumstances (no SSL, some FQDN issues, whatever - IE will more likely pass through your authentication than Chrome and FF).
If you want to see which method is used or available, use fiddler and look into the authentication-headers. There's also an optical difference between NTLM and Basic, and NTLM needs the Domain (Basic knows a default domain).
All three methods CAN authenticate against ActiveDirectory (do Windows-Authentication) but only NTLM has to.