I did this once where the requirements abosolutely required allowing the file extension to be changed while keeping the version history intact. It required a lot of customization.
The first piece was the upload, which required a custom page. This used SPFile.MoveTo(), as Vedran mentioned. The drawback is there is no OOTB history of what the extension used to be once it is changed.
So the second piece was an event handler to store the file extension of the current version in the property bag of the list item (eg, ".doc") each time a new version was checked in.
The third part was a custom version history page that swapped the download links (among other things), with a link to a custom aspx page that could deal with fixing the filename for previous versions.
The download page would grab the document version, get the correct file extension out of that version's property bag, and stream the correctly named file to the browser.
As you can see it was a lot of work. So it is probably not worth it unless absolutely required.