Is it possible to handle the click event of a file in a sharepoint document library so that instead of opening the actual file, it opens an application page?
E.g. Open letter.xml goes to letter editor.aspx in layouts.
Though, you need to tweak it a little bit to fit your needs. You could also use jQuery but in my case I didn't want to rely on a third-party lib for such a simple task.
If you want a file to be open by making user to open it via "ECB" menu for an item then you can create custom action (check here for simple walkthrought http://codename-srini.blogspot.com/2010/04/using-custom-actions-in-sharepoint-2010.html) and check the respective location group setting name i.e. ECB (complete ref here : http://msdn.microsoft.com/en-us/library/ie/bb802730.aspx).
You can work with the UrlActions to redirect to a ASHX handler or _layouts page and open the file.