On my SharePoint farm I have a custom solution installed which contains a Site feature with an index.aspx page (has no master page). I am looking at how to setup custom 404 pages. However when I try to open the URL to one of the feature's pages under a non-existent site collection I don't get the 404 page but instead I see an error.
It seems SharePoint will return a 404 for pages that are not in the feature, say "foo.html", but if construct a URL to my feature's index.aspx page https://sharepointdev/sites/nonexistentcollection/_layouts/MyFeature/index.aspx (does not exist) SharePoint does not return a 404. Instead it tries to serve the page and that page returns a 500 error (for obvious reasons).
Is this a normal SharePoint behavior or is something in my solution or farm setup incorrectly?