Besides the fact that you are essentially disabling the end user's ability to customize the navigation through the UI, the only real issue I could see with this is performance. Iterating through all of the SPWeb objects will not be as efficient as using the OOTB site map provider. I would urge you to build in some sort of caching strategy. Also, you will need to account for the fact that not all users will have access to all sites within the collection. Another approach might be to add a "WebCreating" event receiver to insert an item in a list within the root site and build the navigation from that. At least that way users could remove or edit individual items, and even add links directly to external Urls. I could probably give a better answer if you explained why you need to build a custom control for navigation.
Also, do not be attempted to implement your object model accross site collections unless you plan to implement a timer job that will build the site map (and export it to something like xml).