I go to ManageFeatures.aspx and I see bunch of features that don't belong there and they are not in the 14 hive's feature folders. I go to the backend database and look in alldocs, features, and featuretracking tables and don't see them. Now, where do they came from and how do I get rid of them?
You could use PowerShell to get the GUIDs of the features you want to get rid of, here's a blog form Glyn Clough on how to do this: http://www.glynblogs.com/2010/08/powershell-commands-to-list-sharepoint-features.html
stsadm -o uninstallfeature -id <feature_guid> to uninstall the feature (might be able to do the same with PowerShell, I think).
Assuming they these "orphaned features" are active, you can click the deactivate button and note the featureid in the querystring. With that you should be able to use STSADM deactivatefeature and/or uninstallfeature with the force option to remove them.