I have a site collection and the only way for people to access the site is by adding them to the site collection administrators group. I have several SharePoint Groups setup with varying permission levels from readonly to full control, but none of the users in these groups can access the site. Has anyone seen this before and if so how do you fix it?
Try creating a new site collection (team site) under the same web app. Perform the same security configuration with the groups and all that. Do you experience the same behaviour?
Also, do you have a custom master page? If so, firstly ensure it is in a published state. Are there any custom controls on it that try to access protected resources (i.e. a control that reads data from a list in a subsite that has unique permissions)?
Can they not access any part of the site? Can they access the view all site content page? http://siteurl/_layouts/viewlsts.aspx
If they can view all site content then I'm guessing there is an issue with a webpart on your homepage.
It sounds to me that the content on the site does not inherit from the site permissions. A site collection administrator is always going to have full control which is what allows you to repair mistakes like this.
Is it a publishing site or a regular team site? In a regular team site the default.aspx page should allow access based on the site's permissions.