I noticed that when I added and removed people to/from a group for a subsite (3 levels down), then it in fact added and removed them from the root site! I can swear I was being very diligent and I could clearly see the subsite title not just on the settings page but also in he url, but still when I switched to the settings page for the top-level root site, all the changes I did to the subsite were now appearing on the top level site. How is this possible? It is so ambiguous.
How can we prevent this from happening. One should expect that when they are adding/removing people to/from a group, they are doing at that site level on which they are on and not on the top level site.