The SharePoint People.aspx page is visible for all users by default. Is there any way to restrict this page from certain readers?
This is sort of possible. You can do the following to adjust permissions for the items, however, the result may not be exactly what you desire.
On your Search Center site, goto Site Actions > View All Site Content
In the Pages library, set the permissions on the "People" page to only allow the users you want to see it.
In the Tabs In Search Pages and the Tabs In Search Results lists, set the permissions on the "People" item.
This should result in the Tab only showing for the appropriate users and access to the page as well. However, if they are redirected here from somewhere else (i.e. the My Site search box), they will get an ugly Access Denied page.