There are a few things that control this in site settings --> navigation.
Managed Navigation -- use a term set for navigation
Add new pages to navigation automatically -- a new item is added to the termset for each new page
Create friendly URLs for new pages automatically -- This is what you're concerned about - the page shows up in the URL without the .aspx.
So, a couple options:
disable the create friendly URLs for new pages - this will work, but only applies to new pages.
Do the above, but also open the term store management tool, navigate to the term set for the current site collection (at the bottom on the left), drill down to the particular page (term) you want to change, and on the navigation tab switch it from a term-driven page to a simple link (repeat this step for every existing page)
Change the navigation for both the top and current to structural instead of managed. (this is a separate option and can be done instead of the first two)
edit: to change this setting from powershell:
Add-PSSnapIn Microsoft.SharePoint.PowerShell
$web = get-spweb "http://someSharePointUrl"
$navSettings = New-Object Microsoft.SharePoint.Publishing.Navigation.WebNavigationSettings($web)
$navSettings.AddNewPagesToNavigation = $false
$navSettings.CreateFriendlyUrlsForNewPages = $false
$navSettings.Update()