If the actual Service is stuck in Stopping or Starting:
Firstly, run this:
This will show a list of service Instances. Find your broken Service (Status will be Provisioning or Unprovisioning)
Get the GUID of this Service
$srvc = Get-SPServiceInstance "GUID"
Wait for sometime it will stop
Verify that you have the following administrative credentials to delete UPA:
- To use Central Administration to delete a User Profile Service application, you must be a member of the Farm Administrators group.
- In the Delete Service Application dialog box, verify that you have selected the correct User Profile Service application to delete, select Delete data associated with the Service Applications to delete all data associated with the selected User Profile Service application, and then click OK.
Follow the instructions in Stuck in Stop/Start article