We can achieve it by creating workflow in SharePoint, I am sharing the basic steps to for creating the workflows. In the explanation below I'm using SharePoint 2013 but it is absolutely same in SP2010 or/and on premises, and even in SP2007 the process is very similar.
1) You will need SharePoint Designer installed on your computer.
For Creating workflow:
2) Open SharePoint designer
3) Open your site
4) Go to Workflows
5) Click 'List Workflow' on ribbon and select your list
6) Add 'Send email' action
7) Configure 'Send email' action to use refer to field for determining recipients of email. Also define the body block for the message. You can also use different lookups here, for example inserting values from the item or item URL, etc.
8) Now go to 'Workflow Settings'
9) Setup workflow to start whenever item is created or changed.
10) Publish your workflow.
This workflow will send Email to the user in refer to field whenever list item will change. If you want to track only the change in refer to field then you need to create one more hidden field in your list as "previous refer to".
You can add user name to this field using workflow. Every time after sending email notifications check this previous refer to and refer to is different or not. If they are different then send email and set "previous refer to" field.
How to make hidden field:
Create new person or group field named "previous refer to".
Go to List Settings > Advanced Settings.
Select Yes on Allow management of content types?
Click on the existing content type under the Content Type section.
Next, click on the column you intend to hide, and choose the option Hidden. Click OK.
How to check if the field is empty in workflow:
Set workflow variable to your "previous refer to" field.
Add IF condition in workflow and check if the newly created variable is empty or not.
Source: How to check Person or group field is empty or not.