The users come to my site from a link setup on another site. I need to create an Exit link on the header of my Site Collection. When user clicks on this Exit link they should be send to the site or page they came from.
You could create a custom web control (or a web part) that picked up on the http referer (you can pick http_referer up from ServerVariables) first time a user comes from an external site, and set it for the link back.
Things to look out for is that page isnt cached, as users would then share referer link (you could wrap your control in a substitution control to avoid it being cached if page is cached).
Also as you have a referer every time you click a link you would need to check the content of the referer targeting only the relevant referer control.