I got some CSOM scripts in a .Net Application, that I have to to move to the server, using the server object model/powershell. As our Sharepoint team has no time and skills, it was decided that I have to do this. My question is, how can we set up my authorization to fulfill this task? I am a site collection administrator and all the scripts refer to that site collection. Would it be enough, if I get remote login to the server? If possible I would like to avoid having any extra rights that I do not explicitly need.
The permissions to use the SSOM are highly privileged. If minimal permissions are a requirement you should stick to CSOM.
To use the SSOM you need to have shell admin permissions. See this Technet article for full details.