How do you set the permission on the document library to allow only authors of the document to edit it and all other users will have read-only access? Can this be done on the check out/check in process as well?
You can use event handlers to control edit permissions.
You can have an event handler that registers an event using the SPEventReceiverType.ItemUpdating and SPEventReceiverType.ItemUpdated methods, and then use code to control whether the update should be allowed.
This is extremely powerful, because you can make security decision based on any metadata of a list or item, without affecting the view rendering performance.
We have used workflow, which executes when new item is added and it sets editing permission for the author.
I have dealt with this situation before and the simplest solution coming right out of my mind now, is to create an event handler which will run when the document is uploaded. The code within the handler will remove all permissions and will grant only the admin/contribute rights to the document owner. Also for all the documents that are already present in the library, one can write a simple C# application that you can run to again do the same as above. Hope I have explained it clearly.