Another way, if you can't create a separate document library, is to try defining a boolean column say "IndexDocument" in your document library whose default value will be "Yes".
For all those documents which you don't want to be indexed, set the value to "No".
Perform a crawl to get the property as "Crawled Property".
Define a managed property based on that crawled property. Set it to be used in Scope.
Add a rule based on "Property Query" such as
IndexDocument=No and Select "Behaviour"=Exclude
Update Scopes and you are happy to go.