Hi i configured windos and Farms based authentication for web application,when i login to secured pages it redirect to login page and has options to select windows and farms authentication, but in same web application there is any way to set farms authentication only to a specific site or site collection, instead of changing host name i want to keep same host name for web application.
No, authentication settings are scoped to the Web Application [Zone].
You are out of luck in the required scenario. Authentication is web app level setting not site collection.
Now you can create a new web application and configure the FBA and move the that site collection or extend the existing web application but your URL will be changed.