sharepoint client object model for silverlight security error

Question

I tried to get data from sharepoint site by client object model. Everything worked fine with .NET object model, but as for silverlight with seperate page (not hosted in sharepoint silverlight web part) from silverlight application on a PC, it directly produced security error. Thinking that might be cross domain security problem, I tried to use a clientaccesspolicy.xml and this time sharepoint site login window poped up. But even when I logged in as the sharepoint administrator, it still produced security error. I now wonder whether this might be associated with security mode for the sharepoint site, could there be restricted security mode for silverlight client object model ? Should only FBA or Anonymous mode be used? I appreciate if you would provide some advice the help. Thank you!