I've had a similar issue, let me walk you through
1. When My Site host is on another web application you need to add the clientaccesspolicy.xml file in the root of your web application, in Inetpub. If MySites host in on the same web app it works without (but is not done)
2. I've copied from the OrganizationView.aspx page (which is provisioned in the root of your MySites host), 2 important tags:
a. the **SPWC:PeopleBrowser** tag, which is the actual Organization browser Silverlight web part
b. the `<SPSWC:ProfilePropertyLoader id="m_objLoader" LoadFullProfileOfCurrentUser="true" runat="server" />` which enables actual load of the property values
3. Once this is you could by example use, the same way as Person.aspx does, use the QueryString to pass account name to put in center, otherwise by default it takes currently loggedin user.
Hope it helps,
c:\Marius