The DNS record points to server A only (I am not sure if adding a DNS record for mysite.domain.com pointing to server B will work). The application pool is running on server B which runs under the identity "spservice". I changed my local hosts file pointing mysite.domain.com to server B, which only worked when I restarted the UPS service. Any ideas?
Perhaps this article will help you. Basically when setting up software NLB on a small server farm, some switches (like the Cisco Catalyst mentioned in the article) have problems with multicast MAC addresses in their ARP table.
In the case of the Catalyst you can fix it with a command like this:
arp 172.16.63.241 0300.5311.1111
The article explains:
In this command the first parameter is the IP Address of your cluster (in same VLAN as servers). The second parameter is the MAC address of the cluster, which you can read from NLB manager. It’s in a little different format – for example the above entry would look like 03-00-53-11-11-11 in NLB manager. You can find this information under Cluster->Properties from the NLB manager menus.
Hope that helps resolve the issue!