Lookup field deployed through feature can't be added to new list

Question

I have deployed a number of fields, content types, and list definitions/instances to a site through a couple features. The fields and content types are deployed through a Site scoped feature, and the lists are deployed through a Web scoped feature.

Some of the fields are Lookup fields that are supposed to lookup to the lists I am defining and deploying in the Web scoped feature. Because I'm creating and deploying the list instances, I know what their URL is going to be so I create my Lookup fields like this:

  <Field ID="{some-guid-here}"
         Name="FieldName"
         DisplayName="Field Name"
         Type="Lookup"
         List="Lists/My Custom List"
         ShowField="Title"
         Group="My Custom Columns"
         DisplaceOnUpgrade="TRUE"/>

When I go through the process of adding fields to content type definitions and adding content types to list definitions, VisualStudio copies all of the relevant CAML, so the field definitions in my list Schema.xml files end up with the list URL as the value of the List attribute.

This seems to work, at first. After deploying and activating the features, the Lookup fields are correctly hooked up to the lists they are supposed to be hooked up to.

But now I want to add one of those content types to a new list. I create the list through the regular SP UI (Site Contents -> Add an App). I go into the list settings, enable content type management, and go to add an existing site content type.

But then everything blows up.

Here's the stack trace from the logs:

System.ArgumentException:    
 at Microsoft.SharePoint.Library.SPRequestInternalClass.AddField(String bstrUrl, String bstrListName, String bstrSchemaXml, Int32 grfAdd)    
 at Microsoft.SharePoint.Library.SPRequest.AddField(String bstrUrl, String bstrListName, String bstrSchemaXml, Int32 grfAdd)    
 at Microsoft.SharePoint.SPFieldCollection.AddFieldAsXmlInternal(String schemaXml, Boolean addToDefaultView, SPAddFieldOptions op, Boolean isMigration, Boolean fResetCTCol)    
 at Microsoft.SharePoint.SPContentType.ProvisionFieldOnList(SPField field, Boolean bRecurAllowed)    
 at Microsoft.SharePoint.SPContentType.ProvisionFieldsOnList()    
 at Microsoft.SharePoint.SPContentType.DeriveContentType(SPContentTypeCollection cts, SPContentType& ctNew)    
 at Microsoft.SharePoint.SPContentTypeCollection.AddContentTypeToList(SPContentType contentType)    
 at Microsoft.SharePoint.SPContentTypeCollection.AddContentType(SPContentType contentType, Boolean updateResourceFileProperty, Boolean checkName, Boolean setNextChildByte)    
 at Microsoft.SharePoint.ApplicationPages.AddContentTypeToListPage.Update(Object o, EventArgs e)    
 at System.Web.UI.WebControls.Button.OnClick(EventArgs e)    
 at System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument)    
 at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)

It's throwing an Argument Exception when trying to add a field. I'm guessing that that's because the List attribute in the field definition is a URL, and not an ID.

I cracked open Powershell to poke around, and see what was going on, and indeed on the site level field, the List attribute / LookupList property is still the URL. But if I look at the list level field (which originally had the URL in the list Schema.xml), the List / LookupList has been converted to an ID. I guess that happens during deployment?

Now, from the MSDN documentation on the LookupList property of an SPFieldLookup object, it says:

Once a lookup column is established, the target list cannot be changed.

And, not one to believe it without seeing it, in Powershell I tried to change the LookupList property of the site level field to be the ID of the list it was supposed to be hooked up to, and sure enough, when I tried to Update the field, it threw an error and didn't save the change.

So, what are my options here? How can I add a site level content type that has a lookup field with a URL as the LookupList property to another, new list that was not deployed as part of the feature?

One option I am considering is creating a new feature with definitions for the new lists I need that will have that content type. Then when the lists get deployed, the URL will be converted into an ID.

But I'm wondering if there are other options, or some way of fixing the underlying problem and making that field and content type more usable.

Answer 1

Please try below suggestion if not already done.

Try creating multiple features separating dependencies. I prefer to create different features for columns, content types, list definitions and one for list instance. In case of lookups you will need to create separate feature for lookup list instance. You can bundle the elements in other ways as well, but always ensure you separate dependencies.

When deploying your solution, activate features in the right order i.e. first activate features that provision the lookup list instance first. Then proceed to activate features for other site columns, content types, list definition and instance in the this order. When the feature that contains lookup column is activated the List property would resolve correctly as lookup list already exists on the site.

If this is on your dev environment, do not use visual studio to deploy and activate features. Set the visual studio project deployment template to 'No Activation' and manually enable features to ensure it is done in right order.

Answer 2

I recently ran into this issue again, so I figured I'd finally post the answer of how I dealt with it.

Part of the problem was that I like to deploy my field definitions and content type definitions with a Site scoped feature, and my list definitions/instances with a Web scoped feature. And, in this case there was the added complexity of using lookups to other lists in the lists that had lookup fields for them - basically setting up a chain of lookups to be able to use SPServices Cascade Dropdowns. So the field definitions had to be in place before the lists or the lists wouldn't deploy correctly.

Now, while it is true that you can't programmatically change the LookupList property of a lookup field directly, you can edit the field definition stored in the SchemaXML property, and that will internally update the LookupList property.

I found this solution by Stefan Bauer here (with a near identical post here).

So in the end what I did was deploy the field definitions and content type definitions first, through a Site scoped feature, with the List attribute of the field definitions still set to web relative URLs. Then, I added a feature receiver to the feature that deploys the lists that the lookups point to. In the FeatureActivated event, I grabbed the fields from the Site level (RootWeb.Fields), used the List attribute from the SchemaXML to get the now deployed and existing lists, and then replaced the relative URL with the actual list ID in the List attribute. Then save the SchemaXML back to the field, and update it with the push-down option.

Now all the site level field definitions have the ID instead of the URL, and everyone's happy.