Setting a list item ID only works when using the field GUID based value setter

Question

Due to some weird concomitance of odd application requirements and legacy code to maintain, I am currently stuck having to programmatically create some list items on a SharePoint 2016 list and assign each of them a custom hardcoded ID (and yep, I know that is generally not a good idea, but as I said... it is legacy code that a) we don't wrote b) will probably have to stay as is for now).

Since normally the item ID is an autogenerated value, I searched for a confirmation that it could be also set programmatically and found this answer. Without going into further details on the actual code used in the linked post (some part of which imho could be not actually needed), let's just consider the actual line that sets the ID field value on the list item

listItem[SPBuiltInFieldId.ID] = someValue;

As you can, the original code provided by the answer uses the ID field GUID to access the field setter. Due to a mere case of sheer bad luck, without even noticing at first while applying said code to my actual situation I altered it a little, to use the field internal name instead.

listItem["ID"] = someValue;

To my surprise, when I tried to execute my code, an exception was raised on the above line, with an error message "Can not update the field value".

I started debuging my code until - to even more surprise - I discovered that if I used the first variant of the setter (the one that uses the GUID) the code would work as expected, while the second variant would fail.

Since both field value accessors should access the same list field, I wondered how that would be possible.

---

NOTICE: I didn't attempt to replicate this behavior on others SharePoint version so this "bug" is only tested on SharePoint 2016 for the time being. For that reason, I haven't yet added any version specific tagging on this question since we still don't know for sure if the issue is version specific in the first place.

Answer 1

I am self answering this question in hope this may be useful to other users that will notice the same nonsensical behavior discrepancy in the two available setter in the future.

The observed issue seems indeed to be a bug: due to the way the two different accessor are written, they actually result in two slightly different (and to my knowledge undocumented) behaviors. Let me explain.

I used Resharper to actually debug and have a look at the SharePoint source code that get executed in the two cases. Both samples end up executing an internal method on the SPListItem class:

internal void SetValue(string strName, object value, SPField field, bool protectFields, 
                       bool skipValidation)

the actual difference is in the values that get passed to the method parameters in the two cases.

When using the listItem["stringFieldName"] setter method, the strName parameter is valorized with the supplied field name while the third parameter, the actual SPField is assigned a null value. Basically (notice: for sake of readability, I have skipped some intermediate methods overload calls in the sample. The final result is the same):

this.SetValue(strName, value, (SPField) null, protectFields, false);

When the listItem[fieldGuid] setter is used instead, the opposite happens. The field name is left null and the SPField instance is passed down to the internal method:

this.SetValue((string) null, value, field, true, false);

The final result is the same in both cases: when the field isn't available the field name is used to get it from the underlying list. So why we get an error when setting the ID field value only when using the field name based overload?

Simple: the internal method start with this code.

 if (protectFields && (strName == "_CopySource" || strName == "_HasCopyDestinations" 
     || (strName == "ID" || strName == "Restricted")))
    throw new SPException(SPResource.GetString("UpdatesToThisFieldNotAllowed"));

At this point I think the issue is clear: the internal method SharePoint uses to set the value of a SPListItem field is coded to raise an error when some specific field names are used, and the "ID" field is one of said protected fields. This actually makes some sense in context. The actual problem is that when we use the field GUID to access the field instead, no field name is supplied to the internal method. Therefore, because of the (bad) way the code is written the logic doesn't actually recognize that we are attempting to update the "protected" ID field and no error is raised at all - only the field NAME is checked but since there is no name in this case there is nothing to check!!!. This means that when using the field GUID based setter we are actually unknowingly skipping some basic validation checks SharePoint should be performing...

Notice that while this does indeed answer my original question, it does also leave us with an new - and probably worse - one: which of the two different behavior is the correct intended one and which one is the bug? Should we be able to update the ID field values using either accessor or the actual problem is that we are able to bypass internal validation by using the GUID based one?
Sadly, I think only Microsoft can really answer this last question.

Answer 2

Although this does not directly answer the question, I wanted to share my thoughts.

Although the above technique can be used to set the item Id and it may be required while migrating legacy applications even though not recommended. I found that setting item Id using this technique causes few more problems. One issue I found was that even though it sets the Item Id, it does not increment the internal counter, which SharePoint uses to determine which Id to use for newly created items (atleast on SP 2013).

For example, I set the item Id for two items to 5, 6 and then go to UI and add bunch of items, since SharePoint is not fully aware of next item Id to be 7, it will start at 1 as usual until it hits 5 and then fail with an error. What I also found that clicking the save button will actually increment the counter (despite the error) and on third try will finally assign 7 and create new item. I found this behavior to be problematic to end user if they plan to actively use the list.

I found it much easier to add a bunch of dummy items just to get the desired next item Id and then delete the dummy items at the end. The creation of the dummy items and deletion process can be batched to speed up the process.