2

I discovered this strange problem when using a multi line plain text field to store a url. The behaviour also occurred in single line text fields.

Steps to reproduce:

  1. In a plain text field in a SharePoint list, use the REST api to store something containing "%20", e.g. "Test%20"
  2. Look at the resulting list item, and notice that the text is now "Test+"

This is causing problems when storing url's in text fields, because url's containing spaces are often URL encoded with spaces replaced by %20. With a round trip through the REST api, those spaces are ultimately converted to + signs, and the url no longer works.

I know I can probably use the hyperlink field type, but I have preferred the text field due to its simplicity. Text is just text, right? At least until SharePoint performs its magic...

I have tried several other url encoded values, but only this one character seem to be affected.

Anyone seen this before or know a workaround?

Here is a code sample to reproduce the problem (paste in a script editor and create a list called 'TestUrlEncode'):

<!DOCTYPE html>

<html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
    <meta charset="utf-8" />
    <title>Test of URL encoded values</title>

    <script type="text/javascript" src="//code.jquery.com/jquery-3.1.1.min.js"></script>

    <script type="text/javascript">
        $(document).ready(function () {
            TestUrlEncode();
        });


        function TestUrlEncode() {
            UpdateTestListItem();
        }

        function ReadListItem() {
            var url = _spPageContextInfo.webAbsoluteUrl;

            $.when(GetListItems(url, "TestUrlEncode")).done(function(data) {
                $("#myValue").text(data[0].Title);
            });

        }

        function UpdateTestListItem() {
            var url = _spPageContextInfo.webAbsoluteUrl;
            var myTitle = (new Date().getTime() / 1000) + " %20";
            var reqData = JSON.stringify({
                    '__metadata': { 'type': 'SP.Data.TestUrlEncodeListItem' },
                    'Title': myTitle
                });


            $.when(UpdateListItem(url, "TestUrlEncode", 1, reqData)).done(function() {
                ReadListItem();
            });
        }


function GetListItems(url, listName, filter, orderby, expand) {
    var dfd = new $.Deferred();

    var reqUrl = url + "/_api/web/lists/getbytitle('" + listName + "')/items?$top=5000";

    if (filter != null && filter != "")
        reqUrl += "&" + filter;

    if (orderby != null && orderby != "")
        reqUrl += "&" + orderby;

    if (expand != null && expand != "")
        reqUrl += "&" + expand;



    // Executing our items via an ajax request
    $.ajax({
        url: reqUrl,
        method: "GET",
        headers: { "Accept": "application/json; odata=verbose" },
        success: function (data) {
            var result = null;
            if (data.d.results.length > 0) {
                result = data.d.results;
                dfd.resolve(result);
            } else {
                dfd.resolve([]);
            }
        },
        error: function (object, errorCode, errorMessage) {
            errorMessage = "Error retrieving elements from " + listName + " with url: " + url + " Error messsage: " + errorMessage;
            dfd.reject(object, errorCode, errorMessage);

        }
    });

    return dfd.promise();

}

function UpdateListItem(siteUrl, listName, listItemId, reqData) {

    var dfd = new $.Deferred();
    var reqUrl, reqHeaders;

    reqUrl = siteUrl +
        "/_api/web/lists/getbytitle('" + listName + "')/items(" + listItemId + ")";

    reqHeaders = {
        "content-type": "application/json;odata=verbose",
        "content-length": 0,
        "X-HTTP-Method": "MERGE",
        "X-RequestDigest": $("#__REQUESTDIGEST").val(),
        "IF-MATCH": "*"
    };

    $.when($.ajax({
        url: reqUrl,
        type: "POST",
        data: reqData,
        headers: reqHeaders
    })).done(function () {
        dfd.resolve();
    }).fail(function (object, errorCode, errorMessage) {
        dfd.reject(object, errorCode, errorMessage);
    });


    //Return the Promise
    return dfd.promise();
}



    </script>
</head>
<body>
    Value from SP List goes here:
    <div id="myValue"></div>
</body>
</html>
  • Did the list item contains %20 or it contains + already and REST api simply displaying what is stored in list item. – ThinkB4Code Jun 8 '17 at 8:14
  • See modified post. After REST update, the list item contains + – Thorvald Bøe Jun 8 '17 at 9:34
  • Danny, I don't understand your edit ' ' see comments on answer, this is not a Date issue, but a loading MomentJS issue' - did you intend to edit another post...? – Thorvald Bøe Jun 8 '17 at 11:34
  • I think I have found a workaround: By replacing the % in %20 with an escaped unicode character, I was able to get it right: reqData = reqData.split("%20").join("\\u002520"); Not very elegant, but an awkward solution to an awkward problem. – Thorvald Bøe Jun 8 '17 at 12:57
  • First of all, look carefully at: your string " %20". There's a space there. Of course the resulting string will never be "Test%20", at best it will be "Test %20" which I can't think of a case where that is what you want. – JanErikGunnar Jun 9 '17 at 11:49
1

Using your code, was able to reproduce this issue.

From Wikipedia:

When data that has been entered into HTML forms is submitted, the form field names and values are encoded and sent to the server in an HTTP request message using method GET or POST, or, historically, via email. The encoding used by default is based on a very early version of the general URI percent-encoding rules, with a number of modifications such as newline normalization and replacing spaces with "+" instead of "%20". The MIME type of data encoded this way is application/x-www-form-urlencoded, and it is currently defined (still in a very outdated manner) in the HTML and XForms specifications.

So, the real percent encoding uses %20 while form data in URLs is in a modified form that uses +.

Now, to send the %20 i would suggest that you use the encodeURIComponent method of javascript to encode your string.

Your code would be replaced as :

var myTitle = encodeURIComponent((new Date().getTime() / 1000) + " %20");

encodeURIComponent

  • Thanks for pointing that out, Gautam, that is really useful in order to understand the issue. However, your code does not work for me. The myTitle variable gets a value like this: "1496926357.708%20%2520" which displays like this after being saved to SharePoint: 1496926357.708+%2520. – Thorvald Bøe Jun 8 '17 at 12:55
  • What are trying to do exactly ? Am a bit confused:) , storing date in title field ? – Gautam Sheth Jun 8 '17 at 13:04
  • No, this has nothing to do with dates. It seems Danny Engelman edited the wrong post. I will remove the edit comment. – Thorvald Bøe Jun 9 '17 at 7:42
0

I did a test with an extra Single Line of Text on a standard Tasks list

No issues at all

Do that same test (Chrome or Edge to use the Fetch API), then check the item with:

var tasks = new Request(
    _spPageContextInfo.webAbsoluteUrl+"/_api/web/lists/GetByTitle('Tasks')/items"
    , {
        method: 'GET',
        credentials: 'same-origin',    // or credentials: 'include'         
        headers: new Headers({
            "Accept": "application/json; odata=verbose",
        })
    });

fetch(tasks)
    .then((response) => response.json())
    .then((data) =>  {
        let items = data.d.results;
        items.forEach(item => console.log(item.ID,item.Title,item));
    });

Cause?

Could it be your values are URLencoded by Client Side code before written to SharePoint ??

Do you override the PreSaveAction with custom code?

  • Thanks for the quick reply! I will try out your example to see if I have some issues with my headers or something. The values are fine after they are written to SP, the strange behaviour definitely happens in the GET request. We are using some standard utility functions to read the list items though, I will go through them to see if there is anything special there and maybe try to produce some code to reproduce the problem. – Thorvald Bøe Jun 8 '17 at 8:18
  • I was wrong, it was in fact the write operation that caused the problem. Still unable to find the cause. See modified post above with working code sample. – Thorvald Bøe Jun 8 '17 at 9:27
0

Gautam put me on the right track, so I was able to find a workaround. The workaround is simply to replace the "%" in "%20" with a unicode escaped character, so that it avoids being replaced on the server, while still keeping the correct value. Full working code sample with workaround here:

<!DOCTYPE html>

<html lang="en" xmlns="http://www.w3.org/1999/xhtml">
<head>
    <meta charset="utf-8" />
    <title>Test of URL encoded values</title>

    <script type="text/javascript" src="//code.jquery.com/jquery-3.1.1.min.js"></script>

    <script type="text/javascript">
        $(document).ready(function () {
            TestUrlEncode();
        });


        function TestUrlEncode() {
            UpdateTestListItem();
        }

        function ReadListItem() {
            var url = _spPageContextInfo.webAbsoluteUrl;

            $.when(GetListItems(url, "TestUrlEncode")).done(function(data) {
                $("#myValue").text(data[0].Title);
            });

        }

        function UpdateTestListItem() {
            var url = _spPageContextInfo.webAbsoluteUrl;
            var myTitle = (new Date().getTime() / 1000) + " %20";
            var reqData = JSON.stringify({
                    '__metadata': { 'type': 'SP.Data.TestUrlEncodeListItem' },
                    'Title': myTitle
                });


            $.when(UpdateListItem(url, "TestUrlEncode", 1, reqData)).done(function() {
                ReadListItem();
            });
        }


function GetListItems(url, listName, filter, orderby, expand) {
    var dfd = new $.Deferred();

    var reqUrl = url + "/_api/web/lists/getbytitle('" + listName + "')/items?$top=5000";

    if (filter != null && filter != "")
        reqUrl += "&" + filter;

    if (orderby != null && orderby != "")
        reqUrl += "&" + orderby;

    if (expand != null && expand != "")
        reqUrl += "&" + expand;



    // Executing our items via an ajax request
    $.ajax({
        url: reqUrl,
        method: "GET",
        headers: { "Accept": "application/json; odata=verbose" },
        success: function (data) {
            var result = null;
            if (data.d.results.length > 0) {
                result = data.d.results;
                dfd.resolve(result);
            } else {
                dfd.resolve([]);
            }
        },
        error: function (object, errorCode, errorMessage) {
            errorMessage = "Error retrieving elements from " + listName + " with url: " + url + " Error messsage: " + errorMessage;
            dfd.reject(object, errorCode, errorMessage);

        }
    });

    return dfd.promise();

}

function UpdateListItem(siteUrl, listName, listItemId, reqData) {

    var dfd = new $.Deferred();
    var reqUrl, reqHeaders;

    reqUrl = siteUrl +
        "/_api/web/lists/getbytitle('" + listName + "')/items(" + listItemId + ")";

    reqHeaders = {
        "content-type": "application/json;odata=verbose",
        "content-length": 0,
        "X-HTTP-Method": "MERGE",
        "X-RequestDigest": $("#__REQUESTDIGEST").val(),
        "IF-MATCH": "*"
    };

    reqData = reqData.split("%20").join("\\u002520"); //replace the % in %20 with unicode escaped character

    $.when($.ajax({
        url: reqUrl,
        type: "POST",
        data: reqData,
        headers: reqHeaders
    })).done(function () {
        dfd.resolve();
    }).fail(function (object, errorCode, errorMessage) {
        dfd.reject(object, errorCode, errorMessage);
    });


    //Return the Promise
    return dfd.promise();
}



    </script>
</head>
<body>
    Value from SP List goes here:
    <div id="myValue"></div>
</body>
</html>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.