How to hex encode refiners

Question

I'm trying to build some custom links from search results to a content search webpart page that also uses refiners. I examined the hash added to the url when using the refiner from the CSWP page and have been able to reconstruct it in my search item template.

I've found that the refinement token used in the url hash for the refiners are hex encoded text. And I found this code to transform strings to hex and it mostly works:

function String2Hex(tmp) {
    var str = '';
    for(var i = 0; i < tmp.length; i++) {
        str += tmp[i].charCodeAt(0).toString(16);
    }
    return str;
}

However we have some values that include registered trademark ®.

The function I found returns AE for that, but the value that Sharepoint's internal code is generating has C2AE in the refinement token. Which from this resource I understand is the difference between the hex in UTF-8 and UTF-16.

I'm not sure how to get the same encoding as SP is using. I've poked around and can't find anything internal and any searches which include refinement or refinement token get a bunch of other stuff about how to use them in general.

Does anybody know of an SP native function to encode the refinement token?

I'm also worried that there are other characters that might cause problems in the future, but don't know about yet!

Answer 1

I have a bit of a hack that may point you in the right direction based off of this:

var e = encodeURIComponent("®"); // results in %C2%AE

Which is what you want except the percent signs, so:

function String2Hex(tmp) {
    var str = '';
    for(var i = 0; i < tmp.length; i++) {
        var c = tmp[i].charCodeAt(0);
        if(c === 174 /* ® */) {
            str += encodeURIComponent(tmp[i]).replace(/%/g, '');
        }
        else {
            str += c.toString(16);
        }
    }
    return str;
}

This is of course just your function except that it calls encodeURIComponent for char code 174 (registerd trademark) and strips out the percent signs.

I tried to look around for a pure JavaScript implementation of encodeURIComponent so I could come up with a less hacky solution, but the only ones I found were really old and returned just AE for registered trademark (obviously not helpful).

I also looked at a number of blog posts about encoding refiners, but they all seem to use the exact same logic you were already using (i.e. registered trademark resulted in just AE).

So this is the best I could come up with, but it does work. Of course, if you have any other odd characters in your refiner values, you might need to send more characters the encode URI direction. And no guarantees that encodeURIComponent even does the same thing as SharePoint's internal code for other special characters of course, but it seems likely.

And if you're really feeling adventurous, you could take a look at the ECMAScript Language Specification, section 15.1.3 URI Handling Function Properties, to figure out what encodeURIComponent is actually doing, but that was a bit more than I was willing to take on this evening ;)

Answer 2

I kept searching and found this blog. He talks about how javascript is native UTF-16, but refiner tokens need to be UTF-8 encoded. His answer is to use unescape() which returns UTF-8.

var refToken = 'ǂǂ'+String2Hex(unescape(encodeURIComponent(value)));

This kind of pattern somehow strikes me as "very Microsoft" and seems like the simplest version that would have been available back in the day.