I'm trying to build some custom links from search results to a content search webpart page that also uses refiners. I examined the hash added to the url when using the refiner from the CSWP page and have been able to reconstruct it in my search item template.

I've found that the refinement token used in the url hash for the refiners are hex encoded text. And I found this code to transform strings to hex and it mostly works:

function String2Hex(tmp) {
    var str = '';
    for(var i = 0; i < tmp.length; i++) {
        str += tmp[i].charCodeAt(0).toString(16);
    return str;

However we have some values that include registered trademark ®.

The function I found returns AE for that, but the value that Sharepoint's internal code is generating has C2AE in the refinement token. Which from this resource I understand is the difference between the hex in UTF-8 and UTF-16.

I'm not sure how to get the same encoding as SP is using. I've poked around and can't find anything internal and any searches which include refinement or refinement token get a bunch of other stuff about how to use them in general.

Does anybody know of an SP native function to encode the refinement token?

I'm also worried that there are other characters that might cause problems in the future, but don't know about yet!


I kept searching and found this blog. He talks about how javascript is native UTF-16, but refiner tokens need to be UTF-8 encoded. His answer is to use unescape() which returns UTF-8.

var refToken = 'ǂǂ'+String2Hex(unescape(encodeURIComponent(value)));

This kind of pattern somehow strikes me as "very Microsoft" and seems like the simplest version that would have been available back in the day.

  • 1
    Wow, that's strange. unescape is basically a deprecated form of decodeURIComponent. encodeURIComponent("®") equals "%C2%AE". unescape("%C2%AE") equals "®", which basically means it's treating UTF-16 like it's UTF-8 (also probably why it's deprecated). Then String2Hex("®") gives you "c2ae". So you're basically taking advantage of a deprecated (i.e. broken by modern standards) decode function. Still more efficient than what I was doing, wonder if I can find a pure javascript version of unescape? – Joe McShea Feb 20 '19 at 16:57
  • A very nice description of how this works can be found here: monsur.hossa.in/2012/07/20/utf-8-in-javascript.html – Joe McShea Feb 20 '19 at 17:22

I have a bit of a hack that may point you in the right direction based off of this:

var e = encodeURIComponent("®"); // results in %C2%AE

Which is what you want except the percent signs, so:

function String2Hex(tmp) {
    var str = '';
    for(var i = 0; i < tmp.length; i++) {
        var c = tmp[i].charCodeAt(0);
        if(c === 174 /* ® */) {
            str += encodeURIComponent(tmp[i]).replace(/%/g, '');
        else {
            str += c.toString(16);
    return str;

This is of course just your function except that it calls encodeURIComponent for char code 174 (registerd trademark) and strips out the percent signs.

I tried to look around for a pure JavaScript implementation of encodeURIComponent so I could come up with a less hacky solution, but the only ones I found were really old and returned just AE for registered trademark (obviously not helpful).

I also looked at a number of blog posts about encoding refiners, but they all seem to use the exact same logic you were already using (i.e. registered trademark resulted in just AE).

So this is the best I could come up with, but it does work. Of course, if you have any other odd characters in your refiner values, you might need to send more characters the encode URI direction. And no guarantees that encodeURIComponent even does the same thing as SharePoint's internal code for other special characters of course, but it seems likely.

And if you're really feeling adventurous, you could take a look at the ECMAScript Language Specification, section 15.1.3 URI Handling Function Properties, to figure out what encodeURIComponent is actually doing, but that was a bit more than I was willing to take on this evening ;)

  • I'm not sure I understand how I would use this. stringEncodeUTF16("®") returns a Uint16Array. I need it to return the string "C2AE". It seems unlikely that internally SP is using typed arrays given that SP 2013 was originally developed to run in version of IE that wouldn't have supported them. I'd like to find something that is closer to whatever method SP would be using. – Rothrock Feb 19 '19 at 18:28
  • Sorry @Rothrock, you're absolutely right of course, I seem to have posted the answer to a different question here by mistake. Anyway, I've updated my answer to address your question, but it's a bit of a hack. – Joe McShea Feb 20 '19 at 2:00
  • Thank you for reworking that. I think I found what is most likely the way SP is doing it. – Rothrock Feb 20 '19 at 15:50
  • You should post it here, and mark it as the answer. I'm sure others (myself included) will be interested, and it's certainly not easy to find. – Joe McShea Feb 20 '19 at 15:52
  • I did. It is fairly underwhelming, but I feel like is the most likely solution. – Rothrock Feb 20 '19 at 16:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.