SPFile.OpenBinary - Encoding Issues

Question

I am copying some SharePoint pages from one site collection to another and I noticed that certain special characters get mangled up in the destination file. Here is an example:

Source Text:

Nous dialoguons avec les entreprises pour générer des résultats positifs pour nos clients et la collectivité.

Destination Text:

Nous dialoguons avec les entreprises pour générer des résultats positifs pour nos clients et la collectivité.

So far, I've narrowed down the problem to this portion of my code:

var bytes = sourceFile.OpenBinary();
var targetFile = targetFiles.Add(sourceFile.Name, bytes, new SPFileCollectionAddParameters()
{
    Overwrite = true,
    CheckInComment = "Migrated from " + sourceFile.ServerRelativeUrl,
    CheckRequiredFields = false,
});

I have considered converting the bytes to UTF-8 first before adding the new file:

var sourceBytes = sourceFile.OpenBinary();
byte[] convertedBytes = Encoding.Convert(Encoding.Default, Encoding.UTF8, sourceBytes);
var targetFile = targetFolder.Files.Add(sourceFile.Name, convertedBytes, new SPFileCollectionAddParameters()
{
    Overwrite = true,
    CheckInComment = "Migrated from " + sourceFile.ServerRelativeUrl,
    CheckRequiredFields = false,
});

But this results in further mangling of additional characters:

Nous dialoguons avec les entreprises pour générer des résultats positifs pour nos clients et la collectivité.

And if I try:

byte[] convertedBytes = Encoding.Convert(Encoding.ASCII, Encoding.UTF8, sourceBytes);

I get this:

Nous dialoguons avec les entreprises pour g??n??rer des r??sultats positifs pour nos clients et la collectivit??.

It looks like I definitely need to convert the bytes from one encoding to another, but I just can't figure out the direction.

Please help.

UPDATE: I have

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

in the head section of my master page.

Hope that helps.

Answer 1

Not sure exactly why this works, but I solved this problem by using this code:

byte[] convertedBytes = Encoding.Convert(Encoding.UTF8, Encoding.UTF32, sourceBytes);

Hope that helps someone.