First of all, sorry for the lack of updates, I have been busy working on several other projects.
I wanted to share my progression and what are the issues I am facing now.
I was able to "translate" the C# to PowerShell:
$xmlDocument = Get-Content -Path "C:\Users\<censored>\desktop\table_matieres.webpart" -Raw
$wpManager = $page.GetLimitedWebPartManager([System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)
$importWP = $wpManager.ImportWebPart($xmlDocument)
$wp = $wpManager.AddWebPart($importWP.WebPart,"wpz",0)
$context.Load($wp)
$context.ExecuteQuery()
$props = $page.ListItemAllFields
$context.load($props)
$props["WikiField"] = "<div class=""ms-rtestate-read ms-rte-wpbox""><div class=""ms-rtestate-notify ms-rtegenerate-notify ms-rtestate-read " + $wp.Id + """ id=""div_"+ $wp.Id + """></div><div id=""vid_"+ $wp.Id + """ style=""display:block""></div></div>"
$page.Update()
$context.ExecuteQuery()
Although, now, the behavior is different and it does add the code to the ASPX page.
It seems to append the code even if there was already a wiki zone on the page (taken from SharePoint Designer):
<SharePoint:EmbeddedFormField id="WikiField" FieldName="WikiField" ControlMode="Display" runat="server"><div class="ExternalClassA538D3FA9C374CD6B8B7E2F117F2C08D">
<table id="layoutsTable" style="width:100%;">
<tbody>
<tr style="vertical-align:top;">
<td colspan="2"><div class="ms-rte-layoutszone-outer" style="width:100%;"><div class="ms-rte-layoutszone-inner" style="word-wrap:break-word;margin:0px;border:0px;"><div class="ms-rtestate-read ms-rte-wpbox" ><div class="ms-rtestate-read 1aaac0d0-d055-4652-8be9-8880d2f16583" id="div_1aaac0d0-d055-4652-8be9-8880d2f16583"></div>
<div class="ms-rtestate-read" id="vid_1aaac0d0-d055-4652-8be9-8880d2f16583" style="display:none;"></div>
</div>
</div></div></td>
</tr>
<tr style="vertical-align:top;">
<td style="width:49.95%;"><div class="ms-rte-layoutszone-outer" style="width:100%;"><div class="ms-rte-layoutszone-inner" style="word-wrap:break-word;margin:0px;border:0px;">
<WpNs1:SiteFeedWebPart runat="server" __MarkupType="xmlmarkup" WebPart="true" __WebPartId="{0E290259-6B6D-4524-ACBC-4EBD6C93A6F3}" >
<WebPart xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/WebPart/v2">
<Title>Flux de site</Title>
<FrameType>None</FrameType>
<Description>Le flux de site contient des conversations de microblog sur un site de groupe.</Description>
<IsIncluded>true</IsIncluded>
<ZoneID>wpz</ZoneID>
<PartOrder>1</PartOrder>
<FrameState>Normal</FrameState>
<Height />
<Width />
<AllowRemove>true</AllowRemove>
<AllowZoneChange>true</AllowZoneChange>
<AllowMinimize>true</AllowMinimize>
<AllowConnect>true</AllowConnect>
<AllowEdit>true</AllowEdit>
<AllowHide>true</AllowHide>
<IsVisible>true</IsVisible>
<DetailLink />
<HelpLink />
<HelpMode>Modeless</HelpMode>
<Dir>Default</Dir>
<PartImageSmall />
<MissingAssembly>Impossible d'importer ce composant WebPart.</MissingAssembly>
<PartImageLarge />
<IsIncludedFilter />
<ExportControlledProperties>true</ExportControlledProperties>
<ConnectionID>00000000-0000-0000-0000-000000000000</ConnectionID>
<ID>g_a2fc68fc_4cad_4acf_bca1_0d25a47e533f</ID>
</WebPart>
</WpNs1:SiteFeedWebPart>
</div></div></td>
<td class="ms-wiki-columnSpacing" style="width:49.95%;"><div class="ms-rte-layoutszone-outer" style="width:100%;"><div class="ms-rte-layoutszone-inner" style="word-wrap:break-word;margin:0px;border:0px;"><div class="ms-rtestate-read ms-rte-wpbox" ><div class="ms-rtestate-read bf7102be-ad8c-4a20-83cb-5421f84c6e65" id="div_bf7102be-ad8c-4a20-83cb-5421f84c6e65"></div>
<div class="ms-rtestate-read" id="vid_bf7102be-ad8c-4a20-83cb-5421f84c6e65" style="display:none;"></div>
</div>
</div></div></td>
</tr>
</tbody>
</table>
<span id="layoutsData" style="display:none;">true,false,2</span></div></SharePoint:EmbeddedFormField>
Note: Check after:
</WpNs1:SiteFeedWebPart>
This results in the webpart being in an "invisible" zone.
The goal being to use a layout such as this:

I think it's called "2 columns and a header" in English.
I wanted to add a table of content webpart in the "top section", keep the "Site RSS feed" on the bottom left and have the bottom right empty.
Would there be a way to modify the ASPX page to achieve this goal please?
Thank you for your time and help again.