Thanks to mastrdev's solution I was able to change the background color on the all width of the groupe header row except for the left 35 pixels.
But after some text I succeed using this trick :
"class": "ms-bgColor-communicationTint20 ms-bgColor-communicationTint20--hover"
We all know that in reality, as shown by Ganesh Sanap we are not really changing the background color of the groupHeaderContainer itself but instead making the sp-field-customFormatter div child to be as large as its parent and cover it.
But the result is equivalent or so close ;).
The last remaining issue on my side is the hover color effect that I would like to change (or cancel) like I did on the list rows, but sadly the trick of the class ms-bgColor-XYZ--hover with the same color has background doesn't work here.
I think this is because the hover effect is attached to the groupHeaderContainer object and not to the sp-field-customFormatter object.
I hope my enhancement will help someone.