When you use a calculated column to format a date the field becomes text and is then sorted alphabetically. What you want is to keep the field as a Date so that it is sortable (and filterable) in the way you'd expect but then apply formatting to get the look you want. Here is a format you can apply to the view (View Menu > Format this View > Advanced):
{
"$schema": "https://developer.microsoft.com/json-schemas/sp/v2/row-formatting.schema.json",
"groupProps": {
"headerFormatter":{
"elmType": "span",
"txtContent": "=if(getMonth(@group.fieldData)==0,'Jan ',if(getMonth(@group.fieldData)==1,'Feb ',if(getMonth(@group.fieldData)==2,'Mar ',if(getMonth(@group.fieldData)==3,'Apr ',if(getMonth(@group.fieldData)==4,'May ',if(getMonth(@group.fieldData)==5,'Jun ',if(getMonth(@group.fieldData)==6,'Jul ',if(getMonth(@group.fieldData)==7,'Aug ',if(getMonth(@group.fieldData)==8,'Sept ',if(getMonth(@group.fieldData)==9,'Oct ',if(getMonth(@group.fieldData)==10,'Nov ','Dec '))))))))))) + getYear(@group.fieldData) + ' (' + @group.count + ')'"
}
}
}
Here's what it looks like (grouped by Due):
