could be a couple of things!
as your using xslt it could be somthing simple ;)
Simply change the   to   for the XSLT code will solve this
problem.
Unable to display this Web Part. To troubleshoot the problem, open this Web page ... & Error: Reference to undefined entity 'nbsp'
It is indeed a little weird.
Generally, this error happens when restoring content in SharePoint.
Please check the ListID value following steps below:
Go to site actions > site settings > modify all site settings > site administration > site libraries and lists
Right-click your list and select copy shortcut and paste it into notepad N1. Then you can get the ListID from the URL like:
http://sevenma:2009/sites/Intranet/_layouts/ListEdit.aspx?List={4C101CF3-FD7E-4BBB-8F28-B0770AFD6534}
Then go to your page in the SharePoint designer in split mode. And select the DFWP in the Design mode part and copy the code in
the Code mode part into notepad N2. Then search ListID in N2 and you
can get the list id value from DefaultValue property.
Then compare them. If they are different, replace all the ListID value with the current one in the code part in N2. Then copy it
back your page. If they are same, please provide information about
your keyword filed and other tests. Also, you can try to save it and
open it in your site and then check the event logs and uls logs to get
more information.
http://social.msdn.microsoft.com/Forums/sharepoint/en-US/f859069d-3b10-4eec-b633-dcf0dc90513e/grouping-in-dfwp-causes-unable-to-display-this-web-part-in-browser
you can also get this message when there is too much data to return resulting in that error message! you would need to set some filtering on the result when querying!
Groupby programmatically
SPList plList=oWeb.Lists["MyList"];
// create new default view that is grouped
string viewQuery = @"
<GroupBy Collapse=""FALSE"" GroupLimit=""100"">
<FieldRef Name=""Year"" Ascending=""FALSE"" />
</GroupBy>
<OrderBy>
<FieldRef Name=""Year"" Ascending=""FALSE"" />
<FieldRef Name=""Period"" Ascending=""FALSE"" />
</OrderBy>";
StringCollection viewFields = new StringCollection();
viewFields.Add("DocIcon");
viewFields.Add("LinkFilename");
viewFields.Add("Period");
viewFields.Add("Created");
plList.Views.Add("Grouped By Year", viewFields, viewQuery, 100, true, true, SPViewCollection.SPViewType.Html, false);
http://moonistahir.wordpress.com/2009/07/27/create-spview-programmatically-with-group-by-option/
another method inline with your way using sharepoint designer!
Creating an XSLT DataView for a sharepoint list
http://www.sharepoint-tips.com/2007/05/sharepoint-designer-article-2-creating.html
EDIT
if your talking about "Groupby programmatically" means you will have to do a custom webpart that will have that code within! its not hard atall! if that is too much than you can do this way!
http://office.microsoft.com/en-gb/sharepoint-server-help/connect-a-filter-web-part-to-a-list-view-web-part-HA101785233.aspx
all else fails than it sounds like a bug! for me i would cutom code it as it would work but for you it should work! I did find this tho and its seems like the same issue and that you would need to contact microsoft for the fix, its an exe file and needs to come from them only!
http://englando.wordpress.com/2012/01/05/unable-to-display-this-web-part-xsltlistview-and-xsltdataview-web-part-issues-in-ie/
hope it solves your buggy issue!