This could be most likely resolved by increase of throttling limit, but...
Quote form Best Practices: Common Coding Issues When Using the SharePoint Object Model
If the list contains more than 2,000 items, you will need to paginate
the list in increments of no more than 2,000 items.
So when working with large lists (over 2000 items) you need to optimize your code for better performance. If you follow suggestions from Best Practices for LARGE SharePoint Lists and Documents Libraries your code should look like this:
DataTable dt = null;
SPList list = web.Lists["Announcements"];
SPQuery query= new SPQuery();
query.Query = "<Where><And><Geq><FieldRef Name=\"ID\" />
<Value Type=\"Counter\"> 1</Value></Geq><Leq><FieldRef Name=\"ID\" />
<Value Type=\"Counter\">500000</Value></Leq></And></Where>";
query.RowLimit = 2000;
do
{
SPListItemCollection myItems = list.GetItems(query);
if(dt == null)
{
dt = myItems.GetDataTable();
}
else
{
dt.Merge(myItems.GetDataTable());
}
query.ListItemCollectionPosition = myItems.ListItemCollectionPosition;
} while (query.ListItemCollectionPosition != null);
(I have tested this on smaller list)
Some more reading material on working with large lists can be found here:
Managing Lists and Libraries with Thousands or Millions of Items
I really wonder why do you need to put so many records in table.
Addition:
Slowest command in above code is myItems.GetDataTable().
To speed things up a bit there is possibility to filter fields (columns) returned by SPQuery. By doing so datatable will contain only desired columns. So after query.Query line put:
query.ViewFields = "<FieldRef Name='AssignedTo' /><FieldRef Name='LinkTitle' />";
Of course this is just example and you need to include 'real' fields you need.
Further more you can 'play' around with RowLimit (increase/decrease) to see if it helps.
