My question is this.
Is there an upper limit of items (20000?) that the total amount of items in all the lists in sharepoint online are aggregated against? So that when the total in all lists reach 20k you can't add more items?
There is an upper limit on list items in SharePoint Online but it is much higher than 20,000 items. Here is a 21,100 item list I created earlier on SharePoint Online:
I haven't proven it, but I believe the upper limits for items in lists in SharePoint Online are the same as for SharePoint 2013 described here (30,000,000 items)
You may be getting confused with the list view threshold limit. This is set at 5,000 items for lists in your team site and 20,000 for your OneDrive for Business document library. The list view threshold limit is not a hard limit; you can have more items than that (as explained above) but certain operations and tools may stop working, for example sorting on an non-indexed column or using the OneDrive for Business sync client tool.
I have a list with 90,000+ items on SPO but can filter it only by ID. Trying to display a view with over 5000 items of course does not work, but I have found that I can create new indexed columns and filter with them on list as big as 10,000 items, anything over 30,000 does not allow additional indexes or filtering. I have also seen bug where working views of lists with 5000+ items fail when you attempt to do on page sorting or filtering. At our request, MS recently fixed that bug.
I've heard that MS might be trying completely address the limit on SPO. I would be fine with a 5000 item limit on Views so long as we could create indexes and filter much bigger lists.
To appreciate this limit you should know SharePoint stores all list data of a site collection in a single SQL Table that has shared generic column names like Date7.