It looks like it's not possible without manual filtering.
All the entries are stored in the site collection database in the table dbo.AuditData so GetEntries is just a method to get the data from the table.
Let's look at the table structure:
[dbo].[AuditData](
[SiteId] [uniqueidentifier] NOT NULL,
[ItemId] [uniqueidentifier] NOT NULL,
[ItemType] [smallint] NOT NULL,
[UserId] [int] NULL,
[MachineName] [nvarchar](128) NULL,
[MachineIp] [nvarchar](20) NULL,
[DocLocation] [nvarchar](260) NULL,
[LocationType] [tinyint] NULL,
[Occurred] [datetime] NOT NULL,
[Event] [int] NOT NULL,
[EventName] [nvarchar](128) NULL,
[EventSource] [tinyint] NOT NULL,
[SourceName] [nvarchar](256) NULL,
[EventData] [nvarchar](max) NULL
)
As you can see, there is no WebId or something like that column, so it means that there is no way to filter entries by that attribute.
As I understand there is "Web" value in ItemType column, but that is only for audit event directly related to web instead of events on the web.
To sum up, there is no options to restrict SPAuditQuery by the way you need.