Unique tag sets for inherited logs
My initial idea for tags for InheritedLogs was two have two sets of tags (the original event.tags
set from the EventLog, and an additional inheritedlog.superevent.tags
set) that are combined into one set that is queryable and filterable. Which is easy enough.
The problem arises from combining ManyToMany
sets:
Because ManyToManyField attributes and reverse relations can have multiple related rows, including these can have a multiplier effect on the size of your result set. This will be especially pronounced if you include multiple such fields in your values() query, in which case all possible combinations will be returned.
So in practice, on gracedb-dev1
for a test InheritedLog:
In [23]: il
Out[23]: <InheritedLog: G414269 -> S230213b>
In [24]: il.source_event_log.tags.all()
Out[24]: <QuerySet [<Tag: Sky Localization>]>
In [25]: il.superevent_tags.all()
Out[25]: <QuerySet [<Tag: Public>]>
In [26]: combined_set = il.source_event_log.tags.all() | il.superevent_tags.all()
In [27]: combined_set.count()
Out[27]: 616735
In [28]: %time combined_set.count()
CPU times: user 0 ns, sys: 2.11 ms, total: 2.11 ms
Wall time: 1.35 s
Out[28]: 616735
By combining the querysets, the database is constructing a set of all the possible combinations of those tags, which is 600,000+ on dev1's small set of events and superevents, and it still takes over a second of wall time to count or construct a .distinct()
set. I suspect on playground's massive database, it would absolutely destroy querying and rendering the superevent page.
I also tried the *.union()
method to combine the tag sets, which is nearly instantaneous, but it kills the ability to *.filter()
, or *.get()
tags in the set... so that's a dealbreaker for querying and rendering the view.
So, I'm going to give up on adding new tags to InheritedLogs from the superevent (InheritedLog.supervent_tags
) and ONLY have it inherit EventLog tags. We can revisit this if it becomes a dealbreaker, but it at least looks like GWCelery is adding the public
tag to the EventLog anyway, so it might all just work out.
The public
tag doesn't do anything on a g-event page, but I... think.... it might just work for exposing a superevent inherited log to the public.