Conversation
| return wm.min, wm.max, true | ||
| } | ||
|
|
||
| mn, mx, err := e.fetchTimestamps(ctx, rollup.Database, rollup.DatabaseSchema, rollup.Table) |
There was a problem hiding this comment.
Instead of directly querying for watermarks here, I explored using metrics_time_range resolver approach so that we can rely on user defined cache_key_ttl and cache_key_sql to not fetch watermark until needed. It also helps with automatically invalidating cache if rollups are Rill managed as the resolver cache key will rely on metrics view status updated on.
However, the only issue I see is for external olap (majority of cases), mv cache is disabled by default so resolver ends up querying watermarks for all eligible rollups for every single query (contrast with current behaviour where time range is only queried once for time picker). In this case I was thinking of an L1 cache having simple time based ttl of lets say 1 or 5 minutes in this file and only if that expires then call metrics_time_range resolver. I already have the changes locally if needed. Thoughts?
Checklist: