Skip to content

Fix slow /api/v1/jobs endpoint by optimizing RunDao queries #3091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merobi-hub merged 2 commits into from
Apr 12, 2026
+22 −4
Merged

Fix slow /api/v1/jobs endpoint by optimizing RunDao queries #3091

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c9062b2
Fix slow /api/v1/jobs endpoint by optimizing RunDao queries (#2987)
jni-bot Feb 12, 2026
e8c2898
Merge branch 'main' into bug/fix-slow-findByLatestJob-query
merobi-hub Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api/src/main/java/marquez/db/JobDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -282,8 +282,8 @@ default List<Job> findAllWithRun(
.peek(
j -> {
List<Run> runs =
runDao.findByLatestJob(
j.getNamespace().getValue(), j.getName().getValue(), 10, 0);
runDao.findCurrentRunByJob(
j.getNamespace().getValue(), j.getName().getValue(), 1, 0);
this.setJobData(runs, j);
})
.toList();
Expand Down
22 changes: 20 additions & 2 deletions api/src/main/java/marquez/db/RunDao.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -511,13 +511,31 @@ default RunRow upsertRunMeta(
BASE_FIND_RUN_SQL
+ """
WHERE r.uuid IN (
SELECT r.uuid FROM runs_view r
INNER JOIN jobs_view j ON j.namespace_name=r.namespace_name AND j.name=r.job_name
SELECT j.current_run_uuid FROM jobs_view j
WHERE j.namespace_name=:namespace AND (j.name=:jobName OR j.name=ANY(j.aliases))
)
ORDER BY transitioned_at DESC, started_at DESC
LIMIT :limit OFFSET :offset
""")
List<Run> findCurrentRunByJob(String namespace, String jobName, int limit, int offset);

@SqlQuery(
BASE_FIND_RUN_SQL
+ """
WHERE r.job_uuid IN (
SELECT j.uuid FROM jobs j
WHERE j.uuid IN (
SELECT jv.uuid FROM jobs_view jv
WHERE jv.namespace_name=:namespace AND (jv.name=:jobName OR jv.name=ANY(jv.aliases))
)
OR j.symlink_target_uuid IN (
SELECT jv.uuid FROM jobs_view jv
WHERE jv.namespace_name=:namespace AND (jv.name=:jobName OR jv.name=ANY(jv.aliases))
)
)
ORDER BY transitioned_at DESC, started_at DESC
LIMIT :limit OFFSET :offset
""")
List<Run> findByLatestJob(String namespace, String jobName, int limit, int offset);

@Builder
Expand Down
Loading