Skip to content

👷(CLDSRV-860) Monitor async await migration#6088

Open
DarkIsDude wants to merge 2 commits intodevelopment/9.3from
feature/CLDSRV-860/monitor-async-await-migration
Open

👷(CLDSRV-860) Monitor async await migration#6088
DarkIsDude wants to merge 2 commits intodevelopment/9.3from
feature/CLDSRV-860/monitor-async-await-migration

Conversation

@DarkIsDude
Copy link
Contributor

Pull request template

Description

Motivation and context

Why is this change required? What problem does it solve?

Related issues

Please use the following link syntaxes #600 to reference issues in the
current repository

Checklist

Add tests to cover the changes

New tests added or existing tests modified to cover all changes

Code conforms with the style guide

Sign your work

In order to contribute to the project, you must sign your work
https://github.com/scality/Guidelines/blob/master/CONTRIBUTING.md#sign-your-work

Thank you again for contributing! We will try to test and integrate the change
as soon as we can.

@DarkIsDude DarkIsDude self-assigned this Feb 25, 2026
@bert-e

This comment was marked as resolved.

@bert-e

This comment was marked as resolved.

@DarkIsDude DarkIsDude changed the base branch from development/9.2 to development/9.3 February 25, 2026 15:48
@bert-e

This comment was marked as resolved.

@codecov

This comment was marked as resolved.


const propertyAccesses = sourceFile.getDescendantsOfKind(SyntaxKind.PropertyAccessExpression);
for (const access of propertyAccesses) {
if (access.getName() !== 'then') continue;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't allow then but we ask them in the migration. What do you think ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we actually need this extra "check"?

  • I guess we would only use then() because we have a callback function... which should be reported already
  • even after the migration, do you thing we may have some cases where it makes sense to use the raw .then() instead of await ?

@DarkIsDude DarkIsDude marked this pull request as ready for review March 10, 2026 16:09
@DarkIsDude DarkIsDude requested review from a team, benzekrimaha and delthas March 10, 2026 16:09
@DarkIsDude DarkIsDude force-pushed the feature/CLDSRV-860/monitor-async-await-migration branch from 93020af to 8981239 Compare March 10, 2026 16:10
import { execSync } from 'node:child_process';
import { Project, SyntaxKind } from 'ts-morph';

const CALLBACK_PARAM_PATTERN = /^(cb|callback|next|done|err)$/i;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

err is a common name for error parameters (e.g. (err, result) => {}, .on("error", (err) => {})), not callbacks. Including it here will cause false positives on error handlers and error-first callback signatures where err is the error, not the continuation.

— Claude Code

run: pip install flake8
- name: Lint Javascript
run: yarn run --silent lint -- --max-warnings 0
run: yarn run --silent lint
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing --max-warnings 0 means the entire eslint run now tolerates unlimited warnings — not just the new promise/prefer-await-to-then and n/callback-return rules but every other warning too. This weakens overall lint enforcement. Consider keeping --max-warnings 0 and either (a) setting a higher threshold, or (b) adding eslint-disable comments to existing violations so new warnings still fail CI.

— Claude Code

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind of agree with Claude here, we don't want to disable all warnings, but only the prefer-await-to-than and callback-return... but we want to get a report on these...

  • could we instead run eslint multiple times? First without these checks but with strict warning; then only on these "migration" checks (and ignoring warnings).
  • another option may be to create a custom eslint reporter, to generate a separate report AND filter out (=ignore) irrelevant warnings ? c.f. [1]
  • maybe we could use a custom reporter to automatically handle the check-diff-async part (i.e. ignore defects not in diff).... and maybe (probably if I am being optimistic...) this is a feature of eslint or an existing plugin that we may leverage?

[1] https://stackoverflow.com/questions/66899779/configure-eslint-to-report-only-plugin-specific-errors

console.log(`Migration: ${asyncFunctions}/${totalFunctions} functions (${migrationPercent}%)`);

if (process.env.GITHUB_STEP_SUMMARY) {
const { writeFileSync, appendFileSync } = await import('node:fs');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeFileSync is imported but never used. Only appendFileSync is needed here.

— Claude Code

const base = process.env.GITHUB_BASE_REF
? `origin/${process.env.GITHUB_BASE_REF}`
: 'HEAD';
const diff = execSync(`git diff ${base} -- ${filePath}`, { encoding: 'utf8' });
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filePath is interpolated directly into a shell command via execSync. A filename with shell metacharacters could break or inject commands. Use execFileSync with an array of arguments instead to avoid shell interpretation. Same applies to line 17.

— Claude Code

- name: Lint Yaml
run: yamllint -c yamllint.yml $(git ls-files "*.yml")
- name: Check async/await compliance in diff
run: yarn run check-diff-async
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lint job checkout (line 82) uses default fetch-depth 1 (shallow clone). check-diff-async.mjs needs the base branch ref which will not exist in a shallow clone. This step will fail on every PR. Add fetch-depth: 0 to the checkout step.

— Claude Code

@claude
Copy link

claude bot commented Mar 10, 2026

  • Lint job checkout uses shallow clone but check-diff-async.mjs needs the base branch ref. Will fail on every PR. Add fetch-depth 0.
  • execSync interpolates filePath into shell commands (lines 17 and 31), risking injection. Use execFileSync with array args.
  • CALLBACK_PARAM_PATTERN includes err, causing false positives on error handlers. Remove err.
  • Removing max-warnings 0 from lint means all eslint warnings are silently ignored in CI.
  • count-async-functions.mjs imports writeFileSync but never uses it (line 77).

Review by Claude Code


if (process.env.GITHUB_STEP_SUMMARY) {
const { writeFileSync, appendFileSync } = await import('node:fs');
appendFileSync(process.env.GITHUB_STEP_SUMMARY, [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really want a summary in every PR?
for monitoring the progress, this may be done in a separate job (eg. nightly on on dev branches) which builds a report giving the status on the "latest branch" (as a badge in the README, as a GitHub page, ....)

run: pip install flake8
- name: Lint Javascript
run: yarn run --silent lint -- --max-warnings 0
run: yarn run --silent lint
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind of agree with Claude here, we don't want to disable all warnings, but only the prefer-await-to-than and callback-return... but we want to get a report on these...

  • could we instead run eslint multiple times? First without these checks but with strict warning; then only on these "migration" checks (and ignoring warnings).
  • another option may be to create a custom eslint reporter, to generate a separate report AND filter out (=ignore) irrelevant warnings ? c.f. [1]
  • maybe we could use a custom reporter to automatically handle the check-diff-async part (i.e. ignore defects not in diff).... and maybe (probably if I am being optimistic...) this is a feature of eslint or an existing plugin that we may leverage?

[1] https://stackoverflow.com/questions/66899779/configure-eslint-to-report-only-plugin-specific-errors


const propertyAccesses = sourceFile.getDescendantsOfKind(SyntaxKind.PropertyAccessExpression);
for (const access of propertyAccesses) {
if (access.getName() !== 'then') continue;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we actually need this extra "check"?

  • I guess we would only use then() because we have a callback function... which should be reported already
  • even after the migration, do you thing we may have some cases where it makes sense to use the raw .then() instead of await ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these scripts not be in the Guidelines repo, so they can be reused?

(and if needed, it may include a github action as well)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have tests for these scripts, to make it easier to test/debug them (even if they are tools).
(this will be easier to implement if they are in a kind of dedicated repo -like Guidelines- and simply used here)

note: for testing against git/diff, maybe we could use https://github.com/kiegroup/mock-github, as we do for testing CI workflows

@@ -0,0 +1,142 @@
/**
* Check that all new/modified functions in the current git diff use async/await.
* Fails with exit code 1 if any additions introduce callback-style functions or .then() chains.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we use https://github.com/tj-actions/eslint-changed-files for this purpose?
seems quite similar...

or run eslint like this: eslint --fix $(git diff --name-only HEAD | xargs) (c.f. https://stackoverflow.com/questions/54511168/enable-eslint-only-for-edited-files)

skipAddingFilesFromTsConfig: true,
});

project.addSourceFilesAtPaths([
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be some parameters of the script (or auto-discovered from package.json/...), to allow reusing this script?


const propertyAccesses = sourceFile.getDescendantsOfKind(SyntaxKind.PropertyAccessExpression);
for (const access of propertyAccesses) {
if (access.getName() === 'then') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above: not sure it makes sense to try to "eradicate" then() (but we can reporting them 😁)

}
}

const migrationPercent = totalFunctions > 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not correct : not all functions will be (ever) be async.

the migration progress is really 1 - callbackFunctions / (callbackFunctions_before_migration)....

as a "kind of proxy", we may compute callbackFunctions / (asyncFunctions + callbackFunctions) ; but not sure it is that useful...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(completion is really just when callbackFunctions == 0)

@bert-e

This comment was marked as resolved.

@bert-e

This comment was marked as resolved.

@bert-e
Copy link
Contributor

bert-e commented Mar 18, 2026

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants