Decouple sync and release actions

[viren-nadkarni]

Background

Moto-Ext current has an action that rebases the codebase with upstream repo and publishes the build to PyPI, introduced in #78.

Changes

Following the hard fork (see #83), Moto-Ext will not be rebased with the upstream repo. This PR therefore reworks this CI pipeline and splits it into two separate workflows:

• Sync moto-ext with upstream: will be able to do the rebasing only. This action will be disabled and exist only for posterity.
• Release moto-ext: will increment the micro semver, create a release and publish to PyPI.

Tests

Successful dry runs for:

• Sync moto-ext with upstream: https://github.com/localstack/moto/actions/runs/21477100477/job/61863753872
• Release moto-ext: https://github.com/localstack/moto/actions/runs/21476950500/job/61863256226

Related

Closes: PNX-558

[alexrashed]

Thanks for jumping on this! Great to see the sync being decoupled to support the future strategy. I only added a few small questions, mostly because I might not fully understand how the release cycles and the upgrade path of LocalStack is going to look like in the future... 😅

[alexrashed]

question: given that we do not have a schedule on the "sync" workflow anymore, does it make sense to have a weekly schedule on this one? Hopefully there won't be any commits in the repo in most weeks? Wouldn't this pipeline fail now if there are no commits fora whole week?
Or are there automated changes similar to #86 created by new automations strictly every single week?

[viren-nadkarni]

The plan is to stick to the same schedule as in Moto for updating the static data. One of the workflows actually scrapes info off the AWS docs website.

Some workflows run weekly, others monthly (see table in PNX-557 for the schedule)

[alexrashed]

Okay, that makes sense. But are there for sure always changes every week? If not, this workflow would fail, right?

[viren-nadkarni]

It won't for sure change every week.

The workflow won't fail if that's the case, it'll make a 'duplicate' release with a bumped version.

[alexrashed]

Ah, okay. That sounds good enough, but is that something that we want though? Would it maybe make sense to just gracefully stop the action (with a successful outcome) if the current commit is already tagged / released?
I think that fact ("duplicate releases") could be a bit confusing, and we wouldn't unnecessarily invalidate caches, pollute PyPi,... WDYT?

[viren-nadkarni]

Interesting you mention this - I was actually considering whether we should change to CalVer because of the planned weekly releases of Moto-Ext but decided against it.

With regards to not releasing if there are not commits, I will add a conditional skip if git log <latest tag>..HEAD is empty. But this won't fully protect against duplicate releases e.g. when changes are made only in .github

[alexrashed]

Interesting you mention this - I was actually considering whether we should change to CalVer because of the planned weekly releases of Moto-Ext but decided against it.

Well, I guess both could be valid, switching to calendar versioning but just not creating releases if there are no changes would also be an option (without saying that I would be in favor of doing that). 😅

With regards to not releasing if there are not commits, I will add a conditional skip if git log ..HEAD is empty. But this won't fully protect against duplicate releases e.g. when changes are made only in .github

Sounds good, and I think this approach would be good enough :)

[viren-nadkarni]

Added in 129b906

[alexrashed]

question: How is this version determined and what do we expect the next few version numbers to look like?

[viren-nadkarni]

The previous action retrieves the latest Git tag. This action provides incremented the semantic versions, and we use the version with patch number bumped. This is because Moto-Ext is effectively feature-frozen, only a specific kind of bugfixes will be allowed.

[alexrashed]

Thanks, that sounds good. I am just still a bit confused how this compares with the upstream versioning. In the other comment you mentioned that there might as well be selective pulls from upstream every now and then. How does the versioning scheme work with that?
Also, just to make sure we are talking about the same thing: When you say "micro" in the PR description, you mean "micro" as in PEP440, i.e. what you are calling "patch" here, right?

I guess my question is basically: What is the difference from the previous versioning to this kind of versioning and how is this going to look like considering the future of this project.

[viren-nadkarni]

In the other comment you mentioned that there might as well be selective pulls from upstream every now and then. How does the versioning scheme work with that?

There won't be any selective pulls. The rebase that happened this week was supposed to be the last, but because Simon and Daniel contributed to Moto upstream this week, we'll make an exception and do another rebase next week. That rebase will be the last rebase and Moto-Ext will effectively be frozen.

Since we've already announced the hard fork, I guess we can start using our own version scheme and not incorporate the minor verion bump that might come with the next week's rebase.

After this PR is merged, the version scheme will be strictly micro bumps.

When you say "micro" in the PR description, you mean "micro" as in PEP440, i.e. what you are calling "patch" here, right?

Yes, basically major.minor.micro. The term 'patch' (same meaning as 'micro') is what we use in our Semantic Versioning spec 😛

[alexrashed]

Okay, thanks for the explanations, that makes sense! I guess we might cherry-pick certain commits in the future (for a while) from upstream, but I understand the next sync is going to be the last. I think my confusion came from the fact that we are keeping the sync workflow, but commit to never ever executing it again 😅

[alexrashed]

question: Does it make sense to even keep this pipeline? You mentioned in the description that there is no plan to sync the fork with upstream anymore? If this is the case, I guess we could just remove this part / this workflow?

[viren-nadkarni]

We seem to need another bump (see #sig-moto) so I'd like to keep it for a while.

I'll make a separate PR in the future which cleans up this and other unused workflows from upstream.

[alexrashed]

Okay, sounds good! As a nitpick on the side I would in this case also opt for renaming this workflow as well 😛

[alexrashed]

nit: Might make sense to rename the workflows a bit. They don't have much to do with Continuous Integration anymore I guess 😅

[alexrashed]

Thanks for clarifying my questions in the comment threads! That helped me understand what the next steps are and how the project and its maintenance will look like in the future. 🚀

[alexrashed]

Thanks for jumping on the comments, but this change would now actually fail the pipeline regularly, which I don't think is the intention.

[alexrashed]

issue: This will now explicitly fail the pipeline if there was no commit.
Usually something like this is implemented by having one job which determines if the actual task should be executed. Something like this:

jobs:
  check-commits:
    runs-on: ubuntu-latest
    outputs:
      has-new-commits: ${{ steps.check.outputs.has-new-commits }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - id: check
        run: |
          LAST_TAG=$(git describe --tags --abbrev=0)
          if git rev-parse "$LAST_TAG"..HEAD >/dev/null 2>&1; then
            echo "has-new-commits=true" >> $GITHUB_OUTPUT
          else
            echo "has-new-commits=false" >> $GITHUB_OUTPUT
          fi

And then have the other job depend on it:

  release:
    runs-on: ubuntu-latest
    needs: check-commits
    if: needs.check-commits.outputs.has-new-commits == 'true'
    steps:
      ...