Skip to content

feat(ci): connect repo to Community-TC#6

Draft
matt-boris wants to merge 1 commit intomainfrom
matt-boris/connectToTC
Draft

feat(ci): connect repo to Community-TC#6
matt-boris wants to merge 1 commit intomainfrom
matt-boris/connectToTC

Conversation

@matt-boris
Copy link
Contributor

@matt-boris matt-boris commented Aug 3, 2022

Fixes #3

This also upgrades to the latest version, v44.23.4.

@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch 3 times, most recently from 9205b53 to dc0af9b Compare August 3, 2022 15:38
@taskcluster taskcluster deleted a comment from community-tc-integration bot Aug 3, 2022
@taskcluster taskcluster deleted a comment from community-tc-integration bot Aug 3, 2022
@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch from dc0af9b to 216553d Compare August 3, 2022 15:41
@matt-boris matt-boris self-assigned this Aug 3, 2022
@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch 5 times, most recently from 6247148 to e247f90 Compare August 3, 2022 17:45
@matt-boris matt-boris closed this Aug 3, 2022
@matt-boris matt-boris reopened this Aug 3, 2022
@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch 16 times, most recently from 7233641 to 298a1ac Compare August 4, 2022 14:39
.taskcluster.yml Outdated
- '--login'
- '-c'
- >-
apt-get update && apt-get install -y build-essential procps file sudo &&
Copy link
Member

@petemoore petemoore Aug 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we build a docker image with the required toolchains, and then run the task in a container using that docker image, to reduce latency and CI overhead, rather than every task needing to perform the toolchain installations themselves.

See e.g. https://firefox-source-docs.mozilla.org/main/69.0/taskcluster/taskcluster/docker-images.html

I'm not sure if we have examples in GitHub of docker image builds based on this workflow, but I think it is a feature of taskgraph - @ahal might know?

In other words, if we use taskgraph, I think it should be able to take care of automatically building and indexing docker images used by CI tasks, so the repo just needs to contain the Dockerfile (e.g. which does all the apt-get installs) and when that is updated, taskgraph will take care of building the updated Docker images, updating the Index, and tasks that use it will automatically get the updated version. Then each task doesn't need to install toolchains etc, they are already part of the docker image.

Aside: we should probably introduce this workflow in the monorepo too. We don't install toolchains on each task run, which is good, but we don't have automated flows for rebuilding docker images when the Dockerfile changes. Instead, we manually republish docker images to dockerhub. We would avoid this if we got taskgraph to do it for us, and the images would be task artifacts, rather than images on hub.docker.com, which would also mean we have one less third party service we would depend on for our CI workflow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've heard there are plans to get rid of docker worker.. but that will only happen when generic worker will support docker-worker-like payloads, so the images would still be used.
Btw, in case of using generic-worker, you'd still need to bootstrap and install everything manually right? Or you would need to re-use some pre-built stuff from somewhere?

@ahal
Copy link

ahal commented Aug 8, 2022

I'm not sure if we have examples in GitHub of docker image builds based on this workflow, but I think it is a feature of taskgraph - @ahal might know?

Correct, I even added docs for this recently :)
https://taskcluster-taskgraph.readthedocs.io/en/latest/howto/docker.html#using-in-project-images

@ahal
Copy link

ahal commented Aug 8, 2022

Also re: generic-worker vs docker-worker.. I think Taskgraph will likely abstract over that when the switch happens, so that Taskgraph consumers won't have to deal with it.

@matt-boris matt-boris marked this pull request as draft August 16, 2022 17:36
@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch from 55059f2 to 7f7e57d Compare August 26, 2022 14:45
@matt-boris matt-boris force-pushed the matt-boris/connectToTC branch 18 times, most recently from b4d7f49 to d03c289 Compare November 16, 2022 16:24
@community-tc-integration
Copy link

Uh oh! Looks like an error! Details

Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

  • method: rerunTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2022-11-16T18:18:52.730Z

@community-tc-integration
Copy link

Uh oh! Looks like an error! Details

Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

  • method: rerunTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2022-11-16T18:20:17.494Z

@community-tc-integration
Copy link

Uh oh! Looks like an error! Details

Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

  • method: rerunTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2022-11-16T18:21:13.186Z

@community-tc-integration
Copy link

Uh oh! Looks like an error! Details

Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q/ZEvStHXnQwSmGbhj8pWh2Q",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/ZEvStHXnQwSmGbhj8pWh2Q"
      ]
    }
  ]
}

  • method: rerunTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2022-11-16T18:22:23.183Z

@community-tc-integration
Copy link

Uh oh! Looks like an error! Details

Client ID static/taskcluster/github does not have sufficient scopes and is missing the following scopes:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/VnQV-biqRNi4plDrkcl6sA/VnQV-biqRNi4plDrkcl6sA",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/VnQV-biqRNi4plDrkcl6sA"
      ]
    }
  ]
}

This request requires the client to satisfy the following scope expression:

{
  "AnyOf": [
    "queue:rerun-task:homebrew-level-1/VnQV-biqRNi4plDrkcl6sA/VnQV-biqRNi4plDrkcl6sA",
    "queue:rerun-task-in-project:none",
    {
      "AllOf": [
        "queue:rerun-task",
        "assume:scheduler-id:homebrew-level-1/VnQV-biqRNi4plDrkcl6sA"
      ]
    }
  ]
}

  • method: rerunTask
  • errorCode: InsufficientScopes
  • statusCode: 403
  • time: 2022-11-21T15:14:34.813Z

@matt-boris matt-boris mentioned this pull request Dec 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

Connect to Community-TC

4 participants