Unable to use a SPN for auth running tests

[storey247]

Azure Databricks allows for authentication via SPN as well as PAT, but currently Nutter only supports the PAT token auth approach.

This recently blocked us using Nutter on an engagement with a customer because they wouldn't allow us to generate PAT tokens and insisted that we auth via SPN.

It would be really great if the Nutter cli could be configured use the standard SPN auth flow of the current user, instead of only allowing PAT

[giventocode]

Thanks for the feedback.

[ChrisDelaforce]

This is several years late, however, I still encountered the same issue with no obvious solutions posted online, so here is my solve using a Managed Identity (plus a client secret generated in Databricks).

`variables:

• group: 'DatabricksAccess' # vars including databricksDevWorkspaceURL, azureClientID, sharedClusterID, and azureClientSecret

trigger: none

stages:

• stage: onPush
  jobs:

  • job: onPushJob
    timeoutInMinutes: 10
    cancelTimeoutInMinutes: 2
    pool:
    vmImage: ubuntu-latest

    steps:

    • script: |
      curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh
      displayName: 'Install Databricks CLI'

    • script: |
      python -m pip install pip nutter
      python -m pip install --upgrade databricks-cli
      displayName: 'Install Dependencies'

    • script: | # M2M OAuth method from: https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m
      curl --request POST
      --url $(databricksDevWorkspaceURL)/oidc/v1/token
      --user "$(azureClientID):$(azureClientSecret)"
      --data 'grant_type=client_credentials&scope=all-apis' > auth.json

      jq .token_type auth.json # check result as expected

      displayName: Get OAuth Token

    • script: |
      export DATABRICKS_TOKEN=$(jq .access_token auth.json | sed 's/"//g') # export token minus qoute marks inside step
      nutter run /Workspace/Repos/user@email/repo_name/tests/unit-databricks/target_test_notebook --cluster_id $(sharedClusterID) --junit_report --timeout 500
      env:
      DATABRICKS_HOST: $(databricksDevWorkspaceURL)
      displayName: 'Run Nutter Tests

    • task: PublishTestResults@2
      inputs:
      testResultsFormat: 'JUnit'
      testResultsFiles: '**/test-*.xml'
      failTaskOnFailedTests: true
      `

I utilised the above manual OAuth method found in the link. That generates a json result that I save. I access the token with jq and most importantly I remove the " quote marks from the string using sed. If you don't remove them, it doesn't work, very annoying.

Then you can run nutter using the OAuth token.

This took me forever to figure out, but once it worked there was much dancing :D