Scriptworker Readme

Scriptworker implements the TaskCluster worker model, then launches a pre-defined script.

This worker was designed for Releng processes that need specific, limited, and pre-defined capabilities.

Free software: MPL2 License

Usage

• Create a config file. By default scriptworker will look in ./scriptworker.yaml, but this config path can be specified as the first and only commandline argument. There is an example config file, and all config items are specified in scriptworker.constants.DEFAULT_CONFIG.

Credentials can live in ./scriptworker.yaml, ./secrets.json, ~/.scriptworker.

• Launch: scriptworker [config_path]

Testing

Without integration tests install tox, then

NO_CREDENTIALS_TESTS=1 tox -e py36

Without any tests connecting to the net, then NO_TESTS_OVER_WIRE=1 tox -e py36

With integration tests, first create a client in the Taskcluster UI with the scopes:

queue:cancel-task:test-dummy-scheduler/*
queue:claim-work:test-dummy-provisioner/dummy-worker-*
queue:create-task:lowest:test-dummy-provisioner/dummy-worker-*
queue:define-task:test-dummy-provisioner/dummy-worker-*
queue:get-artifact:SampleArtifacts/_/X.txt
queue:scheduler-id:test-dummy-scheduler
queue:schedule-task:test-dummy-scheduler/*
queue:task-group-id:test-dummy-scheduler/*
queue:worker-id:test-dummy-workers/dummy-worker-*

Then generate a no priviledge personal access token in Github for the scriptworker_github_token (to avoid rate limiting) and create a ./secrets.json or ~/.scriptworker that looks like:

{
    "integration_credentials": {
        "clientId": "...",
        "accessToken": "...",
    }
    "scriptworker_github_token": "..."
}

then to run all tests: tox

Table of Contents

• Maintenance
  • New docker shas
  • Chain of Trust settings
  • Ed25519 keys
• Scriptworker Releases
  • Code changes
  • Tests and test coverage
  • Versioning
  • Changelog
  • Release files
    • Versioning
  • Tagging
  • Pypi
  • Rollout
• Adding new scriptworker instance types
  • Is scriptworker the right tool?
    • Chain of Trust considerations
  • Creating a new scriptworker instance type
    • Write a script
  • Deployment considerations
    • Graph
    • GCP
• Adding new scriptworker instances of an existing type
• Chain of Trust
  • Overview
    • High level view
  • Chain of Trust Key Management
    • verifying new ed25519 keys
    • Rotating the FirefoxCI CoT keys
  • Chain of Trust Artifact Generation
    • Embedded ed25519 keys
    • Chain of Trust artifacts
  • Chain of Trust Verification
    • Decision Task
    • Ed25519 key management
    • Building the chain
    • Verifying the chain
  • Chain of Trust Testing / debugging
    • Create the virtualenv
    • Set up the test env
    • Find a task to test
    • Run the test
• Scriptworker Readme
  • Usage
  • Testing
• scriptworker package
  • Submodules
  • scriptworker.artifacts module
  • scriptworker.client module
  • scriptworker.config module
  • scriptworker.constants module
  • scriptworker.context module
  • scriptworker.cot.generate module
  • scriptworker.cot.verify module
  • scriptworker.ed25519 module
  • scriptworker.github module
  • scriptworker.exceptions module
  • scriptworker.log module
  • scriptworker.task module
  • scriptworker.utils module
  • scriptworker.worker module
  • Module contents

Indices and tables

• Index
• Module Index
• Search Page