Scriptworker Readme

Scriptworker implements the TaskCluster worker model, then launches a pre-defined script.

This worker was designed for Releng processes that need specific, limited, and pre-defined capabilities.

Free software: MPL2 license

Usage

• Create a config file. By default scriptworker will look in ./scriptworker.yaml, but this config path can be specified as the first and only commandline argument. There is an example config file, and all config items are specified in scriptworker.constants.DEFAULT_CONFIG.

Credentials can live in ./scriptworker.yaml, ./secrets.json, ~/.scriptworker, or in environment variables: TASKCLUSTER_ACCESS_TOKEN, TASKCLUSTER_CLIENT_ID, and TASKCLUSTER_CERTIFICATE.

• Launch: scriptworker [config_path]

Building a docker image

First, create a secrets.json. For integration testing, you’ll need to define the integration_credentials; to do any other authenticated work, you’ll need to define credentials.

Then:

PY_DOT_VERSION=3.7  # or 3.6
docker build -t scriptworker-test-$PY_DOT_VERSION --build-arg PY_DOT_VERSION=$PY_DOT_VERSION  --file docker/Dockerfile.test .

Testing

Note: GPG tests require gpg 2.0.x!

Without integration tests, install tox, then

NO_TESTS_OVER_WIRE=1 tox -e py36

With integration tests, first create a client with the scopes:

assume:project:taskcluster:worker-test-scopes
queue:cancel-task:test-dummy-scheduler/*

Then create a ./secrets.json or ~/.scriptworker that looks like:

{
    "integration_credentials": {
        "clientId": "...",
        "accessToken": "...",
        "certificate": "..."
    }
}

(certificate is only specified if using temp creds)

then

tox

It’s also possible to test in docker. First, build the docker image, making sure to add integration credentials to secrets.json. Then:

docker run -i scriptworker-test-$PY_DOT_VERSION

GPG Homedir testing

Sometimes it’s nice to be able to test things like rebuild_gpg_homedirs. To do so, build the docker image, then:

docker run -i scriptworker-test-$PY_DOT_VERSION gpg
# in the docker shell,
rebuild_gpg_homedirs gnupg.yaml

Table of Contents

• Maintenance
  • New docker shas
  • Chain of Trust settings
  • GPG keys
• Scriptworker Releases
  • Code changes
  • Tests and test coverage
  • Versioning
  • Changelog
  • Release files
    • Requirements
    • Versioning
  • Tagging
  • Pypi
  • Puppet
• Adding new scriptworker instance types
  • Is scriptworker the right tool?
    • Chain of Trust considerations
  • Creating a new scriptworker instance type
    • Write a script
  • Deployment considerations
    • Graph
    • Puppet
    • Monitoring
• Adding new scriptworker instances of an existing type
  • 1. initial setup
  • 2. creating an AWS instance
  • 2a - using automation script
  • 2b - using AWS console
  • 3. Puppetize the instance
  • monitoring
• Chain of Trust
  • Overview
    • High level view
  • Chain of Trust Artifact Generation
    • Embedded GPG keys
    • Chain of Trust artifacts
  • Chain of Trust Verification
    • Decision Task
    • GPG homedir management
    • Building the chain
    • Verifying the chain
  • Chain of Trust GPG Key Management
    • Adding new git commit signing gpg keys
    • Adding new worker gpg keys
  • Chain of Trust Testing / debugging
    • Create the virtualenv
    • Set up the test env
    • Find a task to test
    • Run the test
• Scriptworker Readme
  • Usage
  • Building a docker image
  • Testing
    • GPG Homedir testing
• scriptworker package
  • Submodules
  • scriptworker.client module
  • scriptworker.config module
  • scriptworker.context module
  • scriptworker.exceptions module
  • scriptworker.gpg module
  • scriptworker.log module
  • scriptworker.task module
  • scriptworker.utils module
  • scriptworker.worker module
  • Module contents

Indices and tables

• Index
• Module Index
• Search Page