Scriptworker Readme
Scriptworker implements the TaskCluster worker model, then launches a pre-defined script.
This worker was designed for Releng processes that need specific, limited, and pre-defined capabilities.
Free software: MPL2 License
Usage
Create a config file. By default scriptworker will look in
./scriptworker.yaml, but this config path can be specified as the first and only commandline argument. There is an example config file, and all config items are specified in scriptworker.constants.DEFAULT_CONFIG.
Credentials can live in ./scriptworker.yaml, ./secrets.json, ~/.scriptworker.
Launch:
scriptworker [config_path]
Testing
Without integration tests install tox, then
NO_CREDENTIALS_TESTS=1 uv run tox -e py311
Without any tests connecting to the net, then NO_TESTS_OVER_WIRE=1 uv run tox -e py311
With integration tests, first create a client in the Taskcluster UI with the scopes:
queue:cancel-task:test-dummy-scheduler/*
queue:claim-work:test-dummy-provisioner/dummy-worker-*
queue:create-task:lowest:test-dummy-provisioner/dummy-worker-*
queue:define-task:test-dummy-provisioner/dummy-worker-*
queue:get-artifact:SampleArtifacts/_/X.txt
queue:scheduler-id:test-dummy-scheduler
queue:schedule-task:test-dummy-scheduler/*
queue:task-group-id:test-dummy-scheduler/*
queue:worker-id:test-dummy-workers/dummy-worker-*
Then generate a no priviledge personal access token in Github for the scriptworker_github_token (to avoid rate limiting) and create a ./secrets.json or ~/.scriptworker that looks like:
{
"integration_credentials": {
"clientId": "...",
"accessToken": "...",
}
"scriptworker_github_token": "..."
}
then to run all tests: uv run tox
Table of Contents
- Flow of Control
- Maintenance
- Scriptworker Releases
- Adding new scriptworker instance types
- Adding new scriptworker instances of an existing type
- Chain of Trust
- Scriptworker Readme
- scriptworker package
- Submodules
- scriptworker.artifacts module
assert_is_parent()compress_artifact_if_supported()create_artifact()download_artifacts()get_and_check_single_upstream_artifact_full_path()get_artifact_url()get_artifacts_matching_glob()get_expiration_arrow()get_optional_artifacts_per_task_id()get_single_upstream_artifact_full_path()get_upstream_artifacts_full_paths_per_task_id()guess_content_type_and_encoding()list_latest_artifacts()retry_create_artifact()retry_list_latest_artifacts()upload_artifacts()
- scriptworker.client module
- scriptworker.config module
- scriptworker.constants module
- scriptworker.context module
- scriptworker.cot.generate module
- scriptworker.cot.verify module
DECISION_TASK_TYPESPARENT_TASK_TYPESlogAuditLogFormatterChainOfTrustLinkOfTrustadd_link()build_task_dependencies()build_taskcluster_yml_url()check_and_update_action_task_group_id()check_interactive_docker_worker()check_interactive_generic_worker()compare_jsone_task_definition()create_test_workdir()download_cot()download_cot_artifact()download_cot_artifacts()find_sorted_task_dependencies()get_action_context_and_template()get_all_artifacts_per_task_id()get_in_tree_template()get_jsone_context_and_template()get_pushlog_info()get_scm_level()get_source_url()get_valid_task_types()get_valid_worker_impls()guess_task_type()guess_worker_impl()is_artifact_optional()populate_jsone_context()raise_on_errors()trace_back_to_tree()verify_build_task()verify_chain_of_trust()verify_cot_cmdln()verify_cot_signatures()verify_docker_image_sha()verify_docker_image_task()verify_docker_worker_task()verify_generic_worker_task()verify_link_ed25519_cot_signature()verify_link_in_task_graph()verify_parent_task()verify_parent_task_definition()verify_partials_task()verify_repo_matches_url()verify_scriptworker_task()verify_task_in_task_graph()verify_task_types()verify_worker_impls()
- scriptworker.ed25519 module
- scriptworker.github module
- scriptworker.exceptions module
- scriptworker.log module
- scriptworker.task module
REPO_SCOPE_REGEXlogclaim_work()complete_task()get_action_callback_name()get_and_check_tasks_for()get_base_branch()get_base_revision()get_branch()get_commit_message()get_decision_task_id()get_head_revision()get_parent_task_id()get_project()get_provisioner_id()get_pull_request_number()get_push_date_time()get_repo()get_repo_scope()get_revision()get_run_id()get_task_definition()get_task_id()get_task_maxruntime()get_triggered_by()get_worker_pool_id()get_worker_type()is_action()is_github_task()is_pull_request()is_try()is_try_or_pull_request()prepare_to_run_task()reclaim_task()retry_get_task_definition()run_task()worst_level()
- scriptworker.utils module
logadd_enumerable_item_to_dict()add_projectid()add_taskqueueid()calculate_sleep_time()cleanup()create_temp_creds()datestring_to_timestamp()download_file()filepaths_in_dir()format_json()get_hash()get_loggable_url()get_parts_of_url_path()get_results_and_future_exceptions()get_single_item_from_sequence()load_json_or_yaml()load_json_or_yaml_from_url()makedirs()match_url_path_callback()match_url_regex()raise_future_exceptions()read_from_file()remove_empty_keys()request()retry_async()retry_async_decorator()retry_request()retry_sync()rm()scriptworker_session()semaphore_wrapper()to_unicode()write_to_file()
- scriptworker.worker module
- Module contents