Trebuchet is a two-phase deployment system with reporting implemented using SaltStack. Its primary transportation method uses git.
See the design documentation at Wikimedia's Wikitech wiki.
SaltStack is a hard requirement.
A mininum of two nodes is required: salt master/deployment server and a deployment target. It is recommended to use three nodes, splitting the salt master apart from the deployment server.
Include the salt states, based on the node type:
- salt master: deployment.salt_master
- deployment server: deployment.deployment_server
- deployment target: deployment.deployment_target
Note: It's best to include deployment.deployment_target from states that configure your targets. For instance, if you are running MediaWiki, you should include this state from the state that installs MediaWiki's prerequisites.
TODO: Add these salt states to the repo
Using the trebuchet puppet module (https://github.com/trebuchet-deploy/puppet).
TODO: Add puppet module documentation
All of Trebuchet is configured using Salt Pillars. Two pillars are used, deployment_config and repo_config.
-
parent_dir (default: none; required)
A string that defines the parent directory for all repositories on both the deployment server and the targets. A repo of name test/testrepo would reside at <parent_dir>/test/testrepo.
-
servers (default: none; required)
A hash that defines datacenter to deployment server mappings.
-
redis (default: none; required)
A hash that defines redis information; required fields: host, port, db
-
grain (default: none; required)
A string that defines the grain this repo targets. This should be the same value used in the deployment::target definition for the hosts in puppet.
-
upstream (default: none)
A string that defines the url of the upstream repo associated with this deployment repo. This is used to initialize the repo on the deployment server. If no upstream is defined then an empty repository will be created.
-
shadow_reference (default: false)
A boolean that defines whether or not this repo will create a reference clone of this repo during the fetch stage. Example: test/testrepo would also have a test/.testrepo clone on the targets that is fully checked out to the deployment tag during the fetch stage.
-
fetch_module_calls (default: {})
A hash of salt modules with a list of arguments that will get called on the minion at the end of the fetch stage of deployment.
-
checkout_module_calls (default: {})
A hash of salt modules with a list of arguments that will get called on the minion at the end of the checkout stage of deployment. The following argument expansions exist: __REPO__ expands to the name of the repo.
-
checkout_submodules (default: false)
A boolean that defines whether or not this repo should also do submodule actions. The following argument expansions exist: __REPO__ expands to the name of the repo.
-
location (default: /srv/deployment/)
The string location of the repository on the deployment system and on the minion. You should avoid setting this unless you really know what you are doing.
-
sync_script (default: shared.py)
A string that defines the sync script used for this repository. Options are depends.py and shared.py. This is used by the git-deploy hooks and is otherwise unused.
-
dependencies (default: {})
A hash of repositories with dependency scripts that this repository depends on. These repositories will be deployed automatically before this repository. Example to add the l10n-slot0 dependency for a repo, with that dependency using the l10n dependency script: l10n-slot0: l10n. Dependencies are awkward and should be avoided if possible.
-
automated (default: false)
A boolean that defines whether or not this repository is automatically or manually deployed. This is used by the git-deploy hooks and is otherwise unused.
The following configuration will configure two repositories: mediawiki/slot0 and test/testrepo. Each repository will exist under /srv/deployment on both the deployment server (deployment-west.example.org) and all deployment targets with a matching grain.
Note: It's possible to break this pillar configuration into multiple files and include the configuration on sets of deployment targets, to limit the ability for a target to access a repo. To do so, you'll need to use the extend support for pillars available in the current development release.
deployment_config:
parent_dir: /srv/deployment
servers:
uswest: deployment-west.example.org
redis:
host: deployment-west.example.org
db: 0
port: 6379
repo_config:
test/testrepo:
grain: test
mediawiki/slot0:
grain: mediawiki
upstream: https://gerrit.wikimedia.org/r/p/mediawiki/core
shadow_reference: true
checkout_submodules: true
fetch_module_calls:
mediawiki.generate_localization_cache:
- __REPO__
checkout_module_calls:
mediawiki.update_localization_cache:
- __REPO__
It it currently easiest to interact with Trebuchet using Trebuchet Trigger, a git interface. Future plans will allow the use of Trebuchet through the Trebuchet Ricochet web frontend.
Assuming you've installed trigger support as described in trigger's Installation section, the basic use of trebuchet is controlled through trigger:
git deploy start
<git changes you would like to make>
git deploy sync
Wikimedia Foundation is currently using Trebuchet for deployment, and has more detailed documentation that is specific to their use case.
For bugs, please use the github issues tracker for this project. To discuss usage or development of trigger, please join me on #trebuchet-deploy on Freenode.