Lean Drupal Repositories with Composer and Drush

Lean Drupal Repositories with
Composer and Drush
by Greg Anderson
Photo by Adam Wyles
2016

Pantheon.io 2
Session Description
Composer is the industry-standard PHP dependency manager that is now in use in Drupal 8 core. This session will show the current best practices for
using Composer, drupal-composer, drupal-scaffold, Drush, Drupal Console and Drush site-local aliases to streamline your Drupal 7 and Drupal 8 site
repositories for optimal use on teams.
We will answer such gripping questions as:
● How do I avoid placing a copy all of the Drupal core and contrib modules into my repository?
● How do I keep my core and contrib modules up-to-date?
● What if I need to customize .htaccess, or some other file?
● Can I work with a "lean" repository, and still seamlessly use a full repository to deploy?
● How do I share Drush aliases with team members, without using another repository, and without making my alias list too long?
● What is a Drush wrapper script, and how can it help me?
● How does Drupal Console fit in to all of this?
● Should I use test fixtures in my Behat tests, or make a copy of the production site database?
These techniques will also be helpful for solo site developers--you never know, the next person who needs to check out, build and test your Drupal site
from scratch might be you!

Pantheon.io
Then create a Git repository:
cd myproject
git init
git add .
git commit -m “My new project”
3
Getting Started with Drupal and Composer
One easy step in Composer:
composer create-project drupal-composer/drupal-project myproject --stability=dev
Install and run with Drush:
cd web
drush qd --db-url=mysql://root@localhost/myprojectdb
Answer “yes” when Composer
asks to remove VCS files, or run
with --no-interaction option

Pantheon.io 4
Preferred Project Structure
webcomposer.json
drupal console
circle.yml
external libraries
behat.yml
behat drush
drupal/core
index.php
custom-installers
drupal-scaffoldmodules
contrib
custom
vendor

Pantheon.io 5
Specifying Project Layout
{
"name": "drupal-composer/drupal-project",
"scripts": {
"drupal-scaffold": "DrupalComposerDrupalScaffoldPlugin::scaffold",
"post-install-cmd": "sh ./scripts/composer/post-install.sh"
},
"extra": {
"installer-paths": {
"web/core": ["type:drupal-core"],
"web/modules/contrib/{$name}": ["type:drupal-module"],
"web/profiles/contrib/{$name}": ["type:drupal-profile"],
"web/themes/contrib/{$name}": ["type:drupal-theme"],
"drush/contrib/{$name}": ["type:drupal-drush"]
}
}
}

Pantheon.io 6
Renaming Document Root
# Ignore directories generated by Composer
drush/contrib
vendor
docroot/core
docroot/modules/contrib
docroot/themes/contrib
docroot/profiles/contrib
# Ignore Drupal's file directory
docroot/sites/default/files
.gitignore
#!/bin/sh
DOCUMENTROOT=docroot
# Prepare the scaffold files if they are not
if [ ! -f $DOCUMENTROOT/autoload.php ]
then
composer drupal-scaffold
mkdir -p $DOCUMENTROOT/modules
mkdir -p $DOCUMENTROOT/themes
mkdir -p $DOCUMENTROOT/profiles
post-install.sh
{
"extra": {
"docroot/core": ["type:drupal-co
"docroot/modules/contrib/{$name}
"docroot/profiles/contrib/{$name
"docroot/themes/contrib/{$name}"
"drush/contrib/{$name}": ["type:
}
}
}
composer.json

Pantheon.io 7
Scaffold Files
web
drupal/core
scaffold files
modules
contrib
custom
.htaccess
autoload.php
index.php
robots.txt
updates.php
sites
examples.sites.php
default
default.services.yml
default.settings.php
Drupal.org
drupal-scaffold

Pantheon.io 8
Custom Scaffold Configuration
{
"name": "pantheon-systems/drupal-project",
"scripts": {
},
"extra": {
"drupal-scaffold": {
"source": "https://github.com/pantheon-systems/drops-8/archive/{version}.tar.gz",
"includes": [
"sites/default/settings.php",
"sites/default/settings.pantheon.php"
]
}
}
}

Pantheon.io 9
Document Root at Project Root
composer.json
drupal console
external libraries
drush
core
index.php
custom-installers
contrib
custom
vendor
circle.yml
private
Privatesubfolder

Pantheon.io 10
composer.json Relocated to Private Directory
{
"scripts": {
},
"extra": {
"../core": ["type:drupal-core"],
"../modules/contrib/{$name}": ["type:drupal-module"],
"../profiles/contrib/{$name}": ["type:drupal-profile"],
"../themes/contrib/{$name}": ["type:drupal-theme"],
"../drush/contrib/{$name}": ["type:drupal-drush"]
}
}
}

Pantheon.io 11
Updating Code
web
drupal/core
scaffold files
modules
contrib
custom
.htaccess
autoload.php
index.php
robots.txt
updates.php
sites
examples.sites.php
default
default.services.yml
default.settings.php
Drupal.org
drupal-scaffold
composer
update
composer drupal-
scaffold
git pull
Project Files
Only
Scaffold Files
Only
Composer and
Scaffold
Files

Pantheon.io 12
Customizing Scaffold Files
{
"scripts": {
"post-install-cmd": "sh ./scripts/composer/post-install.sh",
"post-drupal-scaffold-cmd": "cat htaccess-append.txt >> ../.htaccess"
}
}
To patch non-scaffold files, use:
cweagans/composer-patches

Pantheon.io 13
Checking for Security Updates
With Drush
With Composer
$ drush pm-updatestatus
Checking available update data ... [ok]
Checking available update data for Drupal. [ok]
Checking available update data for Token (token). [ok]
Name Installed Version Proposed version Message
Token (token) 7.x-1.2 7.x-1.6 SECURITY UPDATE available
$ composer require roave/security-advisories:dev-master
$ composer require drupal-composer/drupal-security-advisories:7.x-dev
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.
Problem 1
- drupal-composer/drupal-security-advisories 7.x-dev conflicts with drupal/token[7.1.2].
- Installation request for drupal-composer/drupal-security-advisories 7.x-dev ->
satisfiable by drupal-composer/drupal-security-advisories[7.x-dev].
- Installation request for drupal/token 7.1.2 -> satisfiable by drupal/token[7.1.2].
n.b. Requires git_deploy module if using Composer with
--prefer-source, or if dev modules are used.

Pantheon.io 14
Manage Custom Modules with Composer
webcomposer.json
drupal console
circle.yml
external libraries
behat.yml
behat drush
drupal/core
index.php
custom-installers
contrib
custom
vendor
Variant
WHY?
Use private modules in
multiple projects.

Pantheon.io 15
Let’s Make a Module!
$ drupal generate:module $ drupal generate:controller

Pantheon.io 16
Customize Module’s composer.json
{
"name": "greg-1-anderson/snazzy",
"type": "drupal-module",
"description": "A Snazzy Module",
"keywords": ["Drupal"],
"license": "GPL-2.0+",
"homepage": "https://github.com/greg-1-anderson/snazzy",
"minimum-stability": "dev",
"support": {
"issues": "https://github.com/greg-1-anderson/snazzy/issues",
"source": "https://github.com/greg-1-anderson/snazzy"
},
"require": { }
}

Pantheon.io 17
Add Our Custom Module to Drupal Project
{
"repositories": [
{
"type": "vcs",
"url": "https://github.com/greg-1-anderson/snazzy.git"
}
],
"require": {
"composer/installers": "^1.0.20",
"drupal-composer/drupal-scaffold": "^1.0",
"greg-1-anderson/snazzy": "dev-master"
},
"extra": {
"web/modules/custom/{$name}": ["greg-1-anderson/snazzy"],
"web/modules/contrib/{$name}": ["type:drupal-module"]
}
}
}

Pantheon.io 18
Generate require Section from Existing Site
composer create-project drupal-composer/drupal-project myproject --stability=dev
cd myproject
drush composer-generate @remote
Look up the modules and themes
used on @remote and add them to
the require section of the
composer.json in the cwd.

Pantheon.io 19
Lean Repository Deployment
Push
Request Test
Request Build
Behat
Deploy
Pull Lean Repo
Install
Update

Pantheon.io 20
Add a Project to Circle
A couple of clicks will get
Circle CI building your project.
There are a few credentials
that should also be set up:
● GitHub OAuth Token
● Terminus Machine Token
● SSH Key Pair

Pantheon.io 21
Generate OAuth Tokens for all Services
GitHub Personal Access Tokens Pantheon Machine Tokens
OAuth tokens are getting to be very common; many services provide them.
They work like passwords, but can have limited permissions, and may be revoked.

Pantheon.io 22
Copy OAuth Tokens into CI Envrionment Variables
OAuth Tokens:

Pantheon.io
Prevents ‘composer install’
from failing due to hitting the
GitHub rate limit.
23
Use OAuth Environment Variables in CI
dependencies:
pre:
- composer config -g github-oauth.github.com $GITHUB_OAUTH
- terminus auth login --machine-token=$PANTHEON_MACHINE_TOKEN
circle.yml
Log in to Terminus via machine
token, e.g. to later deploy from
dev to test.

Pantheon.io 24
Create SSH Key Pair Specifically for CI
$ ssh-keygen -C me+mysite-ci@example.com -f my-ci-keyfile
Add Private Key to Circle SSH Permissions Add Public Key to Provider SSH Keys
For linux servers: ssh-copy-id user@isp.com

Pantheon.io 25
Dependency Hell
Bootstrapping Drupal 8 via a global Drush will ONLY WORK if the
dependencies of the two projects are in perfect alignment.
Upgrading one without upgrading the other is dangerous.
Drush Drupal 8
require autoload.php
Bootstrap require autoload.php

Pantheon.io 26
Simple Example of Dependency Hell
class Sub extends Base
{
public function foo()
{
return $this->bar();
}
}
class Base
{
private function bar()
{
…
}
}
class Sub extends Base
{
public function foo()
{
return $this->boz();
}
}
class Base
{
private function boz()
{
…
}
}
FancyLib v1.0.1 FancyLib v1.0.2
Sub.php Sub.php
Base.php Base.php
Semantic Versioning will not save
you!
Two autoloaders that contain multiple
copies of the same library can still fail,
even if both conform to the same
public API.

Pantheon.io 27
Dependency Hell Affects Drupal Console Too
Drupal Console, like Drush, keeps dependencies in sync with the Drupal 8 release with the
same version number. Problems can still occur if versions become mismatched.
Drupal Console Drupal 8
require autoload.php
Bootstrap require autoload.php

Pantheon.io
Place Drush and Drupal Console in Drupal’s composer.json file:
cd /path/to/drupal-8-root
composer require drush/drush:8.*
composer require drupal/console
28
Solution is to Use a Single Autoloader
CRITICAL

Pantheon.io 29
Gentoo’s “Composer Problem”
“ As long as the necessary
require statements are left in
the code (where they
belong), we can ignore
Composer entirely and install
the package with the system
package manager. We set
PHP's include directory for
our users, so require('Class.
php'); already looks in the
right place.https://wiki.gentoo.org/wiki/Project:PHP/The_Composer_problem

Pantheon.io 30
How Could We Fix Things for Gentoo?
One Class.php shared by every application is not going to work!
Make an autoload.php that loaded versioned Class.php files from
a global location?
How would all of the different versions of Class.php be managed?
How would you apply a security update for Class.php?
By the time we have solved all of
these issues, we have pretty much re-
invented Composer.

Pantheon.io
Drush startup now happens in four phases:
31
Drush Startup
Drush Finder Drush Wrapper Drush Launcher Drush Application

Pantheon.io 32
Drush Finder
Responsible for finding the correct Drush to run
- Checks to see if it can find a Drupal site
- If the Drupal site contains a Drush script, use it
- If no Site-Local Drush is found, use global Drush
Does not consider alias files.
PHP Script
named
“drush”

Pantheon.io 33
Drush Wrapper
Optional. Located at the Drupal Root if it exists.
User may customize this script to:
- Add site-specific options (e.g. commandfile locations)
- Turn off global search locations with --local option
- Select the location of the Drush launcher (if non-standard)
If there is no Drush Wrapper, then the Drush Finder will find and
execute the Drush Launcher.
Shell Script
named
“drush.wrapper”

Pantheon.io 34
Drush Launcher
Sets up the PHP operating environment.
- Select php.ini file to use
- Select php executable to use
- Passes info about environment to Drush
The launcher will always use the Drush application located in the
same directory.
Shell Script
named
“drush.launcher”

Pantheon.io 35
Drush Application
Contains all the code that is Drush.
- Load configuration and command file
- Parse site alias files
Might dispatch again, e.g. if site alias is remote.
PHP
Application
named
“drush.php”

Pantheon.io 36
Drush Phar
Bundles all of the code from the Drush application into a single file.
- Does not use Drush Launcher (no PHP executable selection).
- Will run a Drush Wrapper script if one is available.
If a Drush Wrapper or site-local Drush is found, they will be executed
via redispatch. The site-local Drush may be a Phar or a regular
Drush application.
Phar file
named
“drush.phar”
Phar

Pantheon.io 37
Sharing Drush Aliases
cd "`dirname $0`"
private/vendor/bin/drush.launcher
--local
--include=../drush/commands
--alias-path=../drush/aliases
--config=../drush/config "$@"
drush.wrapper
drush
web
drush.wrapper
aliases
commands
index.php
aliases.drushrc.php
$ cd web
$ drush sa @live
$aliases["live"] = array (
'remote-host' => 'server.isp.com',
'remote-user' => 'www-admin',
'root' => '/srv/www/live.example.com',
'uri' => 'http://example.com',
);

Pantheon.io 38
Behat Driver Enhancement
cd /path/to/drupal-project
composer require drush-ops/behat-drush-endpoint
YES!
Adds behat Drush command
to your site; used to remotely
create content.

Pantheon.io 39
Behat Fixtures
Background:
Given "places" terms:
| name |
| Kingdom of Imaginarium |
| Empire of Fabrication |
And "offices" terms:
| name |
| Grand Poohbah |
| Undersecretary of Things |
| Minister of Ministering |
$ drush behat import --file=fixtures.yml
create_term:
places:
- name: Kingdom of Imaginarium
- name: Empire of Fabrication
offices:
- name: Grand Poohbah
- name: Undersecretary of Things
- name: Minister of Ministering
Option 1:
Use Behat’s built-in “Background”
Option 2:
Create fixtures with behat-drush-endpoint
Deletes and re-creates fixtures for every test; useful
for testing operations that modify or delete terms.
Taxonomy IDs stay constant for every test run.
ALSO (for Drupal 8): drush site-install --config-dir=export-dir

Pantheon.io 40
That’s a Wrap!
Follow me on Twitter for slides and other Composer + Drush updates:
@greg_1_anderson

Lean Drupal Repositories with Composer and Drush

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to Lean Drupal Repositories with Composer and Drush (20)

More from Pantheon (12)

Recently uploaded (20)

Lean Drupal Repositories with Composer and Drush