Drupal 8 Sitebuilding Notes

To add insult to injury, I decided to supplement my writeup of Drupal 8 migrations with a writeup on what I learned during the sitebuilding process. Let us all hope this one will be shorter.

As before, the code I used for the KWLUG site is on Github: https://github.com/pnijjar/kwlug-drupal8-migration .

PHP and Apache setup

I wanted to set up both Drupal 6 and Drupal 8 on the same box. Because I am a dinosaur, I did not use containers or virtual machines for this. Instead I tried to install a single PHP for both versions.

The development machine I used runs Ubuntu 16.04, which has PHP 7 installed. That is okay for D8, but D6 will not run on PHP 7. Thus I had to use a PPA (via https://phpraxis.wordpress.com/2016/05/16/install-php-5-6-or-5-5-in-ubuntu-16-04-lts-xenial-xerus/):

sudo add-apt-repository ppa:ondrej/php
aptitude update

aptitude install libapache2-mod-php5.6 php5.6-mysql

You probably want to uninstall PHP 7 so that Drush (which uses the CLI version of PHP) does not get confused.

The next problem was that the default Apache Ubuntu wanted to install did not work for this version of PHP:

Apache is running a threaded MPM, but your PHP Module is not compiled to be threadsafe. You need to recompile PHP.

The solution to this is to change the configuration of Apache used:

a2dismod mpm_event
a2enmod mpm_prefork

(I thought there used to be different apache2 versions packaged in Ubuntu, but apparently not. I am a dinosaur.)

You also need to enable the rewrite module for better URLs:

a2enmod rewrite

In order to get admin pages working and Drupal installed you need some more PHP modules:

aptitude install php5.6-xml php5.6-gd

Otherwise you get errors like:

PHP Fatal error: Call to undefined function xml_parser_create() in /var/www/html/d6/public_html/modules/update/update.fetch.inc on line 196

Later I needed an mbstring extension for some dependency composer complained about:

aptitude install php5.6-mbstring

The specific error was:

easyrdf/easyrdf 0.9.0-rc.1 requires ext-mbstring * -> the requested PHP extension mbstring is missing from your system.

To get htaccess files working, you need to add the following to the configuration file for your website:

# Allow .htaccess
<Directory /var/www/html/d6/public_html/>
    AllowOverride All
    Options Indexes FollowSymLinks
    Require all granted
</Directory>

Assuming /var/www/html/d6/public_html is your document root.

Drupal Settings File

You will want to set your trusted_host_patterns in settings.local.php of your Drupal installation:

$settings['trusted_host_patterns'] = array (
  '^d8\.kwlug\.org$',
  '^kwlug\.org$',
);

For this to work you will need to enable the settings.local.php by uncommenting the following from settings.php:

# if (file_exists(__DIR__ . '/settings.local.php')) {
#   include __DIR__ . '/settings.local.php';
# }

Composer Setup

Composer is the trendy new way to manage dependencies for Drupal 8. It is not yet mandatory for core Drupal, but already important projects like Drupal Commerce depend upon it. So we are probably stuck.

I am told that the best starter source for Composer is on Github. Thus you can do the following to get started:

composer create-project drupal-composer/drupal-project:8.x-dev kwlug-install

All of your Drupal files are in the web/ subfolder, so in your Apache config you need to set the document root there.

However, in order to actually use composer you need to be in the root folder, NOT the web/ one. I cannot tell you how many times I made this mistake.

Then to install the dependencies specified by composer.json I typed:

composer update --with-dependencies

To add a new component (for example, the riddler module), I typed:

composer require drupal/riddler 
drush en riddler

I do not think I needed to specify specific versions, but every other Drupal tutorial on the Internet wants me to.

I found I had to change some of the configuration information in composer.json so that Drupal would get updated. I changed:

    "drupal/core": "~8.0",
    "drush/drush": "~8.0",

    "drupal/core": "^8.0",
    "drush/drush": "^8.0",

to avoid "Dependency "drupal/core" is also a root requirement, but is not explicitly whitelisted. Ignoring."

This allows Drupal to upgrade to minor versions (eg 8.1, 8.2). I think the original form locks Drupal to 8.0.x versions (?!).

The versions of the migrate_plus dependency was specified incorrectly as well:

"drupal/migrate_plus": "^1.0",

needed to become:

"drupal/migrate_plus": "^3.0",

(Now you know why I deeply mistrust Composer and its brethren. How am I supposed to know when I have to change dependencies manually?)

With composer installed, usage of Drush becomes different. Fortunately, Drush is smart enough to know when it is being called via composer:

This codebase is assembled with Composer instead of Drush. Use composer update and composer require instead of drush pm-updatecode and drush pm-download. You may override this error by using the --pm-force option.

Never use --pm-force. If you install anything behind composer's back you will be sorry. The usual workflow for updating your Drupal site will look like:

composer update --with-dependencies
drush updb

and you will never use drush up anymore. Yay progress.

Drush on Shared Hosts

kwlug.org is hosted on shared hosting, and I do not have direct access to the PHP configuration. When I tried to run Drush I would get errors like this:

proc_open() has been disabled for security reasons bootstrap.inc

This was indeed the case. The system php.ini being used (which you maybe can find with php --ini) contained the following line:

disable_functions = "show_source, system, shell_exec, passthru, exec, phpinfo, popen, proc_open"

I made a copy of this php.ini file to ~/.drush. Then in the copy I changed the disable_functions line to:

disable_functions = "show_source, system, shell_exec, passthru, exec, phpinfo, popen"

Next I had to tell Drush to use this copy of php.ini . In my .bashrc I added an alias like the following:

alias drush='php -c ~/.drush/php.ini /path/to/drush/drush/drush'

I have no idea why scripts can specify their own php.ini files, but this solution worked for me.

I got the idea for this solution from this blog post: http://inet-design.com/blogs/michael/how-fix-drush-error-exec-has-been-disabled-security-reasons-bootstrapinc.html . That article got me most of the way there but I struggled with actually calling Drush with the modified php.ini.

Building the Menu

I wanted to generate the site menu as part of my site migration. It turns out that modules can create menu items by specifying a .links.menu.yml file. I made a module called kwlug_menu for this. Here are some excerpts from kwlug_menu.links.menu.yml:

kwlug_menu.meetings:
  title: Meetings
  description: "What's the buzz? Tell me what's happening."
  url: internal:/upcoming-meetings
  menu_name: main
  weight: -50
  expanded: 1

kwlug_menu.upcoming-meetings:
  title: About Meetings
  parent: kwlug_menu.meetings
  url: internal:/about-meetings
  menu_name: main
  weight: -49
  expanded: 1

kwlug_menu.presenter-tips:
  title: Tips for Presenters
  parent: kwlug_menu.meetings
  url: internal:/presenter-tips
  menu_name: main
  weight: -47
  expanded: 1

kwlug_menu.community:
  title: Community
  url: internal:/community
  menu_name: main
  weight: -49
  expanded: 1

kwlug_menu.mailing-lists:
  title: Mailing Lists
  url: internal:/mailing-lists
  parent: kwlug_menu.community
  menu_name: main
  weight: -50
  expanded: 1

Here is a partial explanation of the structure:

The headings specify machine names for each menu item. You don't need to prefix these with the name of the module, doing so avoids namespace collisions.
All the url entries are internal links, and following the advice from http://blog.dcycle.com/blog/83/what-content-what-configuration/ all menu items have URL aliases. This was pretty easy because all menu items are targeted either at pages or at views.
The title entries are what show up for users.
The menu_name indicates the menu these items belong to. That menu needs to exist when this module is installed (or maybe a YML file can be included in the module's /config/install/ folder).
The parent is a heading of this menu item's parent, if one exists. This is how I structured the menu.
The weight values are tricky. I came up with these by first organizing my menu in the GUI, and then looking at the menu_tree table of the D8 database:
```
select weight,url from menu_tree where menu_name='main'; 
```
I do not know what expanded means, but I always set it to 1.

Using modules for menu items is a double-edged sword. The default installation profile makes a menu item called Home, and you cannot delete the menu item because it is installed by a module. You can, however, disable menu items, export configuration, and then use the core.menu.static_menu_link_overrides.yml YAML file to make this override permanent.

The inspiration for this hack came from this set of presentation slides: http://www.slideshare.net/exove/goodbye-hookmenu-routing-and-menus-in-drupal-8 , but those examples were super complicated. (I think modules are supposed to specify their own menu entries in the admin menu or something, and this requires code.) Using internal: links was not obvious at first, but works fine.

Making iframes Responsive

I needed to embed archive.org players and Google calendars in the site, but embedding them naively means they are not responsive. The website https://embedresponsively.com helped a lot here.

RSS Enclosures

Sad news: as of this writing the views_rss module is not ready for Drupal 8. This is sad news because I depended upon this module (and its sibling views_rss_itunes) to create podcast feeds with enclosures.

Views comes with builtin RSS feeds, but they are fairly limited and do not support enclosure tags out of the box. So I extended the box by writing a module called kwlug_enclosure_rss.

I did not take adequate notes about how I figured out what to change and why, and now that I am looking at the code again I don't even remember writing it, so these explanations are going to be inadequate. Sorry.

Enclosure Field

The first step was to extend the view by adding fields for enclosure elements (the podcast URL, length, and type). I defined these fields in the podcast content-type, but I still needed some way to insert them into the view. To do this I put a class RssEnclosureFields in src/Plugin/views/row/RssEnclosureFields.php of my module. I then had to override the defineOptions(), buildOptionsForm() and render() methods of the parent:

defineOptions() adds variables to the Views form.
buildOptionsForm() alters the Views form so that you can select elements for these variables.
render() processes the form, populating a variable called $item->enclosure .

In kwlug_enclosure_rss.module I added a method template_preprocess_views_view_row_enclosure_rss, which I believe defines an additional variable (enclosure) for the TWIG template.

Finally I had to define the TWIG template itself. I did this by copying core/modules/views/templates/views-view-row-rss.html.twig to my module, in the templates folder. I did not even rename it! But I did modify it by adding code for the enclosure element.

For some reason I put a bunch of placeholder templates and functions into this module (eg src/Plugin/views/row/RssFields.php) but I do not remember whether this was necessary (because Views was having troubles finding the templates in their proper locations) or whether this was just troubleshooting.

Extending RSS Views

In the Format section of Views new entry called Fields with Enclosures. This is defined in RssEnclosureFields.php as well, in the header:

/**
 * Renders an RSS item based on fields.
 *
 * @ViewsRow(
 *   id = "rss_enclosure_fields",
 *   title = @Translation("Fields with Enclosures"),
 *   help = @Translation("Display fields as RSS items, including
 *   an enclosure."),
 *   theme = "views_view_row_enclosure_rss",
 *   display_types = {"feed"}
 * )
 */
class RssEnclosureFields extends RssFields {

This indicates that the rss_enclosure_fields class should be associated with RSS feed format. The overridden form defined by buildOptionsForm shows up in the Settings section of the Format -> Show menu.

RSS Pictures

RSS feeds defined by Views do not have a picture associated with them. The correct way to fix this would have been to override core/views/Plugin/views/style/Rss.php (and in fact I did this, although I did not use this functionality). The wrong way to do this was to override the TWIG template in my theme, which is what I did. In kwlug_theme/templates/views I copied the template from core/views/templates/views-view-rss.html.twig into kwlug_theme/templates/views/views-view-rss.html.twig and hardcoded information about our icon and license.

This will break if we ever change the theme, and it also applies the templates to ALL RSS feeds, not just the podcast ones.

Downloading Podcast Metadata

The old Drupal 6 version of the website had some custom code to download podcast metadata (specifically the podcast size) from archive.org . I thought it would be a lot of work to migrate this to Drupal 8, but it wasn't bad. In the kwlug_content_types module I used a used a hook_form_alter to download metadata when podcast entries were created or modified. The function prototype changed from:

function kwlug_customcode_form_alter(&$form, &$form_state, $form_id) {

function kwlug_content_types_form_alter(&$form, \Drupal\Core\Form\FormStateInterface $form _state, $form_id) {

There were other changes in the custom module, but much of the code structure was the same. Given the OOP nature of D8 I was expecting the changes to be more significant.

The underlying principle behind downloading podcast metadata is to download the headers from archive.org, and then looking at the Content-Length header to get the podcast's length.

Static Blocks and Configuration Management

We wanted a static sidebar block listing our social media presences. I considered the text in this block to be code, not data (the same way we consider the site title to be code).

Drupal considers blocks of text to be data, and thus stores them in the database. That means you cannot use configuration management to export these blocks easily.

Here is a way to cheat: Use a view. Headers and footers of views are considered configuration and can be exported. Thus, make a dummy view that selects no nodes (I chose content authored before 1991-09-21 as an in-joke) and specify the No Results Behavior to show Global: Unfiltered text. Fill the text field with your static content.

You can make static pages exportable with configuration management using this hack too. Just make sure the view will never show anything!

SSL enabling the site

This gave me trouble. I wanted all website access to lose the www (so www.kwlug.org/sjk would redirect to kwlug.org/sjk) but I wanted everything to be SSL enabled.

The standard advice seems to be adding the following snippet to the bottom of the .htaccess file.

# Rewrite http as https
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteCond %{REQUEST_URI} !^/[0-9]+\..+\.cpaneldcv$
RewriteCond %{REQUEST_URI} !^/[A-F0-9]{32}\.txt(?:\ Comodo\ DCV)?$
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</ifModule>

but this doesn't work. It redirects all of my traffic to the base URL, so http://kwlug.org/sjk becomes https://kwlug.org .

Instead, I had to insert snippets earlier in the file. I changed:

# To redirect all users to access the site WITHOUT the 'www.' prefix,
# (http://www.example.com/foo will be redirected to http://example.com/foo)
# uncomment the following:
RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]

to:

# To redirect all users to access the site WITHOUT the 'www.' prefix,
# (http://www.example.com/foo will be redirected to http://example.com/foo)
# uncomment the following:
RewriteCond %{HTTP_HOST} ^www\.(.+)$ [NC]

# Force everything to https -- pnijjar
# This turns http://www.kwlug.org/sjk to https://kwlug.org/sjk, but
# leaves http://kwlug.org/sjk alone. Sad!
RewriteRule ^ https://%1%{REQUEST_URI} [L,R=301]

# Rewrite http as https
# Added by pnijjar
RewriteCond %{HTTPS} off
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L]

The first rule rewrites URLs beginning with www and the second rewrites ones with no www. The [L] at the end is pretty important, so that the .htaccess file stops processing (and restarts processing from the top of the file? This is not clear to me).

Fixing Adaptivetheme

Because I have no frontend theming skills, I used Adaptivetheme as my base theme and made a subtheme.

Overall, Adaptivetheme worked fairly well for me. If nothing else, responsive menus seem difficult to find in the Drupal 8 world. However, there were quirks with Javascript and menus.

Menu Quirks

The terminology Adaptivetheme uses for its menus confused me. Here is my understanding:

The "Responsive" menu is what is shown when the site is in full width (eg on a desktop with lots of pixels). I chose the dropdown style for this so that it would look like an old-school application menu.
The "Default" menu is shown at small sizes, such as on phones. I chose the slidedown menu for this, because it had big buttons and could nest submenus well.

When Javascript is disabled, the default menu is used for all sizes. I guess Adaptivetheme needs Javascript to swap the default menu for the responsive one.

Because the menus themselves may require Javascript (slidedown does, in any case) all menu items are expanded when Javascript is disabled. This is suboptimal, so I employed some dirty tricks.

I used a tiny jQuery script (addjsclass.js) to add a class has-js to the menu if Javascript was enabled. This Javascript snippet went into the scripts/ folder of the subtheme.

In the CSS I checked for the absence of this class. If the user hovers over the menu bar, the menu displays. Otherwise it is set to be hidden:

/* Make button toggle visibility of below menu (for when javascript
 * is not enabled.
 */
.rm-block__inner:not(.has-js)  #rm-content {
  display: none;
}

.rm-block__inner:not(.has-js):hover  #rm-content {
  display: block;
}

I wish I could have made the menu display when I hovered over the hamburger icon as opposed to anywhere over the toolbar. But the way that the CSS classes are laid out makes this difficult.

This solution has the disadvantage that if you accidentally hover over the menu bar, the menu fills up the screen and is difficult to minimise again.

Javascript Quirks

Adaptivetheme hosts a bunch of its Javascript on Cloudflare, which is irritating (but no doubt standard practice). I wish there was a good way to cache this Javascript in the theme so that the theme could be self-contained, but I do not know how to do this.

Without Javascript the fonts look funny and the menus are ugly. From my understanding this is a regression from Drupal 7. But the rest of the theme works okay, and the site is fairly usable without Javascript. (Given the tinfoil-hat tendencies of KWLUG members, this was important.)

Disabling regions

I found that I wanted to disable certain blocks in kwlug_theme. The easy way to do this is with configuration management. Go to Structure -> Block layout and make the changes you want. Then use drupal config-export to export the structure. Take the blocks that are relevant to your theme (for me, these were named things like block.block.kwlug_theme_tools.yml) and put them in the config/install folder of your theme. Then the next time you install/enable the theme your blocks will be correct.

Allowing Shortcuts for Users

I hid the user block by default, so it is not obvious to users how they can log out from the site. Thus I use the "shortcut" functionality to recreate those parts of the user block that are necessary.

Users need "Use shortcuts" permissions in admin/people/permissions to use shortcuts. However, the shortcut blocks seem to work best with Javascript, and I found that they did not work in Midori.

Increasing the File Upload Size

I ran into trouble when trying to increase the maximum upload size from its default of 10MB per file. The trick is to go to the upload field of the content type and change the limit there. For example, to change the Agenda type's upload site I had to navigate to https://mysite/admin/structure/types/manage/agenda/fields/node.agenda.field_file_attachments and change the maximum upload size there.

You also have to ensure that the PHP file limit is okay. There is a parameter in php.ini called upload_max_filesize that is relevant.

Favicons

I redid the favicon.ico for the site in different sizes thanks to the following tutorial: http://www.catalyst.net.nz/news/creating-multi-resolution-favicon-including-transparency-gimp .

CAPTCHAs

I have found the best CAPTCHA system to be Riddler: https://drupal.org/project/riddler . This lets you make custom questions according to local conditions. KWLUG is geographically based, so most of our questions have to do with local layout. This works surprisingly well for keeping out the spambots. It is also accessible even in a text-only browser.

Configuration for Riddler was tricky to find. It is in /admin/config/people/captcha .