Continuous integration/Entry points

Continuous integration
Allow list Entry points in your repository Tutorials Jenkins jobs Dockerfiles Language Versions Policy Quibble
Infrastructure
Troubleshooting Castor Documentation generation Jenkins Zuul Workflow Qemu

These entry points allow you to easily define and run tests in your repository. We have standardized tools and entry points that run in WMF CI across over 1000+ Git repositories including MediaWiki core, extensions, PHP libraries, and other standalone projects.

Documentation on how to configure and set up these tools for each programming language can be found further down this page.

Language	Command runner	Linting	Code style	Static analysis	Unit tests	Documentation
PHP	composer	php-parallel-lint php -l	PHP CodeSniffer with mediawiki-tools-codesniffer	Phan with mediawiki-phan-config	PHPUnit	Doxygen
JavaScript	npm & grunt	grunt-eslint with eslint-config-wikimedia	grunt-eslint with eslint-config-wikimedia		QUnit	JSDoc
JSON			N/A	json-schema	N/A	N/A
i18n		grunt-banana-checker	N/A	N/A	N/A	Localisation Message documentation
CSS/LESS		grunt-stylelint with stylelint-config-wikimedia		N/A	N/A	N/A
Java	maven wrapper (maven)	?	?	?	?	?
Python	tox	flake8			unittest pytest	sphinx?
Ruby	bundler	rubocop		?	rake test	?

List

Here is a list of the commands that will be executed by the various Jenkins jobs on a MediaWiki core patch. You can control which tests and linters run for a repository's continuous integration by changing how commands such as these are configured.

• api-testing - for API tests
• composer phpunit:unit - for PHPUnit tests and parser tests
• composer test - for PHP linters
• exec vendor/bin/phan - for Phan linting
• npm run-script test - for JavaScript linters, Jest tests, and JSDoc warnings
• selenium-test - for Selenium tests
• Special:JavaScriptTest - for QUnit tests

JavaScript

Testing JavaScript

We are using npm test as an entry point for linters and some test suites such as Jest. If your project has any JavaScript files, it should at least have a package.json file which defines a test script, and the related package-lock.json file to ensure consistency in CI runs and security upgrades. QUnit tests are tested by a Jenkins job that loads a browser and visits Special:JavaScriptTest.

You will need the .eslintrc.json configuration file in your project (see Manual:Coding conventions/JavaScript#Linting). Look at one of the projects listed in the example section below for an example of these files.

On older Linux boxes, if the npm commands fails with "node: No such file or directory", you may need to install the "nodejs-legacy" package.

Grunt task runner

If your project has complex build processes, or is an extension or skin that will benefit from i18n checking and JSON file linting, the convention is to use Grunt as a task runner. Your project still has a package.json file, which has a dependency on grunt and sets "test": "grunt test". In turn, a Gruntfile.js file implements grunt test, and this can run a wide variety of tools and tests:

• eslint, which checks both JS and JSON files.
• stylelint, which checks both CSS and LESS files.
• banana-checker, which checks messages in MediaWiki i18n files.

You can specify configuration settings for these tools in Gruntfile.js. However, it should contain little to no configuration for tools that can run outside grunt so that they operate the same when run standalone or from a text editor plugin. Always use native configuration files where possible, including .eslintrc.json mentioned above.

JavaScript documentation

Use npm run doc as the entry point. The convention is to use JSDoc . The predoc and postdoc script hooks in package.json can be used to run any additional scripts (e.g. build files for inclusion beforehand, or copy additional files for publication afterwards).

Examples

Use the BoilerPlate extension as starting point for a new MediaWiki extension.

Advanced setup using Grunt

package.json

{
	"private": true,
	"scripts": {
		"test": "grunt test"
	},
	"devDependencies": {
		"eslint-config-wikimedia": "0.15.0",
		"grunt": "1.0.4",
		"grunt-banana-checker": "0.8.1",
		"grunt-eslint": "22.0.0",
		"grunt-stylelint": "0.12.0",
		"stylelint-config-wikimedia": "0.7.0"
	}
}

Gruntfile.js

/* eslint-env node, es6 */

module.exports = function ( grunt ) {
	grunt.loadNpmTasks( 'grunt-banana-checker' );
	grunt.loadNpmTasks( 'grunt-eslint' );
	grunt.loadNpmTasks( 'grunt-stylelint' );

	grunt.initConfig( {
		eslint: {
			options: {
				extensions: [ '.js', '.json' ],
				cache: true
			},
			all: [
				'**/*.{js,json}',
				'!{vendor,node_modules}/**'
			]
		},
		stylelint: {
			all: [
				'**/*.{css,less}',
				'!{vendor,node_modules}/**'
			]
		},
		banana: {
			all: 'i18n/'
		}
	} );

	grunt.registerTask( 'test', [ 'eslint', 'stylelint', 'banana' ] );
	grunt.registerTask( 'default', 'test' );
};

Example projects

• Extension:BoilerPlate has a Gruntfile.js that runs jshint, jscs, and banana-checker (for MediaWiki's i18n JSON files).
• jquery-client: package.json (jshint, jscs, karma; no Grunt needed)
• CSSJanus: package.json / Gruntfile.js (jshint, jscs, custom test)
• TemplateData: package.json / Gruntfile.js (jshint, jscs, banana-checker)

Further reading

• package.json format on docs.npmjs.org
• package "scripts" lifecycle on docs.npmjs.org

PHP

• Use the BoilerPlate extension as starting point for a new MediaWiki extension.
• Use the mediawiki-tools-cookiecutter-library for creating a new PHP library.

Testing PHP

We are using composer test as an entry point. If your project has PHP files it should list the test framework packages it needs in composer.json under require-dev and list the commands to be run in the scripts.test property:

{
	"require-dev": {
		"mediawiki/mediawiki-codesniffer": "48.0.0",
		"mediawiki/mediawiki-phan-config": "0.18.0",
		"mediawiki/minus-x": "2.0.0",
		"ockcyp/covers-validator": "1.6.0",
		"php-parallel-lint/php-console-highlighter": "1.0.0",
		"php-parallel-lint/php-parallel-lint": "1.4.0",
		"phpunit/phpunit": "10.5.58"
	},
	"scripts": {
		"test": [
			"parallel-lint . --exclude vendor --exclude node_modules",
			"php -d 'extension=pcov.so' vendor/bin/phpunit",
			"covers-validator",
			"phpcs -sp",
			"phan --allow-polyfill-parser --long-progress-bar",
			"minus-x check ."
		],
		"fix": [
			"minus-x fix .",
			"phpcbf"
		]
	},
	"config": {
		"allow-plugins": {
			"dealerdirect/phpcodesniffer-composer-installer": true
		}
	}
}

See composer.json of the cdb project for a good example.

Note that MediaWiki extensions are not standalone projects and cannot run their own PHPUnit test suite from composer. Those repositories have a separate mediawiki-extensions job. PHPCS and PHP lint are still run via composer.json and composer test:

{
	"require-dev": {
		"mediawiki/mediawiki-codesniffer": "44.0.0",
		"mediawiki/mediawiki-phan-config": "0.14.0",
		"mediawiki/minus-x": "1.1.3",
		"php-parallel-lint/php-console-highlighter": "1.0.0",
		"php-parallel-lint/php-parallel-lint": "1.4.0"
	},
	"scripts": {
		"test": [
			"parallel-lint . --exclude vendor --exclude node_modules",
			"phpcs -sp --cache",
			"minus-x check ."
		],
		"fix": [
			"minus-x fix .",
			"phpcbf"
		],
		"phan": "phan -d . --long-progress-bar"
	},
	"config": {
		"allow-plugins": {
			"dealerdirect/phpcodesniffer-composer-installer": true
		}
	}
}

See composer.json of the AbuseFilter MediaWiki extension for a good example.

PHP Documentation

See: Doxygen .

Use the doxygen program to generate a Doxyfile file in the project root.

Testing Python

See Continuous integration/Tutorials/Test your python.

Ruby

Rake

Use Rake to define your commands, they will be executed via Bundler.

Example Rakefile:

require 'bundler/setup'

require 'rubocop/rake_task'
RuboCop::RakeTask.new(:rubocop) do |task|
  # if you use mediawiki-vagrant, rubocop will by default use it's .rubocop.yml
  # the next line makes it explicit that you want .rubocop.yml from the directory
  # where `bundle exec rake` is executed
  task.options = ['-c', '.rubocop.yml']
end

require 'mediawiki_selenium/rake_task'
MediawikiSelenium::RakeTask.new

task default: [:test]

desc 'Run all build/tests commands (CI entry point)'
task test: [:rubocop]

The above code will create following Rake targets.

$ bundle exec rake -T

rake rubocop               # Run RuboCop
rake rubocop:auto_correct  # Auto-correct RuboCop offenses
rake selenium              # Run Cucumber features
rake test                  # Run all build/tests commands (CI entry point)

The Jenkins job rake-jessie invokes test target by running bundle exec rake test.

Reference: phab:T104024

ruby debug tip

You can use the gem pry to break on error and get shown a console in the context of the failure. To your Gemfile add gem 'pry' then to break:

require 'pry'
binding.pry
your call that fail

You will then be in a console before the breakage that let you inspect the environment (ls). See https://github.com/pry/pry for details.

ci.yml

We have a set of Jenkins jobs that run daily and execute Ruby + Selenium tests. The jobs are named selenium*.

Each repositories has only a single job defined in Jenkins. It is a multi configuration job that spawns one or more child job based on a configuration in each repositories: tests/browser/ci.yml. The main job will then spawn child jobs based on its content.

Example of a simple ci.yml is in mediawiki/core.

BROWSER:
  - firefox

MEDIAWIKI_ENVIRONMENT:
  - beta

PLATFORM:
 - Linux

As you can see, there are three variables, BROWSER, MEDIAWIKI_ENVIRONMENT and PLATFORM.

BROWSER and PLATFORM can be any valid Sauce Labs browser/OS/version combination.

MEDIAWIKI_ENVIRONMENT can have values beta, mediawiki and test, or any other environment configured in environments.yml.

For example:

BROWSER:
  - chrome
  - firefox
  - internet_explorer 9.0
  - safari

MEDIAWIKI_ENVIRONMENT:
  - beta
  - mediawiki
  - test

PLATFORM:
  - Linux
  - OS X 10.9
  - Windows 8.1

Example of a complicated ci.yml is in mediawiki/extensions/MultimediaViewer. For more information see Jenkins Yaml Axis Plugin.

Reference: phab:T128190