Selenium/Reference/Dictionary
This is a living document to come to a common understanding of vocabulary as related to test automation. (Template based off of the Performance Dictionary)
| Word | Abbreviation | Description | More |
|---|---|---|---|
| Webdriver.io | wdio | Webdriverio is the end to end test framework that is most commonly used for end to end testing in mediawiki. | https://webdriver.io/ |
| flakey | Test that fails intermittently. May pass in one environment and fail in another. May fail only 1 or 2 times out of 100. | ||
| headless mode | Running a browser without a visible UI | ||
| cypress | Another end to end testing framework that is used some for mediawiki. Browser tests run in the browser for this framework. | https://www.cypress.io/ | |
| regression test | A test that verifies existing functionality still works as expected after a code change. Catches bugs created by new code that was pushed. | ||
| end to end test | e2e | Simulates the user journey for a full user story. Log in, front end, backend, database, and back to the UI to verify results. | |
| smoke test | Testing only the most critical paths. Does it "smoke" when I turn it on? | ||
| sanity check | A quick shallow test run to confirm the build isn't completely broken. Can be used before running the full e2e suite. | ||
| test spec | spec | A test specification file that contains tests cases that describe the expected behavior of a feature or component. This will also usually contain assertions to verify results (comparing expected to actual results of the test run). In wdio this will be formatted like fileName.spec.js | |
| assertion | assert | A statement that verifies expected vs. actual outcome
example: expect(...).toBe(...) |
|
| page object model | POM | ( ?? need to better define this) A way to abstract any page information away from the tests. All selectors and specific instructions can be stored in a page object file that can then be accessed by the test spec. | https://webdriver.io/docs/pageobjects/ |
| hook | A function that runs at a specific point in the test lifecycle. Webdriverio native hooks include before, after, beforeEach, etc. Custom hooks can be added in the wdio.conf.js | ||
| retry flaky test | A fall back so that if a test flakes, it can be automatically retried. The number of retries can be set in the wdio.conf.js, but the default is 2. | ||
| Parallelization | Running multiple test specs simultaneously in multiple browser instances to reduce total time taken for tests to run. This can be configured by maxInstances: x in wdio.conf.js | ||
| XPath selectors | An alternative to CSS selectors for locating elements in the DOM. | ||
| wdio-mediawiki | A plugin for WebdriverIO providing utilities to simplify testing of MediaWiki features. | Gerrit repo, | |
| stale element | A reference to a DOM node that no longer exists | ||
| reporter | the (software? package? ?) that collects and outputs the test results. It formats the full test results and displays or sends them somewhere. For wdio it is included in devDependencies in the package.json, for example: @wdio/spec-reporter | ||
| Test Suite | A collection of related test spec files. There could be multiple suites within one repo. |