What you’ll learn

How to run Cypress tests in Continuous Integration

How to configure Cypress in various CI Providers

How to record tests to the Cypress Dashboard

How to run tests in parallel on CI

Setting up CI

Basics

Running Cypress in Continuous Integration is almost the same as running it locally in your terminal. You generally only need to do two things:

Install Cypress
```
npm install cypress --save-dev
```
Run Cypress
```
cypress run
```

Depending on which CI provider you use, you may need a config file. You’ll want to refer to your CI provider’s documentation to know where to add the commands to install and run Cypress. For more configuration examples check out our examples.

Boot your server

Challenges

Typically you will need to boot a local server prior to running Cypress. When you boot your web server, it runs as a long running process that will never exit. Because of this, you’ll need it to run in the background - else your CI provider will never move onto the next command.

Backgrounding your server process means that your CI provider will continue to execute the next command after executing the signal to start your server.

Many people approach this situation by running a command like the following:

npm start & cypress run // Do not do this

The problem is - what happens if your server takes time to boot? There is no guarantee that when the next command runs (cypress run) that your web server is up and available. So your Cypress test may start and try to visit your local server before it is ready to be visited.

Solutions

Luckily, there are some solutions for this. Instead of introducing arbitrary waits (like sleep 20) you can use a better option.

wait-on module

Using the wait-on module, you can block the cypress run command from executing until your server has booted.

npm start & wait-on http://localhost:8080

cypress run

Most CI providers will automatically kill background processes so you don’t have to worry about cleaning up your server process once Cypress finishes.

However, if you’re running this script locally you’ll have to do a bit more work to collect the backgrounded PID and then kill it after cypress run.

start-server-and-test module

If the server takes a very long time to start, we recommend trying the start-server-and-test module.

npm install --save-dev start-server-and-test

Pass the command to boot your server, the url your server is hosted on and your Cypress test command.

{
  "scripts": {
    "start": "my-server -p 3030",
    "cy:run": "cypress run",
    "test": "start-server-and-test start http://localhost:3030 cy:run"
  }
}

In the example above, the cy:run command will only be executed when the URL http://localhost:3030 responds with an HTTP status code of 200. The server will also shut down when the tests complete.

Gotchas

When working with webpack-dev-server that does not respond to HEAD requests, use an explicit GET method to ping the server like this:

{
  "scripts": {
    "test": "start-server-and-test start http-get://localhost:3030 cy:run"
  }
}

When working with local https in webpack, set an environment variable to allow local certificate:

{
  "scripts": {
    "start": "my-server -p 3030 --https",
    "cy:run": "cypress run",
    "cy:ci": "START_SERVER_AND_TEST_INSECURE=1 start-server-and-test start https-get://localhost:3030 cy:run"
  }
}

Record tests

Cypress can record your tests and make the results available in the Cypress Dashboard.

Recording tests allow you to:

See the number of failed, pending and passing tests.
Get the entire stack trace of failed tests.
View screenshots taken when tests fail and when using cy.screenshot().
Watch a video of your entire test run or a clip at the point of test failure.
See which machines ran each test when parallelized.

To record tests:

Set up your project to record
Pass the --record flag to cypress run within CI.

cypress run --record --key=abc123

Read the full guide on the Dashboard Service.

Run tests in parallel

Cypress can run tests in parallel across multiple machines.

You’ll want to refer to your CI provider’s documentation on how to set up multiple machines to run in your CI environment.

Once multiple machines are available within your CI environment, you can pass the --parallel flag to have your tests run in parallel.

cypress run --record --key=abc123 --parallel

Read the full guide on parallelization.

Examples

Cypress should run on all CI providers. We have provided some example projects and configuration for some CI providers to help you get started.

CI Provider	Example Project	Example Config
AppVeyor	cypress-example-kitchensink	appveyor.yml
Azure DevOps / VSTS CI / TeamFoundation	cypress-example-kitchensink	vsts-ci.yml
BitBucket	cypress-example-kitchensink	bitbucket-pipelines.yml
BuildKite	cypress-example-kitchensink	.buildkite/pipeline.yml
CircleCI	cypress-example-kitchensink	circle.yml
CodeShip Basic (has cy.exec() issue)
CodeShip Pro	cypress-example-docker-codeship
Concourse
Docker	cypress-docker-images
GitLab	cypress-example-kitchensink	.gitlab-ci.yml
Jenkins	cypress-example-kitchensink	Jenkinsfile
Semaphore
Shippable	cypress-example-kitchensink	shippable.yml
Solano
TravisCI	cypress-example-kitchensink	.travis.yml

Travis

Example `.travis.yml` config file

language: node_js
node_js:
  - 10
addons:
  apt:
    packages:
      # Ubuntu 16+ does not install this dependency by default, so we need to install it ourselves
      - libgconf-2-4
cache:
  # Caches $HOME/.npm when npm ci is default script command
  # Caches node_modules in all other cases
  npm: true
  directories:
    # we also need to cache folder with Cypress binary
    - ~/.cache
install:
  - npm ci
script:
  - $(npm bin)/cypress run --record

Caching folders with npm modules saves a lot of time after the first build.

CircleCI

New Example CircleCI Orb

The Cypress CircleCI Orb is a piece of configuration set in your circle.yml file to correctly install, cache and run Cypress with very little effort.

Full documentation can be found at the cypress-io/circleci-orb repo.

A typical project can have:

version: 2.1
orbs:
  # "cypress-io/cypress@1" installs the latest published
  # version "1.x.y" of the orb. We recommend you then use
  # the strict explicit version "cypress-io/[email protected]"
  # to lock the version and prevent unexpected CI changes
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/run # "run" job comes from "cypress" orb

A more complex project that needs to install dependencies, build an application and run tests across 4 CI machines in parallel may have:

version: 2.1
orbs:
  cypress: cypress-io/cypress@1
workflows:
  build:
    jobs:
      - cypress/install:
          build: 'npm run build'  # run a custom app build step
      - cypress/run:
          requires:
            - cypress/install
          record: true        # record results on Cypress Dashboard
          parallel: true      # split all specs across machines
          parallelism: 4      # use 4 CircleCI machines to finish quickly
          group: 'all tests'  # name this group "all tests" on the dashboard
          start: 'npm start'  # start server before running tests

In all cases, you are using run and install job definitions that Cypress provides inside the orb. Using the orb brings simplicity and static checks of parameters to CircleCI configuration.

You can find multiple examples at our orb examples page and in the cypress-example-circleci-orb project.

Example `circle.yml` v2 config file

version: 2
jobs:
  build:
    docker:
      - image: cypress/base:8
        environment:
          ## this enables colors in the output
          TERM: xterm
    working_directory: ~/app
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
            - v1-deps-{{ .Branch }}
            - v1-deps
      - run:
          name: Install Dependencies
          command: npm ci
      - save_cache:
          key: v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
          # cache NPM modules and the folder with the Cypress binary
          paths:
            - ~/.npm
            - ~/.cache
      - run: $(npm bin)/cypress run --record --key <record_key>

Example `circle.yml` v2 config file with `yarn`

version: 2
jobs:
  build:
    docker:
      - image: cypress/base:8
        environment:
          ## this enables colors in the output
          TERM: xterm
    working_directory: ~/app
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
            - v1-deps-{{ .Branch }}
            - v1-deps
      - run:
          name: Install Dependencies
          command: yarn install --frozen-lockfile
      - save_cache:
          key: v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
          paths:
            - ~/.cache  ## cache both yarn and Cypress!
      - run: $(yarn bin)/cypress run --record --key <record_key>

Find the complete CircleCI v2 example with caching and artifact upload in the cypress-example-docker-circle repo.

Docker

We have created an official cypress/base container with all of the required dependencies installed. Just add Cypress and go! We are also adding images with browsers pre-installed under cypress/browsers name. A typical Dockerfile would look like this:

FROM cypress/base
RUN npm install
RUN $(npm bin)/cypress run

Mounting a project directory with an existing node_modules into a cypress/base docker image will not work:
docker run -it -v /app:/app cypress/base:8 bash -c 'cypress run'
Error: the cypress binary is not installed
Instead, you should build a docker container for your project’s version of cypress.

Docker images & CI examples

See our examples for additional information on our maintained images and configurations on several CI providers.

Advanced setup

Dependencies

If you are not using one of the above CI providers then make sure your system has these dependencies installed.

apt-get install xvfb libgtk-3-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2

Caching

As of Cypress version 3.0, Cypress downloads its binary to the global system cache - on linux that is ~/.cache/Cypress. By ensuring this cache persists across builds you can shave minutes off install time by preventing a large binary download.

Cache the ~/.cache folder after running npm install, yarn, npm ci or equivalents as demonstrated in the configs below.
Do not cache node_modules across builds. This bypasses more intelligent caching packaged with npm or yarn, and can cause issues with Cypress not downloading the Cypress binary on npm install.
If you are using npm install in your build process, consider switching to npm ci and caching the ~/.npm directory for a faster and more reliable build.
If you are using yarn, caching ~/.cache will include both the yarn and Cypress caches. Consider using yarn install --frozen-lockfile as an npm ci equivalent.

If you need to override the binary location for some reason, use CYPRESS_CACHE_FOLDER environment variable.

Environment variables

You can set various environment variables to modify how Cypress runs.

Configuration Values

You can set any configuration value as an environment variable. This overrides values in your cypress.json.

Typical use cases would be modifying things like:

CYPRESS_BASE_URL
CYPRESS_VIDEO_COMPRESSION
CYPRESS_REPORTER
CYPRESS_INSTALL_BINARY

Refer to the Environment Variables recipe for more examples.

Record Key

If you are recording your runs on a public project, you’ll want to protect your Record Key. Learn why.

Instead of hard coding it into your run command like this:

cypress run --record --key abc-key-123

You can set the record key as the environment variable, CYPRESS_RECORD_KEY, and we’ll automatically use that value. You can now omit the --key flag when recording.

cypress run --record

Typically you’d set this inside of your CI provider.

CircleCI Environment Variable

TravisCI Environment Variable

Git information

Cypress uses the @cypress/commit-info package to extract git information to associate with the run (e.g. branch, commit message, author).

It assumes there is a .git folder and uses Git commands to get each property, like git show -s --pretty=%B to get commit message, see src/git-api.js.

Under some environment setups (e.g. docker/docker-compose) if the .git directory is not available or mounted, you can pass all git related information under custom environment variables.

Branch: COMMIT_INFO_BRANCH
Message: COMMIT_INFO_MESSAGE
Author email: COMMIT_INFO_EMAIL
Author: COMMIT_INFO_AUTHOR
SHA: COMMIT_INFO_SHA
Remote: COMMIT_INFO_REMOTE

If the commit information is missing in the Dashboard run then GitHub Integration or other tasks might not work correctly. To see the relevant Cypress debug logs, set the environment variable DEBUG on your CI machine and inspect the terminal output to see why the commit information is unavailable.

DEBUG=commit-info,cypress:server:record

Custom Environment Variables

You can also set custom environment variables for use in your tests. These enable your code to reference dynamic values.

export "EXTERNAL_API_SERVER=https://corp.acme.co"

And then in your tests:

cy.request({
  method: 'POST',
  url: Cypress.env('EXTERNAL_API_SERVER') + '/users/1',
  body: {
    foo: 'bar',
    baz: 'quux'
  }
})

Refer to the dedicated Environment Variables Guide for more examples.

Module API

Oftentimes it can be much easier to programmatically control and boot your servers with a Node script.

If you’re using our Module API then you can write a script that boots and then shuts down the server later. As a bonus you can easily work with the results and do other things.

// scripts/run-cypress-tests.js

const cypress = require('cypress')
const server = require('./lib/my-server')

// start your server
return server.start()
.then(() => {
  // kick off a cypress run
  return cypress.run()
  .then((results) => {
    // stop your server when it's complete
    return server.stop()
  })
})

node scripts/run-cypress-tests.js

Known Issues

In Docker

If you are running long runs on Docker, you need to set the ipc to host mode. This issue describes exactly what to do.

W3cubDocs

Continuous Integration

What you’ll learn

Setting up CI

Basics

Boot your server

Challenges

Solutions

Record tests

Recording tests allow you to:

To record tests:

Run tests in parallel

Examples

Travis

Example `.travis.yml` config file

CircleCI

New Example CircleCI Orb

Example `circle.yml` v2 config file

Example `circle.yml` v2 config file with `yarn`

Docker

Docker images & CI examples

Advanced setup

Dependencies

Caching

Environment variables

Configuration Values

Git information

Custom Environment Variables

Module API

Known Issues

In Docker

See also

Continuous Integration

What you’ll learn

Setting up CI

Basics

Boot your server

Challenges

Solutions

Record tests

Recording tests allow you to:

To record tests:

Run tests in parallel

Examples

Travis

Example .travis.yml config file

CircleCI

New Example CircleCI Orb

Example circle.yml v2 config file

Example circle.yml v2 config file with yarn

Docker

Docker images & CI examples

Advanced setup

Dependencies

Caching

We recommend users:

Environment variables

Configuration Values

Git information

Custom Environment Variables

Module API

Known Issues

In Docker

See also

Example `.travis.yml` config file

Example `circle.yml` v2 config file

Example `circle.yml` v2 config file with `yarn`