Ruby Raider

Running tests and results

Running a test opens a browser, replays every step and tells you whether the test passed. You can run one test, the tests with a tag, or the whole project, and read the results on the Dashboard.

Run one test

Open the test on the Tests tab and press Run, or ⌘R (Ctrl+R). The result shows below the steps. Last run shows the full output of the last run of that test.

A test you have not saved yet cannot run: the app asks you to save it first. Press Stop & save, then Run.

A project on a ruby_raider older than 4.0 cannot run until you update it; see Projects.

If the test fails, the app shows the step that failed and offers fixes; see Fixing a failed test.

Run tagged tests

On the Tests tab, pick a tag in the Tag list, open More and choose Run tagged tests. To tag a test, see The Tests tab.

Run the whole project

On the Files tab, click the run button on the project's row at the top of the file tree. It runs every test the Run setting on the toolbar selects:

Next to it:

The run panel

While a run is going, the run panel lists each test as Queued, Pending, Running, Passed, Failed, Quarantined or Skipped, with a count such as "3 of 10". Before the first test, it may say "Installing gems…" or "Starting Ruby…".

If you leave the page, an indicator keeps showing the progress; click Show the run to come back. When the run finishes, your computer shows a notification with how many tests passed and failed.

Only one run can go at a time. If a run is already going, in this project or another, the app tells you.

To run tests on a schedule, even when the app is closed, see Scheduling tests.

Dashboard

The Dashboard tab shows the results of the last run.

Quarantined tests failed, but were quarantined in Raider, so they did not fail the build. To treat them like any other failure, turn on Count quarantined failures as failures under More options on the Files tab.

Failure video

Raider can record a video of a failed test. To watch it, click a failed result card on the Dashboard, then expand Failure video.

Recording a video needs ffmpeg. Until a release ships with its own ffmpeg, set the RAIDER_FFMPEG_PATH environment variable to an ffmpeg program built with libvpx before you start Raider Desktop.

Accessibility results

In a project with Accessibility testing (axe), a failed accessibility check lists each problem it found on the page, with how serious it is and a suggestion for fixing it.

Screenshot comparison

When a test compares the page with a saved picture, the result shows the Expected, Actual and Diff pictures and how many pixels differ. If the change is intended, click Accept as baseline to make the new picture the one to compare with.