Testing Yosys¶

Running the included test suite¶

The Yosys source comes with a test suite to avoid regressions and keep everything working as expected. Tests can be run by calling make test from the root Yosys directory. By default, this runs vanilla and unit tests.

Vanilla tests¶

These make up the majority of our testing coverage. They can be run with make vanilla-test and are based on calls to make subcommands (make makefile-tests) and shell scripts (make seed-tests and make abcopt-tests). Both use run-test.sh files, but make-based tests only call tests/gen-tests-makefile.sh to generate a makefile appropriate for the given directory, so only afterwards when make is invoked do the tests actually run.

Usually their structure looks something like this: you write a .ys file that gets automatically run, which runs a frontend like read_verilog or read_rtlil with a relative path or a heredoc, then runs some commands including the command under test, and then uses Selections with -assert-count. Usually it’s unnecessary to “register” the test anywhere as if it’s being added to an existing directory, depending on how the run-test.sh in that directory works.

Unit tests¶

Running the unit tests requires the following additional packages:

Ubuntu

sudo apt-get install libgtest-dev

macOS

No additional requirements.

Unit tests can be run with make unit-test.

Functional tests¶

Testing functional backends (see Writing a new backend using FunctionalIR) has a few requirements in addition to those listed in Build prerequisites:

Ubuntu

sudo apt-get install racket
raco pkg install rosette
pip install pytest-xdist pytest-xdist-gnumake

macOS

brew install racket
raco pkg install rosette
pip install pytest-xdist pytest-xdist-gnumake

If you don’t have one of the CAD suite(s) installed, you should also install Z3 following their instructions.

Then, set the ENABLE_FUNCTIONAL_TESTS make variable when calling make test and the functional tests will be run as well.

Docs tests¶

There are some additional tests for checking examples included in the documentation, which can be run by calling make test from the yosys/docs sub-directory (or make -C docs test from the root). This also includes checking some macro commands to ensure that descriptions of them are kept up to date, and is mostly intended for CI.

Automatic testing¶

The Yosys Git repo has automatic testing of builds and running of the included test suite on both Ubuntu and macOS, as well as across range of compiler versions. For up to date information, including OS versions, refer to the git actions page.