Skip to content

Commit 82bec93

Browse files
committed
[libc++] Update the documentation for running Lit to reflect reality
Our documentation for running LIT is basically wrong, since it doesn't mention `llvm-lit`, and nothing works without it. Note that this documentation improvement reflects the current reality outside of the recent efforts on improving the test suite. My goal is to document the current way of running the test suite, so that everybody agrees on how things currently work. As the configuration system for libc++ gets simplified, we can come back and keep this documentation up to date to reflect those changes. Differential Revision: https://reviews.llvm.org/D77673
1 parent 3d4d25f commit 82bec93

File tree

1 file changed

+40
-36
lines changed

1 file changed

+40
-36
lines changed

libcxx/docs/TestingLibcxx.rst

Lines changed: 40 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Getting Started
1010

1111
libc++ uses LIT to configure and run its tests.
1212

13-
The primary way to run the libc++ tests is by using `make check-cxx`.
13+
The primary way to run the libc++ tests is by using ``make check-cxx``.
1414

1515
However since libc++ can be used in any number of possible
1616
configurations it is important to customize the way LIT builds and runs
@@ -21,38 +21,18 @@ Please see the `Lit Command Guide`_ for more information about LIT.
2121

2222
.. _LIT Command Guide: https://llvm.org/docs/CommandGuide/lit.html
2323

24-
Setting up the Environment
25-
--------------------------
24+
Usage
25+
-----
2626

27-
After building libc++ you must setup your environment to test libc++ using
28-
LIT.
29-
30-
#. Create a shortcut to the actual lit executable so that you can invoke it
31-
easily from the command line.
32-
33-
.. code-block:: bash
34-
35-
$ alias lit='python path/to/llvm/utils/lit/lit.py'
36-
37-
#. Tell LIT where to find your build configuration.
38-
39-
.. code-block:: bash
40-
41-
$ export LIBCXX_SITE_CONFIG=path/to/build-libcxx/test/lit.site.cfg
42-
43-
Example Usage
44-
-------------
45-
46-
Once you have your environment set up and you have built libc++ you can run
47-
parts of the libc++ test suite by simply running `lit` on a specified test or
48-
directory. For example:
27+
After building libc++, you can run parts of the libc++ test suite by simply
28+
running ``llvm-lit`` on a specified test or directory. For example:
4929

5030
.. code-block:: bash
5131
52-
$ cd path/to/src/libcxx
53-
$ lit -sv test/std/re # Run all of the std::regex tests
54-
$ lit -sv test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test
55-
$ lit -sv test/std/atomics test/std/threads # Test std::thread and std::atomic
32+
$ cd <monorepo-root>
33+
$ <build>/bin/llvm-lit -sv libcxx/test/std/re # Run all of the std::regex tests
34+
$ <build>/bin/llvm-lit -sv libcxx/test/std/depr/depr.c.headers/stdlib_h.pass.cpp # Run a single test
35+
$ <build>/bin/llvm-lit -sv libcxx/test/std/atomics libcxx/test/std/threads # Test std::thread and std::atomic
5636
5737
Sometimes you'll want to change the way LIT is running the tests. Custom options
5838
can be specified using the `--param=<name>=<val>` flag. The most common option
@@ -62,30 +42,54 @@ that. However if you want to manually specify the option like so:
6242

6343
.. code-block:: bash
6444
65-
$ lit -sv test/std/containers # Run the tests with the newest -std
66-
$ lit -sv --param=std=c++03 test/std/containers # Run the tests in C++03
45+
$ <build>/bin/llvm-lit -sv libcxx/test/std/containers # Run the tests with the newest -std
46+
$ <build>/bin/llvm-lit -sv libcxx/test/std/containers --param=std=c++03 # Run the tests in C++03
6747
6848
Occasionally you'll want to add extra compile or link flags when testing.
6949
You can do this as follows:
7050

7151
.. code-block:: bash
7252
73-
$ lit -sv --param=compile_flags='-Wcustom-warning'
74-
$ lit -sv --param=link_flags='-L/custom/library/path'
53+
$ <build>/bin/llvm-lit -sv libcxx/test --param=compile_flags='-Wcustom-warning'
54+
$ <build>/bin/llvm-lit -sv libcxx/test --param=link_flags='-L/custom/library/path'
7555
7656
Some other common examples include:
7757

7858
.. code-block:: bash
7959
8060
# Specify a custom compiler.
81-
$ lit -sv --param=cxx_under_test=/opt/bin/g++ test/std
61+
$ <build>/bin/llvm-lit -sv libcxx/test/std --param=cxx_under_test=/opt/bin/g++
8262
8363
# Enable warnings in the test suite
84-
$ lit -sv --param=enable_warnings=true test/std
64+
$ <build>/bin/llvm-lit -sv libcxx/test --param=enable_warnings=true
8565
8666
# Use UBSAN when running the tests.
87-
$ lit -sv --param=use_sanitizer=Undefined
67+
$ <build>/bin/llvm-lit -sv libcxx/test --param=use_sanitizer=Undefined
68+
69+
Using a custom site configuration
70+
---------------------------------
71+
72+
By default, the libc++ test suite will use a site configuration that matches
73+
the current CMake configuration. It does so by generating a ``lit.site.cfg``
74+
file in the build directory from the ``libcxx/test/lit.site.cfg.in`` template,
75+
and pointing ``llvm-lit`` (which is a wrapper around ``llvm/utils/lit/lit.py``)
76+
to that file. So when you're running ``<build>/bin/llvm-lit``, the generated
77+
``lit.site.cfg`` file is always loaded first, followed by the actual config in
78+
``libcxx/test/lit.cfg``. However, it is sometimes desirable to use a custom
79+
site configuration. To do that, you can use ``--param=libcxx_site_config`` or
80+
the ``LIBCXX_SITE_CONFIG`` environment variable to point to the right site
81+
configuration file. However, you must stop using ``llvm-lit``, or else the
82+
generated ``lit.site.cfg`` will still be preferred:
83+
84+
.. code-block:: bash
85+
86+
$ LIBCXX_SITE_CONFIG=path/to/your/site/configuration llvm/utils/lit/lit.py -sv ...
87+
88+
$ llvm/utils/lit/lit.py -sv ... --param=libcxx_site_config=path/to/your/site/configuration
8889
90+
In both of these cases, your custom site configuration should set up the
91+
``config`` object in a way that is compatible with what libc++'s ``config.py``
92+
module expects.
8993

9094
LIT Options
9195
===========

0 commit comments

Comments
 (0)