|
@@ -0,0 +1,248 @@
|
|
|
|
|
+<html>
|
|
|
|
|
+
|
|
|
|
|
+<head>
|
|
|
|
|
+<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
|
|
|
|
+
|
|
|
|
|
+<title>Boost Internal Regression Test Suite</title>
|
|
|
|
|
+</head>
|
|
|
|
|
+
|
|
|
|
|
+<body bgcolor="#FFFFFF" text="#000000">
|
|
|
|
|
+
|
|
|
|
|
+<table border="1" bgcolor="#007F7F" cellpadding="2">
|
|
|
|
|
+ <tr>
|
|
|
|
|
+ <td bgcolor="#FFFFFF"><img src="../c++boost.gif" alt="c++boost.gif (8819 bytes)" width="277" height="86"></td>
|
|
|
|
|
+ <td><a href="../index.htm"><font face="Arial,Helvetica" color="#FFFFFF"><big>Home</big></font></a></td>
|
|
|
|
|
+ <td><a href="../libs/libraries.htm"><font face="Arial,Helvetica" color="#FFFFFF"><big>Libraries</big></font></a></td>
|
|
|
|
|
+ <td><a href="../people/people.htm"><font face="Arial,Helvetica" color="#FFFFFF"><big>People</big></font></a></td>
|
|
|
|
|
+ <td><a href="../more/faq.htm"><font face="Arial,Helvetica" color="#FFFFFF"><big>FAQ</big></font></a></td>
|
|
|
|
|
+ <td><a href="../more/index.htm"><font face="Arial,Helvetica" color="#FFFFFF"><big>More</big></font></a></td>
|
|
|
|
|
+ </tr>
|
|
|
|
|
+</table>
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+<h1>Internal Regression Test Suite</h1>
|
|
|
|
|
+
|
|
|
|
|
+Boost's internal regression test suite produces the
|
|
|
|
|
+<a href="../status/compiler_status.html">compiler status tables</a>.
|
|
|
|
|
+
|
|
|
|
|
+<h2>Requirements</h2>
|
|
|
|
|
+
|
|
|
|
|
+The test suite has been design according to the following requirements.
|
|
|
|
|
+<ul>
|
|
|
|
|
+<li>Pure ISO C++, no tools required except a C++ compiler.
|
|
|
|
|
+<li>Support for tests which expect an error
|
|
|
|
|
+<li>Flexible configuration
|
|
|
|
|
+<li>HTML output
|
|
|
|
|
+</ul>
|
|
|
|
|
+
|
|
|
|
|
+These requirements rule out any script-based approach such as dejagnu
|
|
|
|
|
+(requires Tcl and expect) or even shell scripts.
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+<h2>Running Regression Tests</h2>
|
|
|
|
|
+
|
|
|
|
|
+The implementation is provided in a single source file named
|
|
|
|
|
+<a href="../status/regression.cpp">regression.cpp</a>.
|
|
|
|
|
+<p>
|
|
|
|
|
+You should be able to compile and link this file using whatever C++
|
|
|
|
|
+compiler is at your disposition. However, you may need to configure
|
|
|
|
|
+both the compiler and the standard library to use "strict" ISO
|
|
|
|
|
+compliance mode. Also, you need to extend the search path for include
|
|
|
|
|
+files with the main boost directory so that the header file
|
|
|
|
|
+<a href="../boost/config.hpp">boost/config.hpp</a>
|
|
|
|
|
+can be found. This header file is required to work around compiler
|
|
|
|
|
+deficiencies.
|
|
|
|
|
+<p>
|
|
|
|
|
+
|
|
|
|
|
+You can then start the resulting executable to run regression tests.
|
|
|
|
|
+By default, the regression test program reads the file "compiler.cfg"
|
|
|
|
|
+in the current directory to determine the list of compilers and their
|
|
|
|
|
+invocation syntax. After that, it reads the file "regression.cfg" in
|
|
|
|
|
+the current directory to determine the regression tests to run. The
|
|
|
|
|
+results of the regression tests are written in an HTML formatted text
|
|
|
|
|
+file. This file is by default named "cs-OS.html" in the current
|
|
|
|
|
+directory, where "OS" is a placeholder for the name of the operating
|
|
|
|
|
+system (e.g., "linux" or "win32").
|
|
|
|
|
+<p>
|
|
|
|
|
+
|
|
|
|
|
+To generate the
|
|
|
|
|
+<a href="../status/compiler_status.html">compiler status tables</a>,
|
|
|
|
|
+boost uses the files
|
|
|
|
|
+<a href="../status/compiler.cfg">status/compiler.cfg</a>
|
|
|
|
|
+and
|
|
|
|
|
+<a href="../status/regression.cfg">status/regression.cfg</a>.
|
|
|
|
|
+<p>
|
|
|
|
|
+
|
|
|
|
|
+The regression test program accepts some command-line options to alter
|
|
|
|
|
+its behavior.
|
|
|
|
|
+<p>
|
|
|
|
|
+<table border="1">
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>-h <em>or</em> --help</td>
|
|
|
|
|
+<td>prints a help message
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--config <em>file</em></td>
|
|
|
|
|
+<td>Use <em>file</em> instead of "compiler.cfg" as the compiler
|
|
|
|
|
+configuration file. This allows for private compiler setups.</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--tests <em>file</em></td>
|
|
|
|
|
+<td>Use <em>file</em> instead of "regression.cfg" as the tests
|
|
|
|
|
+configuration file. This allows individual libraries to specify
|
|
|
|
|
+additional tests not to be published in the main
|
|
|
|
|
+<a href="../status/compiler_status.html">compiler status tables</a>.
|
|
|
|
|
+</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--boost <em>path</em></td>
|
|
|
|
|
+<td>Use <em>path</em> as the filesystem path to the main boost
|
|
|
|
|
+directory. The default is "..", i.e. the parent directory.</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--output <em>file</em><br>-o <em>file</em></td>
|
|
|
|
|
+<td>Write the HTML output to <em>file</em> instead of the default
|
|
|
|
|
+"cs-OS.html".</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--compiler <em>name</em></td>
|
|
|
|
|
+<td>Run the tests only with compiler <em>name</em>. The <em>name</em>
|
|
|
|
|
+must be defined in the second line of an applicable compiler
|
|
|
|
|
+configuration (see below). The default is to run the tests with all
|
|
|
|
|
+compilers suitable for the platform.</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td>--diff</td>
|
|
|
|
|
+<td>Read the HTML output file before writing it. In the HTML output,
|
|
|
|
|
+highlight differences in test outcomes compared to the previous
|
|
|
|
|
+run.</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+
|
|
|
|
|
+<tr>
|
|
|
|
|
+<td><em>test</em></td>
|
|
|
|
|
+<td>Run only the named test. The syntax is the same as in the
|
|
|
|
|
+configuration file (see below).</td>
|
|
|
|
|
+</tr>
|
|
|
|
|
+</table>
|
|
|
|
|
+<p>
|
|
|
|
|
+When running only a selected test, you must also provide an alternate
|
|
|
|
|
+HTML output filename with "--output" so that the full test output is
|
|
|
|
|
+not accidentally overwritten.
|
|
|
|
|
+<p>
|
|
|
|
|
+
|
|
|
|
|
+You should redirect the output (<code>std::cout</code>) and error
|
|
|
|
|
+(<code>std::cerr</code>) channels to a suitable log file for later
|
|
|
|
|
+inspection.
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+<h2>Configuration Files</h2>
|
|
|
|
|
+
|
|
|
|
|
+In both configuration files, single-line comments starting with "//"
|
|
|
|
|
+at the leftmost column are ignored.
|
|
|
|
|
+
|
|
|
|
|
+<h3>Compiler Configuration</h3>
|
|
|
|
|
+
|
|
|
|
|
+The compiler configuration file can contain descriptions for an
|
|
|
|
|
+arbitrary number of compilers. Each compiler is configured by a block
|
|
|
|
|
+of six consecutive text lines.
|
|
|
|
|
+<ol>
|
|
|
|
|
+<li>Name of the operating system for which the entry is applicable
|
|
|
|
|
+(e.g., "linux" or "win32").
|
|
|
|
|
+<li>Name of the compiler; should be unique within one operating
|
|
|
|
|
+system. The name of the compiler should not contain the version
|
|
|
|
|
+number, because it is expected that regression tests are always run
|
|
|
|
|
+with the most recent compiler version available.
|
|
|
|
|
+<li>Name and version number of the compiler. This is printed on
|
|
|
|
|
+<code>std::cout</code> prior to running a test with that compiler.
|
|
|
|
|
+<li>Command-line invocation of the compiler to compile a single source
|
|
|
|
|
+file to an object file.
|
|
|
|
|
+<li>Command-line invocation of the compiler to compile a single source
|
|
|
|
|
+file to an executable.
|
|
|
|
|
+<li>Identification of the compiler for inclusion in the HTML output;
|
|
|
|
|
+may contain HTML tags such as <br>.
|
|
|
|
|
+</ol>
|
|
|
|
|
+
|
|
|
|
|
+The two command-lines are subject to the following substitutions:
|
|
|
|
|
+<ul>
|
|
|
|
|
+<li>Any string starting with a dollar symbol "$", then containing an
|
|
|
|
|
+arbitrary number of underscores, digits, or uppercase letters, is
|
|
|
|
|
+replaced by the value of the associated environment variable.
|
|
|
|
|
+<li>%include is replaced by the path given by the "--boost" command
|
|
|
|
|
+line option (".." by default). This should be used to extend the
|
|
|
|
|
+search path of your compiler so that the boost header files are found.
|
|
|
|
|
+<li>%source is the name of the source file to be compiled, qualified
|
|
|
|
|
+with the path given by the "--boost" command-line option (".." by
|
|
|
|
|
+default).
|
|
|
|
|
+</ul>
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+<h3>Test Configuration</h3>
|
|
|
|
|
+
|
|
|
|
|
+The test configuration file can contain descriptions for an arbitrary
|
|
|
|
|
+number of tests. Each test is described by a single line. The first
|
|
|
|
|
+word (up to the first space) is the type of the test, the next word
|
|
|
|
|
+gives the filename of the test relative the directory to be given by the
|
|
|
|
|
+"--boost" command-line option. Optionally, additional words are
|
|
|
|
|
+passed on as command-line arguments when the test is executed (only
|
|
|
|
|
+for types "run" and "run-fail"). In these arguments, "%boost" is
|
|
|
|
|
+replaced by the path given by the "--boost" command-line option (".."
|
|
|
|
|
+by default).
|
|
|
|
|
+<p>
|
|
|
|
|
+
|
|
|
|
|
+The following test types are available:
|
|
|
|
|
+<ul>
|
|
|
|
|
+<li>compile: The given test file should compile successfully.
|
|
|
|
|
+Otherwise, the test fails.
|
|
|
|
|
+<li>compile-fail: The given test file should not compile successfully,
|
|
|
|
|
+but instead the compiler should give an error message. If the test
|
|
|
|
|
+does compile successfully, the test fails.
|
|
|
|
|
+<li>link: The given test file should compile and link successfully.
|
|
|
|
|
+Otherwise, the test fails. In particular, a <code>main</code>
|
|
|
|
|
+function must be present in the test file.
|
|
|
|
|
+<li>link-fail: The given test file should not compile and link
|
|
|
|
|
+successfully. It is not specified whether the compile or the link
|
|
|
|
|
+should not succeed. If the test does compile and link successfully,
|
|
|
|
|
+the test fails.
|
|
|
|
|
+<li>run: The given test file should compile and link successfully.
|
|
|
|
|
+After that, the resulting executable is invoked and should return a
|
|
|
|
|
+zero exit code. If any of these steps fail, the test fails.
|
|
|
|
|
+<li>run-fail: The given test file is compiled, linked, and, if an
|
|
|
|
|
+executable was successfully generated, it is invoked. Either
|
|
|
|
|
+compiling or linking should fail or the resulting executable should
|
|
|
|
|
+return a non-zero exit code. If compiling and linking succeeds and
|
|
|
|
|
+the resulting executable returns a zero exit code, the test fails.
|
|
|
|
|
+</ul>
|
|
|
|
|
+
|
|
|
|
|
+<h2>Adapting for a New Platform</h2>
|
|
|
|
|
+
|
|
|
|
|
+In order to adapt the regression test suite for a new platform, a few
|
|
|
|
|
+changes to the <a href="../status/regression.cpp">regression.cpp</a>
|
|
|
|
|
+are required.
|
|
|
|
|
+
|
|
|
|
|
+<ul>
|
|
|
|
|
+<li>Add an appropriate check for your platform in
|
|
|
|
|
+<code>get_host()</code> and return a unique string identifying the
|
|
|
|
|
+platform (this string is used to filter the compile configuration
|
|
|
|
|
+file).
|
|
|
|
|
+<li>Verify that <code>get_system_configuration()</code> is
|
|
|
|
|
+appropriately defined for your platform. For a Unix platform, it most
|
|
|
|
|
+likely is. The function may return a verbose HTML string describing
|
|
|
|
|
+the platform.
|
|
|
|
|
+</ul>
|
|
|
|
|
+
|
|
|
|
|
+You also need to configure the compilers available on your platform in
|
|
|
|
|
+"compiler.cfg" as described above.
|
|
|
|
|
+<p>
|
|
|
|
|
+<hr>
|
|
|
|
|
+2001-01-30<br>
|
|
|
|
|
+<a href="../people/jens_maurer.htm">Jens Maurer</a>
|
|
|
|
|
+
|
|
|
|
|
+</body>
|
|
|
|
|
+</html>
|
Jens Maurer