samien
/
boost
mirror of https://github.com/boostorg/boost


			
				
					
						
						
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247
							<html>

<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Microsoft FrontPage 5.0">
<title>Boost Regression Test User Documentation</title>
</head>

<body bgcolor="#FFFFFF" text="#000000">

<table border="1" cellpadding="2" bgcolor="#007F7F">
    <tr>
        <td bgcolor="#FFFFFF"><img src="../boost.png"
        alt="boost.png (6897 bytes)" width="277" height="86"></td>
        <td><a href="../index.htm"><font color="#FFFFFF" size="4"
        face="Arial,Helvetica">Home</font></a></td>
        <td><a href="../libs/libraries.htm"><font color="#FFFFFF"
        size="4" face="Arial,Helvetica">Libraries</font></a></td>
        <td><a href="../people/people.htm"><font color="#FFFFFF"
        size="4" face="Arial,Helvetica">People</font></a></td>
        <td><a href="faq.htm"><font color="#FFFFFF" size="4"
        face="Arial,Helvetica">FAQ</font></a></td>
        <td><a href="index.htm"><font color="#FFFFFF" size="4"
        face="Arial,Helvetica">More</font></a></td>
    </tr>
</table>

<h1>Boost Regression Test User Documentation</h1>

<p><a href="#Introduction">Introduction</a><br>
<a href="#Preparation">Preparation</a><br>
<a href="#Execution">Execution</a><br>
<a href="#Adding_new_test">Adding a new test</a><br>
<a href="#Requirements">Requirements</a><br>
<a href="#History">History</a></p>

<h2><a name="Introduction">Introduction</a></h2>

<p>Boost's internal regression test suite produces the <a
href="../status/compiler_status.html">compiler status tables</a>.
</p>

<p>Although not ordinarily run by Boost library users, it is
documented here for the benefit of Boost developers, and for
Boost users porting to a new platform.</p>

<p>Version 3 of the Boost regression testing framework is based
on <a href="../tools/build/index.html">Boost.Build</a>, and uses <i>bjam</i>
to actually run the tests. Because Boost.Build does dependency
analysis, only tests for which some dependency has change are
rerun.</p>

<p>The reporting of test results as HTML files is accomplished by
separate <a href="../tools/regression/index.htm">regression
reporting programs</a> which process the residue and log files
from the <i>bjam</i> run.</p>

<h2><a name="Preparation">Preparation</a></h2>

<p>Install the following programs on your system, in some
location suitable for program executables. Normally that location
must be in a directory which is part of your search path for
executables.</p>

<ul>
    <li><i>bjam</i> - Executables and sources are available; see <a
        href="../tools/build/index.html#Jam">Boost.Build docs</a>.&nbsp;
        Before worrying about regression tests, you might want to
        verify your <i>bjam</i> installation works by <a
        href="../tools/build/index.html#Building">building the
        boost-root/status/Jamfile libraries</a>.<br>
        &nbsp;</li>
    <li><i>process_jam_log</i> and <i>compiler_status</i> -
    <a href="../tools/regression/index.htm">Sources and Jamfile</a> are available
        for these, too. </li>
</ul>

<h2><a name="Execution">Execution</a></h2>

<p>A complete shell script, <a href="../tools/regression/run_tests.sh">
run_test.sh</a>, is available for POSIX systems. For Windows, please read the 
following.</p>

<p>These examples assume several environment variables have been
set:</p>

<ul>
    <li>BOOST_ROOT is set to the directory where the unzipped
        Boost distribution or CVS working copy resides.&nbsp; For
        example:<br>
        <br>
        set BOOST_ROOT=/boost_1_29_0<br>
        &nbsp;</li>
    <li>TOOLS is set to the <a
        href="../tools/build/index.html#Tools">compiler toolsets</a>
        you wish to use. For example:<br>
        <br>
        set TOOLS=borland gcc intel-win32 metrowerks vc7<br>
        &nbsp;</li>
    <li>Any environment variables required for particular
        compilers.&nbsp; You might wish to test first with one
        compiler at a time to make sure each toolset is fully
        operational.</li>
</ul>

<p>Just running bjam for a specific library can be run like this on Windows, 
using the Filesystem Library as an example:</p>

<blockquote>
  <pre>cd \boost-root\libs\filesystem\test
bjam</pre>
</blockquote>

<p>A full set of tests and status tables can be run thusly
on a Windows system:</p>

<blockquote>
    <pre>cd %BOOST_ROOT%\status
bjam --dump-tests test &gt;bjam.log 2&gt;&amp;1
start notepad bjam.log
process_jam_log &lt;bjam.log
compiler_status %BOOST_ROOT% cs-win32.html
rem Specify links file, even though it will be overwritten, so report html includes links
rem This works because the generated bookmark names are the same regardless of other settings
compiler_status --ignore-pass --no-warn %BOOST_ROOT% cs-win32-fail.html cs-win32-links.html
compiler_status --ignore-pass %BOOST_ROOT% cs-win32-warn-or-fail.html cs-win32-links.html
compiler_status %BOOST_ROOT% cs-win32-full.html cs-win32-links.html</pre>
</blockquote>

<p>Modulo syntax adjustments, the same procedure should work on
other operating systems (an example a <a href="../tools/regression/run_tests.sh">
UNIX shell script</a> for this is provided, see the
script itself for comments and directions).&nbsp; Rename the cs-win-xxx
output files as appropriate.</p>

<p>If you execute <i>compiler_status</i> without arguments, you
can see the available options and tailor your own favorite report.</p>

<p>If you want to run just a single test, specify it as the <i>bjam</i>
target rather than &quot;test&quot;.&nbsp; For example, to debug
configurations, it might be useful to just run the <i>config_info</i>
test, with a switch to force even up-to-date programs be rebuilt:</p>

<blockquote>
    <pre>bjam -a --dump-tests config_info &gt;bjam.log 2&gt;&amp;1
process_jam_log &lt;bjam.log
compiler_status %BOOST_ROOT% cs-win32.html
</pre>
</blockquote>

<h2><a name="Adding_new_test">Adding a new test</a></h2>

<h3>The best way - the <a name="subinclude"> <i>subinclud</i>e</a> approach</h3>

<p>The best way to add a test to the <a
href="../status/Jamfile">boost-root/status/Jamfile</a> is usually to use the bjam&nbsp;
<i>subinclude</i> feature, since it allows the same test specification to be run 
either as part of the overall Boost regression test or as a standalone test of 
only the library involved.</p>

<p>The test is specified in a Jamfile, which is usually placed in the library's
<i>test</i> subdirectory. See the <a href="../libs/filesystem/test/Jamfile">
Filesystem Library's test Jamfile</a> for an example, including the boilerplate 
wrapped around the actual tests. By <a href="#Execution">executing bjam</a> 
(with or without the status reporting programs) in the library's <i>test</i> 
subdirectory, the library can be tested without the need to run the full Boost 
regression tests.</p>

<p>Then by adding a <i>subinclude</i> statement to <a
href="../status/Jamfile">boost-root/status/Jamfile</a>, the library's test 
Jamfile will become part of the main Boost regression tests.&nbsp; For example, 
the Filesystem Library's tests are included by this line in the main 
boost-root/status Jamfile:</p>

<blockquote>
  <pre>subinclude libs/filesystem/test ;</pre>
</blockquote>

<p>Don't forget that Jamfiles are white-space sensitive; delete the
space before the semi-colon in the example above, and you will
get an introduction to Jam error messages.</p>

<h3>Another way - direct inclusion in Boost regression tests</h3>

<p>Adding a test is as simple as adding a single line to the <a
href="../status/Jamfile">boost-root/status/Jamfile</a>:</p>

<blockquote>
    <pre>run libs/mylib/test/mytest.cpp ;</pre>
</blockquote>

<p>For creating more complex tests and test-suites, use the
<a href="#subinclude">subinclude approach</a>. Look at
examples in the <a
href="../status/Jamfile">boost-root/status/Jamfile</a>.&nbsp;
The <i>bind</i> and <i>config</i> test-suites are simple
examples, while the <i>regex</i> and <i>threads</i> test suites
show more of the power of Jam based testing.</p>

<h2><a name="Requirements">Requirements</a></h2>

<p>The test suite has been designed to meet to the following
requirements. </p>

<ul>
    <li>Doesn't depend on an external tool chain. This requirement
        is met by supplying all tools in the regular <a
        href="getting_started.html#Download">Boost distribution</a>, except for a
        C++ compiler.</li>
    <li>Tools are written in C++; it is the only language that
        all Boost developers and users are comfortable with.</li>
    <li>Supports tests which expect an error </li>
    <li>Configuration is independent of the target platform or
        compiler. </li>
    <li>HTML output </li>
</ul>

<p>These requirements rule out any script-based approach such as
dejagnu (requires Tcl and expect) or even shell scripts. </p>

<h2><a name="History">History</a></h2>

<p>The version 3 Regression Test system is based on
<a href="../tools/build/index.html">Boost.Build</a>. The&nbsp; testing.jam and status/Jamfile foundation was
contributed by Dave Abrahams. Rene Rivera contributed several pieces. The post-bjam processing programs
were contributed by Beman Dawes.</p>

<p>The version 2 regression.cpp test program was contributed by
Jens Maurer, generalizing and improving an earlier version 1
program by Beman Dawes.</p>

<hr>

<p>Revised <!--webbot bot="Timestamp" startspan s-type="EDITED"
s-format="%d %B, %Y" -->08 January, 2004<!--webbot bot="Timestamp"
i-checksum="38708" endspan --></p>

<p>© Copyright <a href="../people/jens_maurer.htm">Jens
Maurer</a> 2001<br>
© Copyright <a href="../people/beman_dawes.html">Beman Dawes</a> 2001</p>
<p> Use, modification, and distribution are subject to the Boost Software 
License, Version 1.0. (See accompanying file <a href="../LICENSE_1_0.txt">
LICENSE_1_0.txt</a> or copy at <a href="http://www.boost.org/LICENSE_1_0.txt">
www.boost.org/LICENSE_1_0.txt</a>)</p>
</body>
</html>