c++boost.gif (8819 bytes)

Boost Build System

Synopsis

Boost.Build is a system for large project software construction built on FTJam, an open-source make replacement[1]. Key features are:

Here are some of the design criteria that led to these features.

Table of Contents

Getting Started

Installing FTJam

Initiating a Build

Currently, the build system must be invoked with the -fallyourbase-path option, where allyourbase-path is the path to the allyourbase.jam file supplied in this directory. When the system matures, this file may be compiled directly into Jam as the Jambase or we will find another way to make sure the -f option is no longer needed. The environment variables necessary for bootstrapping Jam are not needed once it has been built.

Here are some sample Boost Jam invocations:
Command Line(s) Effects
jam -fallyourbase-path -sTOOLS=gcc my_target
default (debug) BUILD of my_targetwith GCC
jam -fallyourbase-path -sTOOLS="msvc gcc"
default-build all with msvc and gcc
set TOOLS=msvc
jam -fallyourbase-path
Set an NT environment variable to always build with MSVC
default-build all.
jam -fallyourbase-path -sBUILD=release
release build all with default TOOLS:
jam -fallyourbase-path -sBUILD="debug release"
debug and release build all.

Setting Jam Variables

The "-s" options in the command lines above are passing variable settings to the build system. There are actually three ways to do that:

An Example Jamfile

Here is an example of a simple subproject Jamfile. In this example, it is assumed that the user has set BOOST_ROOT, either as an environment variable, on the command-line or in the project's Jamrules file:
subproject foo/bar/baz ; # path to here from project root

# A static library called 'baz'
lib baz : baz1.cpp baz2.cpp # C++ sources
          parser/src/baz4.ll # Lex->C++ sources
          parser/src/baz5.yy  # Yacc->C++ sources
        : <include>$(BOOST_PARENT_DIRECTORY)  # Put boost in #include path
    ;

# An executable called 'test'
exe test : <lib>baz # use the 'baz' library
           baz_test.cpp   # C++ source
         : <include>$(BOOST_ROOT)
    ;

That's it! The build system takes care of the rest. If the you want to be able to build all subprojects from the project root directory, you can add a Jamfile at the root:

project-root ; # declare this to be the project root directory
# Read subproject Jamfiles
subinclude foo/bar/baz foo/bar/... ;
subinclude a/b/c ... ; # more subincludes

Support Files

To use the build system, the following must be located in your project's root directory, or in a directory specified in the BOOST_BUILD_INSTALLATION variable. It is usually convenient to specify the BOOST_BUILD_INSTALLATION in your project's Jamrules file. The Boost Jamrules file shows an example.
Filename(s) Meaning
toolset-tools.jam Feature-to-command-line mapping for toolset.
features.jam Abstract toolset feature descriptions.
boost-base.jam Boost build system-specific rule definitions.
unit-tests.jam Unit tests and assertions for boost Jam code.
The boost-base.jam file is temporary, and will eventually be compiled into our Jam executable.

Basic Design and Terminology

This section gives an overview of the way that the system works, outlining the system's capabilities and overall design. It also introduces the terminology and concepts neccessary to understand the sections on writing Jamfiles and command-line invocations.

Projects and Subprojects

A project is a source directory tree containing at least one Jamfile. The root directory of the project is known as the project root. The root directory of a project may contain a Jamrules file, which contains project-specific Jam code. If the Jamrules file is not present when Jam is invoked, a warning will be issued.

Subdirectories containing Jamfiles are called subproject directories. Each such Jamfile describes a subproject.

The build system installation directory is a directory containing Jam files describing compilers and build variants. The installation directory can be specified explicitly by setting the variable BOOST_BUILD_INSTALLATION. If the installation directory is not specified, it is the same as the project root, and BOOST_BUILD_INSTALLATION is set to refer to that directory.

Targets

Each Jamfile describes one or more main targets.

Each main target is an abstract description of one or more built targets which are expressions of the corresponding main target under particular compilers and build variants. Intermediate files such as .o/.obj files generated by compiling .cpp files as a consequence of building a main target are also referred to as built targets. The term build directory tree refers to the location of built target files.

For each main target, there is a corresponding location in the build directory tree known as the target's build root, where all intermediate and final targets resulting from that main target are located.

Features and Properties

A feature is a normalized (toolset-independent) description of an individual build parameter, such as whether inlining is enabled. Each feature usually corresponds to a command-line option of one or more build tools. Features come in three varieties:

  1. Simple features can take on any of several predetermined values. For example, the feature optimization might take one of the values off, speed, or space. Simple features have a default value. The key aspect of simple features is that they are assumed to affect link compatibility: object files generated with different values for a simple feature are generated into a separate directories, and (with a few exceptions) main targets generated with different values won't be linked together.
  2. Free features can either be single-valued, as above, or may take on any number of user-specified values simultaneously. For example, the define feature for a release build might have the values NDEBUG and BOOST_RELEASE_BUILD. Free features are assumed not to affect link compatibility.
  3. Path features are free features whose values describe paths which may be relative to the subproject (such as linked libraries or #include search directories). The build system treats the values of these features specially to ensure that they are interpreted relative to the subproject directory regardless of the directory where Jam was invoked.
  4. Dependency features are path features whose values describe a dependency of built targets. For example, an external library might be specified with a dependency-feature: if the library is updated, the target will be updated also. The <library-file> feature works this way [2].

A feature-value pair is known as a build property, or simply property. The prefixes simple, free, path, and dependency apply to properties in an analogous way to features.

Build Variants

A build variant, or simply variant is a named set of build properties describing how targets should be built. Typically you'll want at least two separate variants: one for debugging, and one for your release code.

Built targets for distinct build variants and toolsets are generated in separate parts of the build directory tree, known as the variant directories. For example, a (sub)project with main targets foo and bar, compiled with both GCC and KAI for debug and release variants might generate the following structure (target directories in bold).

 bin
 +-foo  <--- foo's build root
 | +-gcc
 | | +-debug
 | | `-release
 | `-kai
 |   +-debug
 |   `-release
 `-bar  <--- bar's build root
   +-gcc
   | +-debug
   | `-release
   `-kai
     +-debug
     `-release

The properties constituting a variant may differ according to toolset, so debug may mean a slightly different set of properties for two different compilers.

Subvariants

When a target is built with simple properties that don't exactly match those specified in a build variant, the non-matching features are called subvariant features and the target is located in a subvariant directory beneath the directory of the base variant. This can occur for two reasons:

  1. Some features are only relevant to certain compilers. When relevant simple features have no value specified in the build variant, a value must be chosen. Even when the default value is used, the target is generated into a subvariant directory. For example, the runtime-link feature may be unspecified in the debug variant, but relevant to MSVC. In that case, a fragment of the target tree might look like:
     bin
     +-foo  <--- foo's build root
     | +-msvc
     | | +-debug
     . . . `-runtime-link-dynamic
     . . .
    
    Because the default value of runtime-link is dynamic, when the debug variant is requested, the runtime-link-dynamic subvariant of foo is built.

  2. It is possible to request (either on the command-line, or as part of a main target description) that particular subvariants be built. For example, it may be desirable to generate builds that link to the runtime both statically and dynamically. In that case, both subvariant directories in the example above would be generated:
     bin
     +-foo  <--- foo's build root
     | +-msvc
     | | +-debug
     . . . +-runtime-link-dynamic
     . . . `-runtime-link-static
     . . .
    
In no case will targets be built directly into bin/foo/msvc/debug, since the debug variant doesn't include the runtime-link feature, which is relevant to MSVC.

When a subvariant includes multiple subvariant features, targets are built into a subvariant directory whose path is determined by concatenating the properties sorted in order of their feature names. For example, the borland compiler, which uses different libraries depending on whether the target is a console or GUI program, might create the following structure for a DLL:

 bin
 +-foo  <--- foo's build root
 | +-msvc
 | | +-debug
 | | | +-runtime-link-dynamic
 | | | | +-user-interface-console
 | | | | `-user-interface-gui
 . . . `-runtime-link-static
 . . .   +-user-interface-console
 . . .   `-user-interface-gui

Any configuration of properties for which a target is built, whether base variant or subvariant, is known as a build configuration, or simply a build.

Dependent Targets

When a main target depends on the product of a second main target (as when an executable depends on and links to a static library), each build configuration of the dependent target is depends on the corresponding build of the dependency. Because only simple features participate in build identity, the dependent and dependency targets may have completely different free features. This puts the onus on the user for ensuring link-compatibility when certain free properties are used. For example, when assert() is used in header files, the preprocessor symbol NDEBUG can impact link-compatibility of separate compilation units. This danger can be minimized by encapsulating such feature differences inside of build variants.

Usage

This section describes how to start a build from the command-line and how to write project and subproject Jamfiles. It also describes the other files written in the Jam language: build-tool specification files, feature descriptions files.

The Command Line

This section describes in detail how the build system can be invoked.

User Targets

The Jam command line ends with an optional list of target names; if no target names are supplied, the built-in pseudotarget all is built. In a large project, naming targets can be dicey because of collisions. Jam uses a mechanism called grist to distinguish targets that would otherwise have the same name. Fortunately, you won't often have to supply grist at the command-line. When you declare a main target, a Jam pseudotarget of the same name is created which depends on all of the subvariants requested for your invocation of the build system. For example, if your subproject declares:

exe my_target : my_source1.cpp my_source2.c ;
and you invoke Jam with -sBUILD="debug release" my_target, you will build both the debug and release versions of my_target.

These simple, ungristed names are called user targets, and are only available for the subproject where Jam is invoked. That way, builds from the top level (which may include many Jamfiles through the subinclude rule) and builds of library dependencies (which may live in other subprojects), don't collide. If it is neccessary to refer more explicitly to a particular target from the command-line, you will have to add ``grist''. Please see this section for a more complete description of how to name particular targets in a build.

Global Variables

This is a partial list of global variables that can be set on the command-line. Of course you are free to write your own Jam rules which interpret other variables from the command-line. This list just details some of the variables used by the build system itself. Note also that if you don't like the default values you can override them in your project's Jamrules file.
Variable Default Example Notes
TOOLS Platform-dependent -sTOOLS="gcc msvc" build with gcc and msvc
-sTOOLS=gcc build with gcc
BUILD debug -sBUILD=release build the release variant
-sBUILD="debug release" build both debug and release variants
-sBUILD="<optimization>speed" build a subvariant of the default variant (debug) with optimization for speed.
-sBUILD="debug release <runtime-link>static/dynamic" build subvariants of the debug and release variants that link to the runtime both statically and dynamically.
ALL_LOCATE_TARGET empty -sALL_LOCATE_TARGET=~/build Generate all build results in the build subdirectory of the user's home directory (Unix).

SubProject Jamfiles

This section describes how to write a Jamfile for a subproject.

The subproject rule

A subproject's Jamfile begins with an invocation of the subproject rule that specifies the subproject's location relative to the top of the project tree:

subproject path-from-top ;

The subproject rule tells the build system where to place built targets from the subproject in case ALL_LOCATE_TARGET is used to specify the build directory tree. If there is a Jamfile in the project root directory, you should use the project-root rule instead:

project-root ;

Describing Main Targets

A main target is described using the following syntax:

target-type name : sources
    [ : requirements [ : default-BUILD ] ] ;

NOTE: for simple features in both requirements and default-BUILD, more-specific qualification overrides less-specific.

Example

This artificially complex example shows how an executable called "foo" might be described in a Jamfile. The executable is composed of the sources ./foo.cpp and ./src/bar.cpp (specified relative to the directory in which the Jamfile resides), and the built target which results from building the target baz as described in ../bazlib/Jamfile.

exe foo : foo.cpp src/bar.cpp <lib>../bazlib/baz
    ## Requirements ##
    : <include>../bazlib/include 
      <define>BUILDING_FOO=1
      <release><define>FOO_RELEASE
      <msvc><*><define>FOO_MSVC
      <msvc><release><define>FOO_MSVC_RELEASE
      <gcc><*><optimization>off
      <gcc><release><optimization>space
      <threading>multi
    
    ## default-BUILD ##
    : debug release
      <debug><runtime-link>static/dynamic
    ;

The requirements section:

The default-BUILD section:

Feature Descriptions

Features are described by stating the feature type (simple features are specified with "feature"), followed by the feature name. An optional second argument can be used to list the permissible values of the feature. Examples can be found in features.jam.

Variant Descriptions

Variants are described with the following syntax:

variant name : [<toolset-name>]<feature>value... ;
The variant rule specifies the list of properties comprising a variant. Properties may be optionally qualified with a toolset name, which specifies that the property applies only to that toolset. Examples can be found in features.jam.

Toolset Description Files

Toolset descriptions are located in the project's root directory, or a directory specified by BOOST_BUILD_INSTALLATION, which may be set in a Jamfile or the project's Jamrules file. Each file is called toolset-name-tools.jam, where toolset-name is the name of the toolset. The toolset description file has two main jobs:

  1. redefine the following rules: These rules should simply invoke the action part of a rule whose name is uniquely defined for the toolset. For example,
    rule C++-action
    {
        msvc-C++-action $(<) : $(>) ;
    }
    
    actions msvc-C++-action
    {
        cl -nologo -GX -c -U$(UNDEFS) -D$(DEFINES) $(CFLAGS) $(C++FLAGS) -I$(HDRS) -I$(STDHDRS) -Fo$(<) -Tp$(>)
    }
    
    
    Note that Link-action may require special care: on platforms where the global variable gEXPORT_SUFFIX(DLL) is defined (e.g. Windows), the first argument may have two elements when linking a shared library. The first is the shared library target, and the second is the import library target, with suffix given by $(gEXPORT_SUFFIX(DLL)). It will always have a third argument which is either ``EXE'' or ``DLL''. This can be used to dispatch to different actions for linking DLLs and EXEs if neccessary, but usually it will be easier to take advantage of the special <target-type> feature, which will have the same value using the flags rule described below.
  2. Translate build settings given in the global gBUILD_PROPERTIES variable into something that can be used by the toolset. The build system provides the flags rule to help translate build properties into elements of global variables which are later attached to targets so that they can affect the build actions. The flags rule is used as follows:
    flags toolset variable condition [: value...]
    
    The parameters are:

    Semantics only affect targets built with the specified toolset, and depend on the target's build configuration:

    1. if any specified property-set is a subset of the target's build properties, the values specified in $(3) will be appended once to variable.
    2. The value of each specified feature that participates in the target's build properaties is appended to variable. In either case, the variable will be set "on" the target so it may be used in the build actions.

    Example

    The description of the flags rule above is actually more complicated than it sounds. For example, the following line might be used to specify how optimization can be turned off for MSVC:

    flags msvc CFLAGS <optimization>off : /Od ;
    
    It says that the string /Od should be added to the global CFLAGS variable whenever a build configuration includes the property <optimization>off.

    Similarly, in the following example,

    flags msvc CFLAGS <runtime-build>release/<runtime-link>dynamic : /MD ;
    
    we add /MD to the CFLAGS variable when both of the specified conditions are satisfied. We could grab all of the values of the free feature <include> in the HDRS variable as follows:
    flags msvc HDRS <include> ;
    

    The use of these variables should be apparent from the declaration of actions msvc-C++-action in the previous section.

    Internals

    Jam Fundamentals

    This section is derived from the official Jam documentation and from my experience using it and reading the Jambase rules. I repeat the information here mostly because it is essential to understanding and using Jam, but is not consolidated in a single place. Some of it is missing from the official documentation altogether. I hope it will be useful to anyone wishing to become familiar with Jam and the Boost build system.

    Please also read The Jam language reference for the additional details, and the Jam release notes for a brief description of recent, but fundamental changes to the Jam language without which you will probably not understand any of the build system code. In particular, note that the return statement does not affect control flow.

    Target Names

    In addition to user targets, which correspond directly to the names the user writes in her subproject Jamfile, several additional targets are generated, regardless of the directory from which Jam was invoked:

    Global Variables

    This section describes some of the global variables used by the build system. Please note that some parts of the system (particularly those in allyourbase.jam) are heavily based on the Jambase file supplied with Jam, and as such do not follow the conventions described below.

    Global variables used in the build system fall into three categories:

    Please note that the build system commonly takes advantage of Jam's Dynamic Scoping feature (see the local command in the "Flow of Control" section below the link target) to temporarily "change" a global variable by declaring a local of the same name.

    Variables Associated with SubProject Identity

    Grist Variables

    Design Criteria

    Assumptions

    The requirements are driven by several basic assumptions:

    Requirements

    This build system was designed to satisfy the following requirements:

    Footnotes

    [1] FTJam is a variant of Jam/MR. It is hoped that crucial features we rely on from FTJam will eventually be incorportated into the official Jam release.

    [2]Note: right now, a dependency feature of a main target makes all resulting built targets dependent, including intermediate targets. That means that if an executable is dependent on an external library, and that library changes, all the sources comprising the executable will be recompiled as well. This behavior should probably be fixed.


    © Copyright David Abrahams 2001. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.

    Revised 29 July, 2001