Improve and expand the readme file a lot

[builder.git] / Readme.txt

Builder
Copyright © 2006-2012 Mikkosoft Productions
Version 2.0 readme

-------------------------------------------------------------------------------

Contents

1. Description
2. Building Builder
3. Using Builder
4. Command line options
5. Architectures and cross-compilation
6. Dependency analysis
7. Writing Build files
8. Contact information
9. License

-------------------------------------------------------------------------------

1. Description

Builder is a program for building other programs, similar to make, scons and
others.  Its main design goals are minimizing duplication of configuration
between packages and being as fast as possible.  Features include integrated
configuration, dependency resolution for C and C++ sources, multi-package
builds, cross-compilation and pkg-config integration.

-------------------------------------------------------------------------------

2. Building Builder

Builder is normally built using itself.  However, if you just downloaded the
source and don't yet have a Builder binary, how is that possible?  To resolve
this problem, there's a script called bootstrap.sh in the Builder main
directory.  Before running it, make sure you have the following libraries
available:

MSP libraries: core datafile

Others: sigc++-2.0

Since the MSP libraries are also normally built with Builder, the script will
need to have their sources available.  By default, it will look at the parent
directory of builder.  You can change this by setting the LIBPATH evironment
variable for the script.  If everything goes well, the script will build a
builder-stage1 binary and then proceed to build a normal version with it.

-------------------------------------------------------------------------------

3. Using Builder

In the simplest and most common case, Builder is invoked in the package's root
directory without any arguments.  It will look for a file called Build and
load it.  An alternative file can be specified with the -f option.  The first
package loaded from this file is called the main package and will determine
what to build by default.

While loading the package, other packages may be pulled in through require
statements.  Builder tries to locate the source for each required package by
consulting pkg-config and looking through its package path.  If the source
can't be found, a binary package is tried next, from pkg-config and Builder's
internal package database.  If the package still can't be found, the build
will fail.  Loading external source packages can be disabled with the -x
option.

When building, the primary targets of each package are placed in the package's
root directory.  To build a subset of targets from the package, the desired
targets can be named on the command line.  Paths are interpreted relative to
Builder's working directory.  The analyzer can be useful in finding out what
targets are available; see section 6 for details.

To configure optional package features, arguments of the form key=value may be
given.  To obtain a list of available features, use the --help option in the
package directory.

Temporary files produced during the build are stored in a dedicated temporary
directory.  By default a directory named temp is created for each package, but
this can be changed with the --tempdir option.  Subdirectories are used to
separate files for different build types and architectures.

When multiple packages are involved in a build, Builder will need to install
files for other packages to use them.  By default these go in $HOME/local, but
this can be changed with the --prefix option.

-------------------------------------------------------------------------------

4. Command line options

When both Builder and GNU make offer the same functionality, the command line
option to invoke it is the same as well.

-a, --analyze MODE
  Perform dependency analysis.  See section 6 for details.

-b, --build
  Perform a build.  This is the default action.  This option can be used to
  build even of some other action is specified.

-c, --clean
  Clean buildable targets.  Any dependencies are cleaned up as well.  Giving
  this option twice cleans all packages.

-f, --file=FILE
  Read build instructions from FILE.  The default is Build.

-h, --help
  Display a summary of command line options as well as configuration for the
  main package.

-j, --jobs NUM
  Run up to NUM tasks in parallel.  It's not very useful to specify a number
  far in excess of the amount of logical CPU cores in the system.

-l, --log LIST
  Enable output from a comma-separated list of log channels.

-n, --dry-run
  Show what would be done without actually doing it.  This is useful in
  combination with -W or -l to find out what Builder would do in different
  situations.

-s, --silent
  Don't print any messages other than errors.

-t, --build-type TYPE
  Sets the build type.  The supported types are defined in the builderrc file.
  The shipped configuration contains build types debug, optimized_debug,
  release and static_release.

-v, --verbose
  Print more information about what's going on.  This option may be specified
  multiple times to enable progressively more output.  Internally it translates
  to enabling predefined sets of log channels.

-x, --no-externals
  Do not load external source packages.

-A, --conf-all
  Apply configuration to all packages.  Normally, only the main package is
  configured, but with this option all packages can be configured at once.

-B, --build-all
  Build all targets, even if their dependencies haven't been modified.

-C, --chdir DIR
  Change to DIR before doing anything else.  This is equivalent to doing cd DIR
  before invoking builder.  It's of little practical value in a shell, but may
  be useful in scripts to avoid cd commands.  All filenames on the command line
  will be interpreted relative to this directory.

-P, --progress
  Display progress while building.

-W, --what-if FILE
  Pretend that FILE has changed.

--arch ARCH
  Build for architecture ARCH.  If the architecture is not native, this will
  result in cross-compilation.  See section 5 for details.

--conf-only
  Stop after configuring packages.  This can be useful if you want to
  configure a package being used in a multi-package build.

--full-paths
  Output full paths in analysis.  Normally only target names are displayed.

--max-depth NUM
  Show up to NUM levels in analysis.  Depths less than 3 are generally not very
  useful.  The default is 4.

--prefix DIR
  Install things to DIR.  The default is $HOME/local.

--tempdir DIR
  Store temporary files in DIR.  If a relative path is specified, each package
  will have its own temp directory.  The default is ./temp.

-------------------------------------------------------------------------------

5. Architectures and cross-compilation

If the necessary compilers and other tools are available, Builder can cross-
compile binaries for other systems.  To activate cross-compilation, use the
--arch option.

An architecture specification consists of cpu type and model, bitness and
operating system.  The fields are separated by dashes and can appear in any
order.  A partial architecture specification can also be given; any missing
fields are filled in from the native architecture.  

The --arch option can also be used to build native binaries optimized for a
specific cpu model.  Simply specify the desired model as an architecture.  By
default, no cpu model is used, which will generally result in code that can be
executed by all cpus of the same type.

In cross-compile mode, the build results are stored in a directory named after
the architecture.  Similarly, the default prefix is $HOME/local/$ARCH.

-------------------------------------------------------------------------------

6. Dependency analysis

Builder can perform a dependency analysis in various ways.  This can provide
useful insight to the relations between targets.  It may also be helpful if
you suspect Builder isn't picking up all of your files.

The following analysis modes are available:

deps
  Display most targets, but skip over those that have a predictable 1-to-1
  relationship with their sources.  Currently this includes ObjectFiles and
  InstalledFiles.

alldeps
  Display all targets.

rebuild
  Display all targets, but stop descending as soon as a target not in need of
  rebuilding is encountered.

rdeps
  Instead of displaying the dependencies of the targets on the command line,
  display which targets depend on them.  This provides similar information as
  builder -a rebuild -W target world, but displayed in a different way.

-------------------------------------------------------------------------------

7. Writing Build files

Before going into details about Builder instruction files, it's beneficial to
understand the basic concepts used in Builder.  There are three kinds of major
entities: packages, components and targets.

Packages are distributable units of software.

Components make up the contents of a package.  They define the high-level
goals such as libraries and executables.

Targets are the individual files taking part in the build process.  They are
created automatically from components.

The canonical name for the instruction file is Build.  It is written in a
structured declarative language, with syntax somewhat resembling the C
language.  The basic unit is a statement, which consists of a keyword,
optional arguments and an optional block of sub-statements.  Each statement is
terminated with a semicolon.

Typically, the Build file for a project defines a single package:

  package "mylib"
  {
    ...
  };

Inside the package can be some informational statements:

  version "0.1";
  description "My awesome library";

Packages may also depend on other packages:

  require "otherlib";

pkg-config is used to locate dependencies, and failing that, some heuristics
are applied to find Builder-enabled packages.

To be useful, a package must define one or more components.  At present, there
are four types:
- program: Build an executable program
- library: Build a library (both shared and static)
- module: Build a module for a program
- tarball: Pack files up in a .tar file

Each of these statements takes the component name as an argument.  The final
target name is composed of the component name and a possible platform-specific
prefix and suffix.  For example, the statement:

  library "mylib"
  {
    ...
  };

Will create a file called libmylib.so on Linux and libmylib.dll on Windows.

Components must contain one or more source statements telling where sources
are found:

  source "mylib.cpp";

Directories may be specified; each file inside the directory will be added to
the source list.  Subdirectories are not included.  The list is filtered
according to the component type and only applicable files are used.

If you want a component to be installed, you can state it:

  install true;

The default is not to install any components.  The installation directory is
automatically determined from the component type.  Library components will also
install any headers which were used to build the library.

Packages may want to offer optional features, for example to allow the user to
choose whether to use a particular external library:

  feature "foolib"
  {
    description "Use foolib to perform magic";
  };

Enabled features will automatically cause a macro to be defined when compiling
C or C++ code.  The macro consists of a WITH_ prefix followed by the feature
name in uppercase.  The above feature would define the macro WITH_FOOLIB.

To specify additional effects of a feature, such as required packages, a
conditional statement may be used:

  if_feature "foolib"
  {
    ...
  };

Any statements inside the conditional block are evaluated if the feature is
enabled.  This can be negated with a !.  Conditionals can appear at the package
level and may contain anything that the package statement can.

When making cross-platform software, it's often necessary to use different
libraries on different platforms.  Another kind of conditional can be used for
this:

  if_arch "windows"
  {
    ...
  };

See section 5 for more information on how to specify an architecture.  The
conditional block is evaluated if all parts in the condition match the target
architecture.

Sometimes a library doesn't support pkg-config and can't be pulled in through a
require statement.  This is commonly the case with system libraries.  The
build_info statement can be used for this:

  build_info
  {
    library "gdi32";
  };

In many cases these libraries are specific to that system, so it's best to wrap
them in an if_arch statement.

Another case for using build_info is if you want to keep your headers separate
from source files:

  incpath "include";

Paths are interpreted relative to the package's root directory.  It is not
recommended to use absolute paths, as this can make your package difficult to
build for others.

For real-life examples, see the Build files in Builder itself or any of the
MSP libraries.

-------------------------------------------------------------------------------

8. Contact information

The latest releases can be found at http://www.tdb.fi/builder.shtml

Bug reports, feature requests etc. may be sent to tdb@tdb.fi

Read-only git access is available at http://git.tdb.fi/builder

-------------------------------------------------------------------------------

9. License

Builder is copyright © 2006-2012  Mikko Rasa, Mikkosoft Productions

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.

Full license text can be found in License.txt.


7. Version history

0.9
- First released version