-Builder 0.9 Readme
+Builder
+Copyright © 2006-2022 Mikkosoft Productions
+Version 3.0 readme
+-------------------------------------------------------------------------------
+
+Contents
+
+1. Description
+2. Building Builder
+3. Using Builder
+4. Locating packages
+5. Command line options
+6. Virtual targets
+7. Tarballs
+8. Architectures and cross-compilation
+9. Dependency analysis
+10. Logging
+11. Writing Build files
+12. Contact information
+13. License
+
+-------------------------------------------------------------------------------
1. Description
Builder is a program for building other programs, similar to make, scons and
others. Its main design goals are minimizing duplication of configuration
between packages and being as fast as possible. Features include integrated
-configuration and dependency resolution for C and C++ sources.
+configuration, dependency resolution for C and C++ sources, multi-package
+builds, cross-compilation and pkg-config integration.
+-------------------------------------------------------------------------------
2. Building Builder
-Builder is normally built using itself. However, if you just downloaded the
-source and don't yet have a Builder binary, how is that possible? To resolve
-this problem, there's a script called bootstrap.sh in the Builder main
-directory. Before running it, make sure you have the following libraries
-available:
+Builder is normally built using itself. This creates a conundrum if you just
+downloaded the source and don't yet have a Builder binary. To resolve the
+problem, there's a script called bootstrap.sh in the Builder main directory.
+Before running it, make sure you have the following libraries available:
-MSP libraries: core datafile fs strings io
+MSP libraries: core crypto datafile
Others: sigc++-2.0
-Since the MSP libraries are also normally built with Builder, the script will
-need to have their sources available. By default, it will look at the parent
-directory of builder. You can change this by setting the LIBPATH evironment
-variable for the script. If everything goes well, the script will build a
+You should be able to get the MSP libraries from the same place you got
+Builder itself from. Since they are also normally built with Builder, the
+script will need to have their sources available. By default, it will look at
+the parent directory of builder. You can change this with the --libpath
+parameter for the script. If everything goes well, the script will build a
builder-stage1 binary and then proceed to build a normal version with it.
+-------------------------------------------------------------------------------
-3. Using Builder to build projects
+3. Using Builder
In the simplest and most common case, Builder is invoked in the package's root
-directory without any arguments. An arbitary working directory may be
-specified with -C, or an alternative Build file with -f.
+directory without any arguments. It will look for a file called Build and
+load it. An alternative file can be specified with the -f option. The first
+package loaded from this file is called the main package and will determine
+what to build by default.
+
+While loading the package, other packages may be pulled in through require
+statements. Refer to section 4 for how packages are located. A package that
+can't be found will cause the build to fail. If the -x option was specified,
+builder will only consider binary packages when looking for dependencies.
+
+When building, the primary targets of each package are placed in the package's
+root directory. To build a subset of targets from the package, the desired
+targets can be named on the command line. Paths are interpreted relative to
+Builder's working directory. The analyzer can be useful in finding out what
+targets are available; see section 9 for details.
+
+To configure optional package features, arguments of the form key=value may be
+given. To obtain a list of available features, use the --help option in the
+package directory.
+
+Temporary files produced during the build are stored in a dedicated temporary
+directory. By default a directory named temp is created for each package, but
+this can be changed with the --tempdir option. Subdirectories are used to
+separate files for different build types and architectures.
+
+When multiple packages are involved in a build, Builder will need to install
+files for other packages to use them. By default these go in $HOME/local, but
+this can be changed with the --prefix option.
+
+-------------------------------------------------------------------------------
+
+4. Locating packages
+
+Most packages are not self-contained, but depend on other packages. When such
+dependencies are encountered, the packages need to be located. This section
+describes the various places Builder looks for them from.
+
+First, Builder asks pkg-config if it knows the location of the source
+directory of the package. This is stored in the source variable in the .pc
+file, and is automatically put there by Builder. If the variable is found,
+Builder further verifies that it refers to an existing directory which
+contains a Build file, so that removed sources do not cause undue failures.
+
+If pkg-config does not know about the source, Builder consults its own package
+path. Currently this is hardcoded to contain the main package's source
+directory and its parent directory. To be found this way, the directory
+should be named the same as the package, optionally followed by a dash and a
+version number (or really anything; the part after the last dash is ignored).
+
+If the source can't be found at all, Builder will then turn to using a binary
+package. Builder's native packages are considered first. They are normally
+stored in <prefix>/share/builder with the extension bpk. The base part of the
+filename should be the as the package's name.
+
+Finally, pkg-config is consulted for the build flags of the package in the
+normal way. The flags need to be translated to Builder's internal
+representation, so if the package requires very unusual flags, it may not
+work.
+
+A package that can't be found after all these steps might as well not exist.
+
+-------------------------------------------------------------------------------
+
+5. Command line options
+
+When both Builder and GNU make offer the same functionality, the command line
+option to invoke it is the same as well.
+
+-a, --analyze MODE
+ Perform dependency analysis. See section 9 for details.
+
+-b, --build
+ Perform a build. This is the default action. This option can be used to
+ build even of some other action is specified.
+
+-c, --clean
+ Clean buildable targets. Any dependencies are cleaned up as well. Giving
+ this option twice cleans all packages.
+
+-f, --file=FILE
+ Read build instructions from FILE. The default is Build.
+
+-h, --help
+ Display a summary of command line options as well as configuration for the
+ main package.
+
+-j, --jobs NUM
+ Run up to NUM tasks in parallel. It's not very useful to specify a number
+ far in excess of the amount of logical CPU cores in the system.
+
+-l, --log LIST
+ Enable output from a comma-separated list of log channels. This is mainly
+ useful for debugging Builder itself; for normal use the -v option is almost
+ always enough. Refer to section 10 for the available log channels.
+
+-n, --dry-run
+ Show what would be done without actually doing it. This is useful in
+ combination with -W or -l to find out what Builder would do in different
+ situations.
+
+-s, --silent
+ Don't print any messages other than errors.
+
+-t, --build-type TYPE
+ Sets the build type. The supported types are defined in the builderrc file.
+ The shipped configuration contains build types debug, optimized_debug,
+ release and static_release.
+
+-v, --verbose
+ Print more information about what's going on. This option may be specified
+ multiple times to enable progressively more output. Internally it translates
+ to enabling predefined sets of log channels.
+
+-x, --no-externals
+ Do not load external source packages.
+
+-A, --conf-all
+ Apply configuration to all packages. Normally, only the main package is
+ configured, but with this option all packages can be configured at once.
+
+-B, --build-all
+ Build all targets, even if their dependencies haven't been modified.
+
+-C, --chdir DIR
+ Change to DIR before doing anything else. This is equivalent to doing cd DIR
+ before invoking builder. It's of little practical value in a shell, but may
+ be useful in scripts to avoid cd commands. All filenames on the command line
+ will be interpreted relative to this directory.
+
+-P, --progress
+ Display progress while building.
+
+-W, --what-if FILE
+ Pretend that FILE has changed.
+
+--arch ARCH
+ Build for architecture ARCH. If the architecture is not native, this will
+ result in cross-compilation. See section 8 for details.
+
+--conf-only
+ Stop after configuring packages. This can be useful if you want to
+ configure a package being used in a multi-package build.
+
+--full-paths
+ Output full paths in analysis. Normally only target names are displayed.
+
+--max-depth NUM
+ Show up to NUM levels in analysis. Depths less than 3 are generally not very
+ useful. The default is 4.
+
+--prefix DIR
+ Install files to DIR. The default is $HOME/local.
+
+--tempdir DIR
+ Store temporary files in DIR. If a relative path is specified, each package
+ will have its own temp directory. The default is ./temp.
+
+-------------------------------------------------------------------------------
+
+6. Virtual targets
+
+Builder provides virtual targets for certain tasks, as well as for internal
+organization of the build graph. They can appear on the command line along
+with file-based targets.
+
+world
+ Depends on everything that can be built.
+
+default
+ Depends on the targets that are built by default. If no targets are
+ specified on the command line, then this target will be built.
+
+install
+ Depends on all installed files. This also includes .pc files, which are
+ otherwise not normally built.
+
+tarballs
+ Depends on source tarballs of all packages.
+
+goals
+ Depends on all targets on the command line. Trying to add this target to the
+ command line will cause Builder to abort due to a circular dependency.
+
+-------------------------------------------------------------------------------
+
+7. Tarballs
+
+Since the source tree usually contains files that do not belong to a release
+archive, be they version control metadata or temporary build files, Builder
+provides a way to create a clean source tarball for a package. It will
+contain all non-buildable files belonging to the package, including its Build
+file.
+
+Source tarballs are named as <package>-<version>.tar. Currently compression
+is not supported.
+
+-------------------------------------------------------------------------------
+
+8. Architectures and cross-compilation
+
+If the necessary compilers and other tools are available, Builder can cross-
+compile binaries for other systems. To activate cross-compilation, use the
+--arch option.
+
+An architecture specification consists of cpu type and model, bitness and
+operating system. The fields are separated by dashes and can appear in any
+order. A partial architecture specification can also be given; any missing
+fields are filled in from the native architecture.
-To build a subset of the targets known to the package, the desired targets can
-be named on the command line. Note that if -C is used, the targets will be
-interpreted relative to that directory, not the one where Builder is invoked
-from.
+The --arch option can also be used to build native binaries optimized for a
+specific cpu model. Simply specify the desired model as an architecture. By
+default, no cpu model is used, which will generally result in code that can be
+executed by all cpus of the same type.
-To change package configuration, arguments of the form key=value may be given.
-By default they will only affect the current package, but with the -A switch,
-all active packages may be configured at once.
+In cross-compile mode, the build results are stored in a directory named after
+the architecture. Similarly, the default prefix is $HOME/local/$ARCH.
-By default, files will be installed in $HOME/local. If you are not happy with
-this, the --prefix switch may be used to use another directory. Note that
-files may be installed due to dependencies when building multiple packages at
-once even if you don't specify the install target.
+-------------------------------------------------------------------------------
+9. Dependency analysis
-4. Using Builder in your project
+Builder can perform a dependency analysis in various ways. This can provide
+useful insight to the relations between targets. It may also be helpful if
+you suspect Builder isn't picking up all of your files.
+
+The following analysis modes are available:
+
+deps
+ Display most targets, but skip over those that have a predictable 1-to-1
+ relationship with their sources. Currently this includes ObjectFiles and
+ InstalledFiles.
+
+alldeps
+ Display all targets.
+
+rebuild
+ Display all targets, but stop descending as soon as a target not in need of
+ rebuilding is encountered.
+
+rdeps
+ Instead of displaying the dependencies of the targets on the command line,
+ display which targets depend on them. This provides similar information as
+ builder -a rebuild -W target world, but displayed in a different way.
+
+-------------------------------------------------------------------------------
+
+10. Logging
+
+Sometimes you need to figure out what Builder is doing, perhaps because it's
+doing it wrong or you're just curious. To this end, a number of logging
+channels are provided. To specify which channels to enable, use the -l
+option. Some channels are also turned on by the -v option; the default
+verbosity level is 1. The -s option disables all channels.
+
+auxcommands
+ Prints commands that are not directly related to building targets, such as
+ pkg-config invocations. Enabled at verbosity level 3.
+
+cache
+ Displays cached values as they are loaded. Generates a lot of spam.
+
+commands
+ Prints commands used for building targets. Internal commands will print
+ "<internal>". Enabled at verbosity level 2.
+
+configure
+ Shows conditionals encountered in package Build files and whether they
+ matched.
+
+environment
+ Displays overall parameters used in the build. Enabled at verbosity level 2.
+
+files
+ Indicates files that are being read or written, as well as directory scans.
+ Enabled at verbosity level 3.
+
+packagemgr
+ Shows what's being done to locate packages.
+
+packages
+ Displays a list of active packages at startup, together with their target
+ counts if they are source packages. Enabled at verbosity level 2.
+
+rebuild
+ Shows why targets are being rebuilt. Similar to -a rebuild.
+
+summary
+ Displays a brief summary of what will be done. Enabled by default.
+
+tasks
+ Displays abstract descriptions of what is being done. Enabled by default.
+
+tools
+ Displays information related to tools, such as versions.
+
+vfs
+ Shows what files are being looked for and where. Generates a lot of spam.
+
+-------------------------------------------------------------------------------
+
+11. Writing Build files
Before going into details about Builder instruction files, it's beneficial to
understand the basic concepts used in Builder. There are three kinds of major
Targets are the individual files taking part in the build process. They are
created automatically from components.
-The canonical name for the insturction file is Build. It is written in a
+The canonical name for the instruction file is Build. It is written in a
structured declarative language, with syntax somewhat resembling the C
language. The basic unit is a statement, which consists of a keyword,
optional arguments and an optional block of sub-statements. Each statement is
require "otherlib";
-pkg-config is used to locate dependencies, and failing that, some heuristics
-are applied to find Builder-enabled packages.
+See section 4 for how packages are located.
To be useful, a package must define one or more components. At present, there
-are five types:
+are four types:
- program: Build an executable program
- library: Build a library (both shared and static)
- module: Build a module for a program
-- tarball: Pack files up in a tarbal
-- includes: Install headers for a library
+- tarball: Pack files up in a .tar file
-Each of these statements takes the component name as an argument. The target
-name is composed of the component name and a possible platform-specific prefix
-and suffix. For example, the statement:
+Each of these statements takes the component name as an argument. The final
+target name is composed of the component name and a possible platform-specific
+prefix and suffix. For example, the statement:
- library "mylib";
+ library "mylib"
+ {
+ ...
+ };
Will create a file called libmylib.so on Linux and libmylib.dll on Windows.
-For headers component, the name defines a subdirectory of the prefix include
-directory to install headers in.
Components must contain one or more source statements telling where sources
are found:
source "mylib.cpp";
Directories may be specified; each file inside the directory will be added to
-the source list. The list is filtered according to the component type and only
-applicable files are used.
+the source list. Subdirectories are not included. The list is filtered
+according to the component type and only applicable files are used.
-If you want a component to be installed, you can specify so:
+If you want a component to be installed, you can state it:
install true;
The default is not to install any components. The installation directory is
-automatically determined from the component type.
+automatically determined from the component type. Library components will also
+install any headers which were used to build the library.
+
+Components can have requirements, just like packages. They can also use
+library components from the same package:
+
+ use "mylib_common";
+
+This can be useful in organizing code when multiple components in a package
+share a common part. If the used library is not specified to be installed,
+it will be linked statically.
-For a complete example, see Builder's own Build file.
+Packages may want to offer optional features, for example to allow the user to
+choose whether to use a particular external library:
+ feature "foolib"
+ {
+ description "Use foolib to perform magic";
+ };
+
+Enabled features will automatically cause a macro to be defined when compiling
+C or C++ code. The macro consists of a WITH_ prefix followed by the feature
+name in uppercase. The above feature would define the macro WITH_FOOLIB.
+
+To specify additional effects of a feature, such as required packages, a
+conditional statement may be used:
+
+ if_feature "foolib"
+ {
+ ...
+ };
+
+Any statements inside the conditional block are evaluated if the feature is
+enabled. This can be negated with a !. Conditionals can appear at package and
+component scopes and may contain anything that the enclosing statement can.
+
+When making cross-platform software, it's often necessary to use different
+libraries on different platforms. Another kind of conditional can be used for
+this:
+
+ if_arch "windows"
+ {
+ ...
+ };
+
+See section 8 for more information on how to specify an architecture. The
+conditional block is evaluated if all parts in the condition match the target
+architecture.
-5. Contact information
+Sometimes a library doesn't support pkg-config and can't be pulled in through a
+require statement. This is commonly the case with system libraries. The
+build_info statement can be used for this:
+
+ build_info
+ {
+ library "gdi32";
+ };
+
+In many cases these libraries are specific to that system, so it's best to wrap
+them in an if_arch statement.
+
+Another case for using build_info is if you want to keep your headers separate
+from source files:
+
+ incpath "include";
+
+Paths are interpreted relative to the package's root directory. It is not
+recommended to use absolute paths, as this can make your package difficult to
+build for others.
+
+To include additional files in the package's source tarball, such as
+documentation, use the source_tarball statement. It takes no argument, as the
+name is generated internally, but otherwise it behaves as a tarball component.
+
+For real-life examples, see the Build files in Builder itself or any of the
+MSP libraries.
+
+-------------------------------------------------------------------------------
+
+12. Contact information
The latest releases can be found at http://www.tdb.fi/builder.shtml
Bug reports, feature requests etc. may be sent to tdb@tdb.fi
-Read-only SVN access is available at http://svn.tdb.fi/builder
+Read-only git access is available at git://git.tdb.fi/builder
+-------------------------------------------------------------------------------
-6. License
+13. License
-Builder is copyright © 2006-2009 Mikko Rasa, Mikkosoft Productions
+Builder is copyright © 2006-2022 Mikko Rasa, Mikkosoft Productions
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Full license text can be found in License.txt.
-
-
-7. Version history
-
-0.9
-- First released version
projects / builder.git / blobdiff