Builder Copyright © 2006-2022 Mikkosoft Productions Version 3.0 readme ------------------------------------------------------------------------------- Contents 1. Description 2. Building Builder 3. Using Builder 4. Locating packages 5. Command line options 6. Virtual targets 7. Tarballs 8. Architectures and cross-compilation 9. Dependency analysis 10. Logging 11. Writing Build files 12. Contact information 13. License ------------------------------------------------------------------------------- 1. Description Builder is a program for building other programs, similar to make, scons and others. Its main design goals are minimizing duplication of configuration between packages and being as fast as possible. Features include integrated configuration, dependency resolution for C and C++ sources, multi-package builds, cross-compilation and pkg-config integration. ------------------------------------------------------------------------------- 2. Building Builder Builder is normally built using itself. This creates a conundrum if you just downloaded the source and don't yet have a Builder binary. To resolve the problem, there's a script called bootstrap.sh in the Builder main directory. Before running it, make sure you have the following libraries available: MSP libraries: core crypto datafile Others: sigc++-2.0 You should be able to get the MSP libraries from the same place you got Builder itself from. Since they are also normally built with Builder, the script will need to have their sources available. By default, it will look at the parent directory of builder. You can change this with the --libpath parameter for the script. If everything goes well, the script will build a builder-stage1 binary and then proceed to build a normal version with it. ------------------------------------------------------------------------------- 3. Using Builder In the simplest and most common case, Builder is invoked in the package's root directory without any arguments. It will look for a file called Build and load it. An alternative file can be specified with the -f option. The first package loaded from this file is called the main package and will determine what to build by default. While loading the package, other packages may be pulled in through require statements. Refer to section 4 for how packages are located. A package that can't be found will cause the build to fail. If the -x option was specified, builder will only consider binary packages when looking for dependencies. When building, the primary targets of each package are placed in the package's root directory. To build a subset of targets from the package, the desired targets can be named on the command line. Paths are interpreted relative to Builder's working directory. The analyzer can be useful in finding out what targets are available; see section 9 for details. To configure optional package features, arguments of the form key=value may be given. To obtain a list of available features, use the --help option in the package directory. Temporary files produced during the build are stored in a dedicated temporary directory. By default a directory named temp is created for each package, but this can be changed with the --tempdir option. Subdirectories are used to separate files for different build types and architectures. When multiple packages are involved in a build, Builder will need to install files for other packages to use them. By default these go in $HOME/local, but this can be changed with the --prefix option. ------------------------------------------------------------------------------- 4. Locating packages Most packages are not self-contained, but depend on other packages. When such dependencies are encountered, the packages need to be located. This section describes the various places Builder looks for them from. First, Builder asks pkg-config if it knows the location of the source directory of the package. This is stored in the source variable in the .pc file, and is automatically put there by Builder. If the variable is found, Builder further verifies that it refers to an existing directory which contains a Build file, so that removed sources do not cause undue failures. If pkg-config does not know about the source, Builder consults its own package path. Currently this is hardcoded to contain the main package's source directory and its parent directory. To be found this way, the directory should be named the same as the package, optionally followed by a dash and a version number (or really anything; the part after the last dash is ignored). If the source can't be found at all, Builder will then turn to using a binary package. Builder's native packages are considered first. They are normally stored in /share/builder with the extension bpk. The base part of the filename should be the as the package's name. Finally, pkg-config is consulted for the build flags of the package in the normal way. The flags need to be translated to Builder's internal representation, so if the package requires very unusual flags, it may not work. A package that can't be found after all these steps might as well not exist. ------------------------------------------------------------------------------- 5. Command line options When both Builder and GNU make offer the same functionality, the command line option to invoke it is the same as well. -a, --analyze MODE Perform dependency analysis. See section 9 for details. -b, --build Perform a build. This is the default action. This option can be used to build even of some other action is specified. -c, --clean Clean buildable targets. Any dependencies are cleaned up as well. Giving this option twice cleans all packages. -f, --file=FILE Read build instructions from FILE. The default is Build. -h, --help Display a summary of command line options as well as configuration for the main package. -j, --jobs NUM Run up to NUM tasks in parallel. It's not very useful to specify a number far in excess of the amount of logical CPU cores in the system. -l, --log LIST Enable output from a comma-separated list of log channels. This is mainly useful for debugging Builder itself; for normal use the -v option is almost always enough. Refer to section 10 for the available log channels. -n, --dry-run Show what would be done without actually doing it. This is useful in combination with -W or -l to find out what Builder would do in different situations. -s, --silent Don't print any messages other than errors. -t, --build-type TYPE Sets the build type. The supported types are defined in the builderrc file. The shipped configuration contains build types debug, optimized_debug, release and static_release. -v, --verbose Print more information about what's going on. This option may be specified multiple times to enable progressively more output. Internally it translates to enabling predefined sets of log channels. -x, --no-externals Do not load external source packages. -A, --conf-all Apply configuration to all packages. Normally, only the main package is configured, but with this option all packages can be configured at once. -B, --build-all Build all targets, even if their dependencies haven't been modified. -C, --chdir DIR Change to DIR before doing anything else. This is equivalent to doing cd DIR before invoking builder. It's of little practical value in a shell, but may be useful in scripts to avoid cd commands. All filenames on the command line will be interpreted relative to this directory. -P, --progress Display progress while building. -W, --what-if FILE Pretend that FILE has changed. --arch ARCH Build for architecture ARCH. If the architecture is not native, this will result in cross-compilation. See section 8 for details. --conf-only Stop after configuring packages. This can be useful if you want to configure a package being used in a multi-package build. --full-paths Output full paths in analysis. Normally only target names are displayed. --max-depth NUM Show up to NUM levels in analysis. Depths less than 3 are generally not very useful. The default is 4. --prefix DIR Install files to DIR. The default is $HOME/local. --tempdir DIR Store temporary files in DIR. If a relative path is specified, each package will have its own temp directory. The default is ./temp. ------------------------------------------------------------------------------- 6. Virtual targets Builder provides virtual targets for certain tasks, as well as for internal organization of the build graph. They can appear on the command line along with file-based targets. world Depends on everything that can be built. default Depends on the targets that are built by default. If no targets are specified on the command line, then this target will be built. install Depends on all installed files. This also includes .pc files, which are otherwise not normally built. tarballs Depends on source tarballs of all packages. goals Depends on all targets on the command line. Trying to add this target to the command line will cause Builder to abort due to a circular dependency. ------------------------------------------------------------------------------- 7. Tarballs Since the source tree usually contains files that do not belong to a release archive, be they version control metadata or temporary build files, Builder provides a way to create a clean source tarball for a package. It will contain all non-buildable files belonging to the package, including its Build file. Source tarballs are named as -.tar. Currently compression is not supported. ------------------------------------------------------------------------------- 8. Architectures and cross-compilation If the necessary compilers and other tools are available, Builder can cross- compile binaries for other systems. To activate cross-compilation, use the --arch option. An architecture specification consists of cpu type and model, bitness and operating system. The fields are separated by dashes and can appear in any order. A partial architecture specification can also be given; any missing fields are filled in from the native architecture. The --arch option can also be used to build native binaries optimized for a specific cpu model. Simply specify the desired model as an architecture. By default, no cpu model is used, which will generally result in code that can be executed by all cpus of the same type. In cross-compile mode, the build results are stored in a directory named after the architecture. Similarly, the default prefix is $HOME/local/$ARCH. ------------------------------------------------------------------------------- 9. Dependency analysis Builder can perform a dependency analysis in various ways. This can provide useful insight to the relations between targets. It may also be helpful if you suspect Builder isn't picking up all of your files. The following analysis modes are available: deps Display most targets, but skip over those that have a predictable 1-to-1 relationship with their sources. Currently this includes ObjectFiles and InstalledFiles. alldeps Display all targets. rebuild Display all targets, but stop descending as soon as a target not in need of rebuilding is encountered. rdeps Instead of displaying the dependencies of the targets on the command line, display which targets depend on them. This provides similar information as builder -a rebuild -W target world, but displayed in a different way. ------------------------------------------------------------------------------- 10. Logging Sometimes you need to figure out what Builder is doing, perhaps because it's doing it wrong or you're just curious. To this end, a number of logging channels are provided. To specify which channels to enable, use the -l option. Some channels are also turned on by the -v option; the default verbosity level is 1. The -s option disables all channels. auxcommands Prints commands that are not directly related to building targets, such as pkg-config invocations. Enabled at verbosity level 3. cache Displays cached values as they are loaded. Generates a lot of spam. commands Prints commands used for building targets. Internal commands will print "". Enabled at verbosity level 2. configure Shows conditionals encountered in package Build files and whether they matched. environment Displays overall parameters used in the build. Enabled at verbosity level 2. files Indicates files that are being read or written, as well as directory scans. Enabled at verbosity level 3. packagemgr Shows what's being done to locate packages. packages Displays a list of active packages at startup, together with their target counts if they are source packages. Enabled at verbosity level 2. rebuild Shows why targets are being rebuilt. Similar to -a rebuild. summary Displays a brief summary of what will be done. Enabled by default. tasks Displays abstract descriptions of what is being done. Enabled by default. tools Displays information related to tools, such as versions. vfs Shows what files are being looked for and where. Generates a lot of spam. ------------------------------------------------------------------------------- 11. Writing Build files Before going into details about Builder instruction files, it's beneficial to understand the basic concepts used in Builder. There are three kinds of major entities: packages, components and targets. Packages are distributable units of software. Components make up the contents of a package. They define the high-level goals such as libraries and executables. Targets are the individual files taking part in the build process. They are created automatically from components. The canonical name for the instruction file is Build. It is written in a structured declarative language, with syntax somewhat resembling the C language. The basic unit is a statement, which consists of a keyword, optional arguments and an optional block of sub-statements. Each statement is terminated with a semicolon. Typically, the Build file for a project defines a single package: package "mylib" { ... }; Inside the package can be some informational statements: version "0.1"; description "My awesome library"; Packages may also depend on other packages: require "otherlib"; See section 4 for how packages are located. To be useful, a package must define one or more components. At present, there are four types: - program: Build an executable program - library: Build a library (both shared and static) - module: Build a module for a program - tarball: Pack files up in a .tar file Each of these statements takes the component name as an argument. The final target name is composed of the component name and a possible platform-specific prefix and suffix. For example, the statement: library "mylib" { ... }; Will create a file called libmylib.so on Linux and libmylib.dll on Windows. Components must contain one or more source statements telling where sources are found: source "mylib.cpp"; Directories may be specified; each file inside the directory will be added to the source list. Subdirectories are not included. The list is filtered according to the component type and only applicable files are used. If you want a component to be installed, you can state it: install true; The default is not to install any components. The installation directory is automatically determined from the component type. Library components will also install any headers which were used to build the library. Components can have requirements, just like packages. They can also use library components from the same package: use "mylib_common"; This can be useful in organizing code when multiple components in a package share a common part. If the used library is not specified to be installed, it will be linked statically. Packages may want to offer optional features, for example to allow the user to choose whether to use a particular external library: feature "foolib" { description "Use foolib to perform magic"; }; Enabled features will automatically cause a macro to be defined when compiling C or C++ code. The macro consists of a WITH_ prefix followed by the feature name in uppercase. The above feature would define the macro WITH_FOOLIB. To specify additional effects of a feature, such as required packages, a conditional statement may be used: if_feature "foolib" { ... }; Any statements inside the conditional block are evaluated if the feature is enabled. This can be negated with a !. Conditionals can appear at package and component scopes and may contain anything that the enclosing statement can. When making cross-platform software, it's often necessary to use different libraries on different platforms. Another kind of conditional can be used for this: if_arch "windows" { ... }; See section 8 for more information on how to specify an architecture. The conditional block is evaluated if all parts in the condition match the target architecture. Sometimes a library doesn't support pkg-config and can't be pulled in through a require statement. This is commonly the case with system libraries. The build_info statement can be used for this: build_info { library "gdi32"; }; In many cases these libraries are specific to that system, so it's best to wrap them in an if_arch statement. Another case for using build_info is if you want to keep your headers separate from source files: incpath "include"; Paths are interpreted relative to the package's root directory. It is not recommended to use absolute paths, as this can make your package difficult to build for others. To include additional files in the package's source tarball, such as documentation, use the source_tarball statement. It takes no argument, as the name is generated internally, but otherwise it behaves as a tarball component. For real-life examples, see the Build files in Builder itself or any of the MSP libraries. ------------------------------------------------------------------------------- 12. Contact information The latest releases can be found at http://www.tdb.fi/builder.shtml Bug reports, feature requests etc. may be sent to tdb@tdb.fi Read-only git access is available at git://git.tdb.fi/builder ------------------------------------------------------------------------------- 13. License Builder is copyright © 2006-2022 Mikko Rasa, Mikkosoft Productions This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. Full license text can be found in License.txt.