Builder
-Copyright © 2006 Mikko Rasa, Mikkosoft Productions
-Version 0.1
+Copyright © 2006-2022 Mikkosoft Productions
+Version 3.0 readme
+-------------------------------------------------------------------------------
+
+Contents
+
+1. Description
+2. Building Builder
+3. Using Builder
+4. Locating packages
+5. Command line options
+6. Virtual targets
+7. Tarballs
+8. Architectures and cross-compilation
+9. Dependency analysis
+10. Logging
+11. Writing Build files
+12. Contact information
+13. License
+
+-------------------------------------------------------------------------------
+
+1. Description
Builder is a program for building other programs, similar to make, scons and
-others. It is specifically designed to suit my needs, with the goal of
-minimizing the amount of per-package configuration.
+others. Its main design goals are minimizing duplication of configuration
+between packages and being as fast as possible. Features include integrated
+configuration, dependency resolution for C and C++ sources, multi-package
+builds, cross-compilation and pkg-config integration.
+-------------------------------------------------------------------------------
-* Building Builder
+2. Building Builder
-Builder is normally built using itself. However, if you just downloaded the
-source and don't yet have a Builder binary, how is that possible? To resolve
-this problem, there's a script called bootstrap.sh in the Builder main
-directory. Before running it, make sure you have the following libraries
-available:
+Builder is normally built using itself. This creates a conundrum if you just
+downloaded the source and don't yet have a Builder binary. To resolve the
+problem, there's a script called bootstrap.sh in the Builder main directory.
+Before running it, make sure you have the following libraries available:
-MSP libraries: core parser path strings
+MSP libraries: core crypto datafile
Others: sigc++-2.0
-Since the MSP libraries are also normally built with Builder, the script will
-need to have their sources available. By default, it will look at the parent
-directory of builder. You can change this by setting the LIBPATH evironment
-variable for the script. If everything goes well, you should have a
-builder-stage1 binary that you can use to build a normal version of Builder.
+You should be able to get the MSP libraries from the same place you got
+Builder itself from. Since they are also normally built with Builder, the
+script will need to have their sources available. By default, it will look at
+the parent directory of builder. You can change this with the --libpath
+parameter for the script. If everything goes well, the script will build a
+builder-stage1 binary and then proceed to build a normal version with it.
+
+-------------------------------------------------------------------------------
+
+3. Using Builder
+
+In the simplest and most common case, Builder is invoked in the package's root
+directory without any arguments. It will look for a file called Build and
+load it. An alternative file can be specified with the -f option. The first
+package loaded from this file is called the main package and will determine
+what to build by default.
+
+While loading the package, other packages may be pulled in through require
+statements. Refer to section 4 for how packages are located. A package that
+can't be found will cause the build to fail. If the -x option was specified,
+builder will only consider binary packages when looking for dependencies.
+
+When building, the primary targets of each package are placed in the package's
+root directory. To build a subset of targets from the package, the desired
+targets can be named on the command line. Paths are interpreted relative to
+Builder's working directory. The analyzer can be useful in finding out what
+targets are available; see section 9 for details.
+
+To configure optional package features, arguments of the form key=value may be
+given. To obtain a list of available features, use the --help option in the
+package directory.
+
+Temporary files produced during the build are stored in a dedicated temporary
+directory. By default a directory named temp is created for each package, but
+this can be changed with the --tempdir option. Subdirectories are used to
+separate files for different build types and architectures.
+
+When multiple packages are involved in a build, Builder will need to install
+files for other packages to use them. By default these go in $HOME/local, but
+this can be changed with the --prefix option.
+
+-------------------------------------------------------------------------------
+
+4. Locating packages
+
+Most packages are not self-contained, but depend on other packages. When such
+dependencies are encountered, the packages need to be located. This section
+describes the various places Builder looks for them from.
+
+First, Builder asks pkg-config if it knows the location of the source
+directory of the package. This is stored in the source variable in the .pc
+file, and is automatically put there by Builder. If the variable is found,
+Builder further verifies that it refers to an existing directory which
+contains a Build file, so that removed sources do not cause undue failures.
+
+If pkg-config does not know about the source, Builder consults its own package
+path. Currently this is hardcoded to contain the main package's source
+directory and its parent directory. To be found this way, the directory
+should be named the same as the package, optionally followed by a dash and a
+version number (or really anything; the part after the last dash is ignored).
+
+If the source can't be found at all, Builder will then turn to using a binary
+package. Builder's native packages are considered first. They are normally
+stored in <prefix>/share/builder with the extension bpk. The base part of the
+filename should be the as the package's name.
+
+Finally, pkg-config is consulted for the build flags of the package in the
+normal way. The flags need to be translated to Builder's internal
+representation, so if the package requires very unusual flags, it may not
+work.
+
+A package that can't be found after all these steps might as well not exist.
+
+-------------------------------------------------------------------------------
+
+5. Command line options
+
+When both Builder and GNU make offer the same functionality, the command line
+option to invoke it is the same as well.
+
+-a, --analyze MODE
+ Perform dependency analysis. See section 9 for details.
+
+-b, --build
+ Perform a build. This is the default action. This option can be used to
+ build even of some other action is specified.
+
+-c, --clean
+ Clean buildable targets. Any dependencies are cleaned up as well. Giving
+ this option twice cleans all packages.
+
+-f, --file=FILE
+ Read build instructions from FILE. The default is Build.
+
+-h, --help
+ Display a summary of command line options as well as configuration for the
+ main package.
+
+-j, --jobs NUM
+ Run up to NUM tasks in parallel. It's not very useful to specify a number
+ far in excess of the amount of logical CPU cores in the system.
+
+-l, --log LIST
+ Enable output from a comma-separated list of log channels. This is mainly
+ useful for debugging Builder itself; for normal use the -v option is almost
+ always enough. Refer to section 10 for the available log channels.
+
+-n, --dry-run
+ Show what would be done without actually doing it. This is useful in
+ combination with -W or -l to find out what Builder would do in different
+ situations.
+
+-s, --silent
+ Don't print any messages other than errors.
+
+-t, --build-type TYPE
+ Sets the build type. The supported types are defined in the builderrc file.
+ The shipped configuration contains build types debug, optimized_debug,
+ release and static_release.
+
+-v, --verbose
+ Print more information about what's going on. This option may be specified
+ multiple times to enable progressively more output. Internally it translates
+ to enabling predefined sets of log channels.
+
+-x, --no-externals
+ Do not load external source packages.
+
+-A, --conf-all
+ Apply configuration to all packages. Normally, only the main package is
+ configured, but with this option all packages can be configured at once.
+
+-B, --build-all
+ Build all targets, even if their dependencies haven't been modified.
+
+-C, --chdir DIR
+ Change to DIR before doing anything else. This is equivalent to doing cd DIR
+ before invoking builder. It's of little practical value in a shell, but may
+ be useful in scripts to avoid cd commands. All filenames on the command line
+ will be interpreted relative to this directory.
+
+-P, --progress
+ Display progress while building.
+
+-W, --what-if FILE
+ Pretend that FILE has changed.
+
+--arch ARCH
+ Build for architecture ARCH. If the architecture is not native, this will
+ result in cross-compilation. See section 8 for details.
+
+--conf-only
+ Stop after configuring packages. This can be useful if you want to
+ configure a package being used in a multi-package build.
+
+--full-paths
+ Output full paths in analysis. Normally only target names are displayed.
+
+--max-depth NUM
+ Show up to NUM levels in analysis. Depths less than 3 are generally not very
+ useful. The default is 4.
+
+--prefix DIR
+ Install files to DIR. The default is $HOME/local.
+
+--tempdir DIR
+ Store temporary files in DIR. If a relative path is specified, each package
+ will have its own temp directory. The default is ./temp.
+
+-------------------------------------------------------------------------------
+
+6. Virtual targets
+
+Builder provides virtual targets for certain tasks, as well as for internal
+organization of the build graph. They can appear on the command line along
+with file-based targets.
+
+world
+ Depends on everything that can be built.
+
+default
+ Depends on the targets that are built by default. If no targets are
+ specified on the command line, then this target will be built.
+
+install
+ Depends on all installed files. This also includes .pc files, which are
+ otherwise not normally built.
+
+tarballs
+ Depends on source tarballs of all packages.
+
+goals
+ Depends on all targets on the command line. Trying to add this target to the
+ command line will cause Builder to abort due to a circular dependency.
+
+-------------------------------------------------------------------------------
+
+7. Tarballs
+
+Since the source tree usually contains files that do not belong to a release
+archive, be they version control metadata or temporary build files, Builder
+provides a way to create a clean source tarball for a package. It will
+contain all non-buildable files belonging to the package, including its Build
+file.
+
+Source tarballs are named as <package>-<version>.tar. Currently compression
+is not supported.
+
+-------------------------------------------------------------------------------
+
+8. Architectures and cross-compilation
+
+If the necessary compilers and other tools are available, Builder can cross-
+compile binaries for other systems. To activate cross-compilation, use the
+--arch option.
+
+An architecture specification consists of cpu type and model, bitness and
+operating system. The fields are separated by dashes and can appear in any
+order. A partial architecture specification can also be given; any missing
+fields are filled in from the native architecture.
+
+The --arch option can also be used to build native binaries optimized for a
+specific cpu model. Simply specify the desired model as an architecture. By
+default, no cpu model is used, which will generally result in code that can be
+executed by all cpus of the same type.
+
+In cross-compile mode, the build results are stored in a directory named after
+the architecture. Similarly, the default prefix is $HOME/local/$ARCH.
+
+-------------------------------------------------------------------------------
+
+9. Dependency analysis
+
+Builder can perform a dependency analysis in various ways. This can provide
+useful insight to the relations between targets. It may also be helpful if
+you suspect Builder isn't picking up all of your files.
+
+The following analysis modes are available:
+
+deps
+ Display most targets, but skip over those that have a predictable 1-to-1
+ relationship with their sources. Currently this includes ObjectFiles and
+ InstalledFiles.
+
+alldeps
+ Display all targets.
+
+rebuild
+ Display all targets, but stop descending as soon as a target not in need of
+ rebuilding is encountered.
+
+rdeps
+ Instead of displaying the dependencies of the targets on the command line,
+ display which targets depend on them. This provides similar information as
+ builder -a rebuild -W target world, but displayed in a different way.
+
+-------------------------------------------------------------------------------
+
+10. Logging
+
+Sometimes you need to figure out what Builder is doing, perhaps because it's
+doing it wrong or you're just curious. To this end, a number of logging
+channels are provided. To specify which channels to enable, use the -l
+option. Some channels are also turned on by the -v option; the default
+verbosity level is 1. The -s option disables all channels.
+
+auxcommands
+ Prints commands that are not directly related to building targets, such as
+ pkg-config invocations. Enabled at verbosity level 3.
+
+cache
+ Displays cached values as they are loaded. Generates a lot of spam.
+
+commands
+ Prints commands used for building targets. Internal commands will print
+ "<internal>". Enabled at verbosity level 2.
+
+configure
+ Shows conditionals encountered in package Build files and whether they
+ matched.
+
+environment
+ Displays overall parameters used in the build. Enabled at verbosity level 2.
+
+files
+ Indicates files that are being read or written, as well as directory scans.
+ Enabled at verbosity level 3.
+
+packagemgr
+ Shows what's being done to locate packages.
+
+packages
+ Displays a list of active packages at startup, together with their target
+ counts if they are source packages. Enabled at verbosity level 2.
+
+rebuild
+ Shows why targets are being rebuilt. Similar to -a rebuild.
+
+summary
+ Displays a brief summary of what will be done. Enabled by default.
+
+tasks
+ Displays abstract descriptions of what is being done. Enabled by default.
+
+tools
+ Displays information related to tools, such as versions.
+
+vfs
+ Shows what files are being looked for and where. Generates a lot of spam.
+
+-------------------------------------------------------------------------------
+
+11. Writing Build files
+
+Before going into details about Builder instruction files, it's beneficial to
+understand the basic concepts used in Builder. There are three kinds of major
+entities: packages, components and targets.
+
+Packages are distributable units of software.
+
+Components make up the contents of a package. They define the high-level
+goals such as libraries and executables.
+
+Targets are the individual files taking part in the build process. They are
+created automatically from components.
+
+The canonical name for the instruction file is Build. It is written in a
+structured declarative language, with syntax somewhat resembling the C
+language. The basic unit is a statement, which consists of a keyword,
+optional arguments and an optional block of sub-statements. Each statement is
+terminated with a semicolon.
+
+Typically, the Build file for a project defines a single package:
+
+ package "mylib"
+ {
+ ...
+ };
+
+Inside the package can be some informational statements:
+
+ version "0.1";
+ description "My awesome library";
+
+Packages may also depend on other packages:
+
+ require "otherlib";
+
+See section 4 for how packages are located.
+
+To be useful, a package must define one or more components. At present, there
+are four types:
+- program: Build an executable program
+- library: Build a library (both shared and static)
+- module: Build a module for a program
+- tarball: Pack files up in a .tar file
+
+Each of these statements takes the component name as an argument. The final
+target name is composed of the component name and a possible platform-specific
+prefix and suffix. For example, the statement:
+
+ library "mylib"
+ {
+ ...
+ };
+
+Will create a file called libmylib.so on Linux and libmylib.dll on Windows.
+
+Components must contain one or more source statements telling where sources
+are found:
+
+ source "mylib.cpp";
+
+Directories may be specified; each file inside the directory will be added to
+the source list. Subdirectories are not included. The list is filtered
+according to the component type and only applicable files are used.
+
+If you want a component to be installed, you can state it:
+
+ install true;
+
+The default is not to install any components. The installation directory is
+automatically determined from the component type. Library components will also
+install any headers which were used to build the library.
+
+Components can have requirements, just like packages. They can also use
+library components from the same package:
+
+ use "mylib_common";
+
+This can be useful in organizing code when multiple components in a package
+share a common part. If the used library is not specified to be installed,
+it will be linked statically.
+
+Packages may want to offer optional features, for example to allow the user to
+choose whether to use a particular external library:
+
+ feature "foolib"
+ {
+ description "Use foolib to perform magic";
+ };
+
+Enabled features will automatically cause a macro to be defined when compiling
+C or C++ code. The macro consists of a WITH_ prefix followed by the feature
+name in uppercase. The above feature would define the macro WITH_FOOLIB.
+
+To specify additional effects of a feature, such as required packages, a
+conditional statement may be used:
+
+ if_feature "foolib"
+ {
+ ...
+ };
+
+Any statements inside the conditional block are evaluated if the feature is
+enabled. This can be negated with a !. Conditionals can appear at package and
+component scopes and may contain anything that the enclosing statement can.
+
+When making cross-platform software, it's often necessary to use different
+libraries on different platforms. Another kind of conditional can be used for
+this:
+
+ if_arch "windows"
+ {
+ ...
+ };
+
+See section 8 for more information on how to specify an architecture. The
+conditional block is evaluated if all parts in the condition match the target
+architecture.
+
+Sometimes a library doesn't support pkg-config and can't be pulled in through a
+require statement. This is commonly the case with system libraries. The
+build_info statement can be used for this:
+
+ build_info
+ {
+ library "gdi32";
+ };
+
+In many cases these libraries are specific to that system, so it's best to wrap
+them in an if_arch statement.
+
+Another case for using build_info is if you want to keep your headers separate
+from source files:
+
+ incpath "include";
+
+Paths are interpreted relative to the package's root directory. It is not
+recommended to use absolute paths, as this can make your package difficult to
+build for others.
+
+To include additional files in the package's source tarball, such as
+documentation, use the source_tarball statement. It takes no argument, as the
+name is generated internally, but otherwise it behaves as a tarball component.
+
+For real-life examples, see the Build files in Builder itself or any of the
+MSP libraries.
+
+-------------------------------------------------------------------------------
+
+12. Contact information
+
+The latest releases can be found at http://www.tdb.fi/builder.shtml
+
+Bug reports, feature requests etc. may be sent to tdb@tdb.fi
+
+Read-only git access is available at git://git.tdb.fi/builder
+
+-------------------------------------------------------------------------------
+
+13. License
+Builder is copyright © 2006-2022 Mikko Rasa, Mikkosoft Productions
-* Using Builder in your project
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
-To use Builder, you need to create a Build file that tells what it should
-build. This file defines two main types of entities: packages and components.
-usually there is only one package definition in a Build file.
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
-The Build file uses a C-like structured language. It consists of statements
-and blocks. Statements are terminated with a semicolon (';'). Blocks are
-enclosed in braces ('{' and '}'). If a statement contains a block of
-substatements, the semicolon comes after the closing brace.
+You should have received a copy of the GNU General Public License along
+with this program; if not, write to the Free Software Foundation, Inc.,
+51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-For a simple example, look at Builder's own Build file.
+Full license text can be found in License.txt.
projects / builder.git / blobdiff