Overview
The MagicMakefileV3 is a new approach to portable c/c++ build systems, using gnu make, for C, C++, and Objective C languages on Posix, Linux, Mac-OSX, and Win32/Mingw32 systems using the GNU C/C++ Compiler.
Also read about the MagicMakefileV4 which is almost identical but less 'invasive' to your directory hierarchy!
Download
Download the magicmakefile from the SVN respository url: http://opensource.jdkoftinoff.com/jdks/svn/trunk/magicmakefile/branches/v3
Use the following command line:
svn export http://opensource.jdkoftinoff.com/jdks/svn/trunk/magicmakefile/branches/v3 magicmakefile
Why?
The Motivation
I have tried a number of existing build and configuration systems:
- GNU autoconf with GNU automake, GNU libtool all utilizing the GNU m4 macro processing system.
- ant
- jam
- cmake
- qmake
This article's purpose is not to criticise these existing projects, but I became frustrated when I had decided to use gnu autoconf and libtool. I purchased the book and found many of the examples to not work with the lastest versions of the software and found that I was spending way too much time tweaking macros and still was not getting a great result from my efforts.
A few of these systems work fine for many users - however I decided to 'make' my own systems since each of these existing project build systems fail me in at least two of the following:
- Micro-management of the project build process
- Too much complexity
- Too many compatibility problems on different platforms
- Too generic in ways that do not matter to me
- Poor or no support for cross compiling
- Poor bootstrapping process on new platforms
- Too many dependancies on specific versions of other tools
- Failure to manage header file dependancies
- Overuse of recursive configures/makes
My Needs
Really, my needs are simple.
Typically everything I write for a project goes into a static library. The only things not in the library are the single source files with int main( int argc, char **argv ) declared. These files need to be compiled and linked with the library.
I needed the ability to both compile on and cross compile for the following platforms:
- Mac-OS X (ppc,i386, and universal binaries) with GCC
- Win32 with cygwin or mingw32
- Linux, intel and embedded powerpc
I wanted the system to be automatic, in that if I added a source file to my 'src' directory, it should automatically get compiled into my library without any changes to any makefiles. This is a key point: When I start a new project I just want to make my "include,src,tests,tools,examples" directories, copy 'configure' and 'magic.mak' to it, make a project.mak file describing my project name and platform specific options for my project, and that's it. run configure, add source, and run make!
I also wanted automatic support for:
- header file dependancy management
- building in a separate directory from the source files, to allow building for multiple platforms at once
- 'make install'
- 'make docs' for automatic Doxygen docs building.
- building and running of tests via valgrind if possible.
- building of examples, tools, and gui programs
- using different source files for different platforms
- having special compiler and linker flags for different platforms
- creation of scripts to hold compiler and linker flags for the specific build, for use with additional projects.
- displaying current compiler options and directories used
I wanted the system to include a 'configure' script file which would be called in a similar fashion as the gnu autoconf configure scripts, but specifically did not try to run every test imaginable on the compiler. I feel that compiling code on different platforms requires more thought and testing than what a autoconf can do:
- Running these tests typically fail horribly when cross compiling as autoconf can not execute the test programs that it builds.
- Doing a universal binary compile on Mac-OSX means that a single 'config.h' is not sufficient as both platforms get compiled at the same time.
The Approach
After learning of all of the advanced features of GNU make, I realized something. If I restricted my build system to require GNU make, GNU GCC, and GNU bash, then these tools provided me with everything I needed to have a dynamic build system that fit my needs. The key is in the little-used-but-super-powerful gnu make features:
These functions together allow my magic.mak makefile to search in predefined directories for source files, and then dynamically build targets and build rules based on these dynamic lists. As long as certain policies are applied to the source code directory layout, the system can work its magic.
Policies and Patterns
I decided on the following policies of project directory and files organization:
- header files live in the include directory.
- library source files live in the src directory.
- command line tool programs live in the tools directory.
- command line test programs and shell scripts live in the tests directory.
- command line example programs live in the examples directory.
- gui programs live in the gui directory.
- all programs in the tools, tests, examples, and gui directories must be single source files which get linked with the library code.
In addition, each of the above source directories can contain subdirectories for each platform:
- posix
- linux
- mingw32
- cygwin
- macosx
- macosx-ppc
- macosx-i386
Note that every source file name must be uniquely named - ideally even if they live in different platform directories. It really reduces confusion when there is a problem.
The Implementation
The following files implement the magic make system and do not need to be modified for use in your own projects. They are commented fairly heavily:
configure is a bash script which sets up the variable defaults and creates a Makefile for you.
magic.mak is where all the magic happens.
The following files must be modified by you to include project specific information:
project.mak is read by the Makefile and contains your project specific compile options for various platforms.
project.sh is read by the configure script and can contain your own customizations of the configure process.
The Example Project
magic.mak and configure must live in the same directory as your custom project.mak and project.sh files.
Step 1 : Download
Download/copy the files From the svn repository into a new directory:
Make sure the configure is marked at executable:
chmod +x configure
Step 2 : Edit project.mak
Edit the project.mak file and change at least the following variables:
- PROJECT
- PROJECT_NAME
- PROJECT_VERSION
Step 3 : Make subdirs
Make the required subdirectories:
mkdir include src tests tools examples gui
Step 4 : Add code
- Put library source code in the src directory
- Put your header files in the include directory
- Put your command line tool program source files in the tools directory.
Make sure that:
- your tools are single source files
- your file names are unique across all directories
Ideally, I also prefer putting header files in their own subdirectory in the include directory.
Step 5 : make build directory
mkdir build cd build
Step 6 : run configure
Unlike gnu autoconf, this configure script is not automatic. It requires that you tell it what platform you are building for.
First, try asking it for help:
../configure --help
If you are running linux, try this:
../configure --target-platform-linux=1
The configure script handles the ---prefix parameter. Unlike the gnu autoconf tools, the prefix does not default to /usr/local - it defaults to install in your build directory.
You should keep each project's install directory separate if you change this.
Step 7 : run make
To see the list of files and compiler options that the magic makefile will process, run:
make compile_info
you may also run:
make help
to see other useful make targets.
Ultimately, you want to build your library and tools with:
make install make install-dev make install-dev-docs
A config tool script will automatically be created in your prefix dir's bin dir. if your project was named example, then there will be a script named example-cfg in the bin dir. You can use this for your own Makefiles to extract the necessary compiler flags. This script accepts the following options:
| --ldflags | print linker flags |
| --ldlibs | print linker libraries |
| --cflags | print c compiler flags |
| --cxxflags | print c++ compiler flags |
| --cppflags | print c preprocessor flags |
| --mflags | print objective c flags |
| --mmflags | print objective c++ flags |
Usage
configure
$ ../configure --help configure script based on J.D. Koftinoff Software's MagicMake system. See http://opensource.jdkoftinoff.com/jdks/trac/wiki/MagicMakefileV3 for more information example usage: Step 1: make a directory to put build results in: mkdir b Step 2: cd into this directory: cd b Step 3: run this configure script: ../configure Some example typical command line arguments for configure: Build native binaries on a generic posix machine: ../configure --target-platform-posix=1 Build native binaries on a linux machine: ../configure --target-platform-linux=1 Build native binaries on a mac os x machine: ../configure --target-platform-macosx=1 Build universal binaries on a mac os x machine: ../configure --target-platform-macosx-universal=1 Build on a mac os x machine and cross compile for windows via mingw32 cross compiler ../configure --native-platform-macosx-universal=1 --cross-compiling=1 --compiler-prefix=i386-mingw32- --target-platform-mingw32=1 Build on a linux machine and cross compile for windows via mingw32 cross compiler ../configure --native-platform-linux=1 --cross-compiling=1 --compiler-prefix=i386-mingw32- --target-platform-mingw32=1 Further options for installation path of 'make install' and 'make install-dev' destinations: ../configure --prefix=/opt/local without the --prefix option, the system defaults to $PWD/install - not /usr/local like gnu autoconf would After running the configure stage, a Makefile will be created for you to run with gnu make
make
$ make help TOP of source is ... compile_info : show compile flags, directories, options lib : build library ... docs : build doxygen docs in ... clean : clean intermediate files distclean / realclean : clean all built files (except docs) tools : build tool programs examples : build example programs tests : build test programs test : run tests, sh scripts and test programs dirs : create output dirs ... install : install tool executables files into ... install-dev : install tool executables, include files and library files into ... install-dev-docs : install tool executables, include files, library files, and doxygen docs into ...
