How to Write a Makefile That Generates a .jar

File Dependencies: make

Last modified: Sep 14, 2017

Abstract

make is perhaps the oldest build management tool. It introduced the idea of dependency-based builds.

In this lesson we will look at the build model implemented by make and at how to describe projects to the make tool.

We will look at how make could be applied to some of our sample projects from the prior lesson.

make

make is a command/program that enacts builds according to a dependency graph expressed in a makefile.

make devised by Dr. Stuart Feldman of Bel Labs in 1977
It has long been a standard component of *nix systems
- GNU make is a popular moden variant

1 The make Command

make looks for its instructions in a file named, by default, makefile or Makefile

The make command can name any file in the graph as the target to be built, e.g.,

              make CppJavaScanner

If no target is given, make builds the first file described in the makefile

make Options

Some useful options:

-n: Print the commands that make would issue to rebuild the target, but don't actually perform the commands.
-k: "Keep going." Don't stop the build at the first failue, but continue building any required targets that do not depend on the one whose construction has failed.
-f filename: Use filename instead of the default makefile or Makefile

2 makefiles

At its heart, a makefile is a collection of rules.

2.1 Rules

A rule describes how to build a single file of the project.

Each rule indicates
- The target file to be constructed
- The dependencies: the other files in this project from which the target is constructed.
- The commands that must be executed to construct the target from its dependencies.
Rules may appear in any order
- Except that the first rule's target is the default built by make when no explicit target is specified in the command line.

The Components of a Rule

A rule has the form

                              target:                dependencies                commands

where

target is the target file,
dependencies is a space-separated list of files on which the target is dependent
commands is a set of zero or more commands, one per line, each preceded by a Tab character.

Rule Examples

          codeAnnotation.jar: code2HTML.class CppJavaScanner.class         jar tvf codeAnnotation.jar code2HTML.class CppJavaScanner.class  CppJavaScanner.class: CppJavaScanner.java         javac CppJavaScanner.java  code2HTML.class: code2HTML.java CppJavaScanner.java         javac code2HTML.java  CppJavaScanner.java: code2html.flex         java -cp JFlex.jar JFlex.Main code2html.flex

Why is This Better than Scripting?

Suppose that we edit code2html.java and then invoke make

Only one javac will be issued, after which the jar command is run.
make has determined the minimum number of steps required to rebuild after a change.

How make Works

Construct the dependency graph from the target and dependency entries in the makefile
Do a topological sort to determine an order in which to construct targets.
For each target visited, invoke the commands if the target file does not exist or if any dependency file is newer
- Relies on file modification dates

2.2 Variables

A makefile can use variables to simplify the rules or to add flexibility in configuring the makefile.

All variables hold strings.
Variables are initialized by a simple assignment

variable = value
Variables are immutable (constants)

Assignments may appear within the makefile or in the command line, e.g.:

              make JOPTIONS=-g codeAnnotation.jar

Referencing Variables

Variables are referenced as $(variable) or ${variable}, e.g.,

          CppJavaScanner.class: CppJavaScanner.java         javac $(JOPTIONS) CppJavaScanner.java  code2HTML.class: code2HTML.java CppJavaScanner.java         javac $(JOPTIONS) code2HTML.java

Adding Power to Variables

GNU make adds some special extensions useful in setting up variables.

Globbing:

              SOURCEFILES=$(wildcard src/*.cpp)

collects a list of all C++ compilation units in the filename{src} directory

Substitutions:

              OBJFILES=$(SOURCEFILES:%.cpp=%.o)

collects a list of all object code files expected by compiling those compilation units.

Example: Using variables

This allows us to write a "generic" rule for compiling C++ programs:

          PROGRAM=myProgramName SOURCEFILES=$(wildcard src/*.cpp) OBJFILES=$(SOURCEFILES:%.cpp=%.o)  $(PROGRAM): $(OBJFILES)         g++ -o $(PROGRAM) $(OBJFILES)

This is technically, incomplete.
- We have not explained how to produce a .o file from a .cpp
Nonetheless, it would work on some systems for the initial build, because they have an "implicit" rule for working with C++
- Still not a good solution by itself – dependencies on .h files have not been captured.

2.3 Implicit Rules and Patterns

Implicit rules describe how to produce a single "kind" (extension) of file from another.
- All make implementations will have some common implicit rules.
- You can modify the list of implicit rules.
Pattern rules are a GNU extension for writing "generic" rules
- Implicit rules could, for the msot part, be written as patterns
- But patterns offer some additional flexibility

Implicit Rules

An implicit rule looks like

          .ext1.ext2:         commands

where ext1 and ext2 are file extensions, and commands are the commands used to convert a file with the first extension into a file wit hthe second.

Example:

          .cpp.o:         g++ -g -c $<

the implicit variable $< holds the dependency file
Also commonly used, $@ denotes the target file.

Using Implicit Rules

The extensions used in implicit rules must be declared:

          .SUFFIXES: .cpp .o

An implicit rule will be used when a target ends in one of these suffixes and

there is no rule listing that file as a target, or
the rule listing that file as a target has no commands

Implicit Rule Example

          PROGRAM=myProgramName SOURCEFILES=src/main.cpp src/adt.cpp OBJFILES=$(SOURCEFILES:%.cpp=%.o) .SUFFIXES: .cpp .o  .cpp.o:         g++ -g -c $<  $(PROGRAM): $(OBJFILES)         g++ -o $(PROGRAM) $(OBJFILES)  src/adt.o: adt.cpp adt.h

Both main.cpp and adt.cpp will be compiled on the initial build.
If adt.h is subsequently modified, then adt.cpp would be re-compiled.

Pattern Rules

A pattern rule looks like a regular rule, but uses '%' as a wildcard in the target and one of their dependencies:

          src/test/java/%.class: src/test/java/%.java junit4.jar         javac -cp junit4.jar -g src/test/java/$*.java

Another implicit variable, $* contains the string matched by the % wildcard.
One advantage of pattern rules, is that we can add dependencies on other files e.g., junit.jar

3 Working with Make

3.1 Touching Files

Modification Dates

make compares the modification dates of targets and dependencies to determine if the target is out of date.
It uses the success/fail status value returned by commands to determine if construction of a target was successful.

Although this is fairly robust, there are ways to fool make

Touching a File

The touch command in *nix sets a files modification date to the current time, without affecting the contents of the file.

Question: What would happen if we touched code2html.flex?

Answer

The next time that make is run, it will

run jflex to produce a new CppJavaScanner.java
run javac to produce a new CppJavaScanner.class
run jar to produce a new codeAnnotation.jar

but it will not recompile code2HTML.java

Sometimes this is a useful thing to do on purpose.

Inadvertant Touches

Suppose we had our code annotation project in a directory project1 and did the following:

          > cd project1 > make > cd .. > cp -rf project1 project2 > cd project2 > make

What would be re-built by the second make?

Almost impossible to tell. All of the files in project2 would have create/modify dates within a second of each other. Ordering, if any, would be arbitrary.
- (better to have done cp -arf project1 project2)

Inadvertant Touches

Similarly, successive calls to make can sometimes be confused if the time between creation of some intermediate targets is within a single clock "tick".
Clock drift between different machines can be particularly troublesome.
- (particular between the server running the make command and a file server responsible for storing the files).

Created != Success

Some commands we might give to create a target will create no file if the command fails.
- e.g., g++ does not create a .o file if compilation errors occur
Others will create some kind of file anyway.
- E.g., any command that is invoked with output redirection,
```
                  command > target                                  
```
- which could cause make to assume that the target need not be re-constructed the next time around.
- Some make programs explcitly delete targets if the command fails.

3.2 Artificial Targets

Fooling make Again

A creative way to fool make:

What happens if we give a rule whose commands never actually create the target?

          target: dependency1 dependency2         echo Nope. Not going make that target!

The first time we run make, the dependencies will be created and the echo performed.
Each subsequent time we run make, the dependencies will be re-created if necessary and the echo performed.

Artificial Targets

We can take advantage of this trick by adding artificial targets that serve as the names for tasks to be performed.

          build: codeAnnotation.jar  install: build         cp codeAnnotation.jar $(INSTALLDIR)  clean:         rm *.class CppJavaScanner.java  codeAnnotation.jar: code2HTML.class CppJavaScanner.class         jar tvf codeAnnotation.jar code2HTML.class CppJavaScanner.class  CppJavaScanner.class: CppJavaScanner.java         javac CppJavaScanner.java  code2HTML.class: code2HTML.java CppJavaScanner.java         javac code2HTML.java  CppJavaScanner.java: code2html.flex         java -cp JFlex.jar JFlex.Main code2html.flex

Common Artificial Targets

all: Often made the first rule in the makefile so that it becomes the default. Builds everything. May also run tests.
build: Build everything.
install: Build, then install
test: Build, then run tests
clean: Delete everything that would have been produced by the makefile in a build or test run.

3.3 Dependency Analysis

Coming up with a list of dependencies (and keeping it current) can be troublesome.

Various tools exist for this purpose for programming languages
The gcc and g++ compilers have a compile-time option, -MMd, which emits a .d file containing a target and dependency line.
- Use this with an implicit rule to give the actual command

Self-Building Makefile

selfBuilding.listing

              MAINPROG=testpicture CPPS:=$(wildcard *.cpp)  CPPFLAGS=-g -D$(DISTR) CPP=g++  OBJS=$(CPPS:%.cpp=%.o) DEPENDENCIES = $(CPPS:%.cpp=%.d)  %.d: %.cpp 	touch $@  %.o: %.cpp 	$(CPP) $(CPPFLAGS) -MMD -o $@ -c $*.cpp  make.dep: $(DEPENDENCIES) 	-cat $(DEPENDENCIES) > $@  include make.dep

On the first make,
- for each .cpp file, an empty .d file is created by touch
- All *.d files are concatenated to for a file make.dep
- The file make.dep is included as part of the makefile.
- As the .cpp files are compiled, the .d are replaced by a rule making the .o file dependent on that .cpp file and on any .h files that it included.
On subsequent make runs,
- the .d files contain the dependencies for each .cpp file.
- All *.d files are concatenated to for a file make.dep
- The file make.dep is included as part of the makefile.
- If any .h or .cpp file has been changed, the .o files dependent on it will be regenerated.

3.4 Managing Subproject Builds

Subprojects are generally handled by giving each subproject its own makefile and using a master makefile to invoke the artificial targets:

          all:         cd model; make         cd vcncurses; make         cd vcjava; make  clean:         cd model; make clean         cd vcncurses; make clean         cd vcjava; make clean

4 Case Studies – Introduction

For make and for the other buiild managers we will be discussing, we will use 3 case studies.

4.1 Simple Java Build

The simple Java build features

Java code to be compiled and built into a jar.
JUnit tests to be compiled and run.

Layout

We will follow the Apache/Android guidelines for source code layout.

          project root/ |--src/ |--|--main/ |--|--|--java/ |--|--|--|-- ...java packages... |--|--test/ |--|--|--java/ |--|--|--|-- ...junit packages... |--|--|--data/ |--|--|--|-- ...test data ... |--lib/ |--|-- ...3rd party libraries (temporary hack)...

We will look at better ways to supply 3rd party libraries in later lessons on configuration management.

Layout

For outputs from the build, we will follow either the Apache

          project root/ |--target/ |--|--classes/ |--|--|-- ...compiled code from src/main |--|--test-classes/ |--|--|-- ...compiled code from src/test |--|--reports/

or the Android standard

          project root/ |--build/ |--|--classes/ |--|--|--main/ |--|--|--|-- ...compiled code from src/main |--|--|--test/ |--|--|--|-- ...compiled code from src/test |--|--lib/ |--|--reports/

whichever is more "natural" for the build manager under discussion.

4.2 Java Build with Code Generation

A slightly less conventional build:

Java code to be compiled and built into a jar.
Some of the Java code is generated automatically
- JFlex program is used to produce lexical analyzers from collections of regular expressions.
JUnit tests to be compiled and run.

Layout

Project layout is the same as the simple Java build, except for an input directory for JFlex specifications and a working directory for the generated Java source code:

          project root/ |--src/ |--|--main/ |--|--|--java/ |--|--|--|-- ...java packages... |--|--|--jflex/ |--|--|--|-- ...JFlex input files... |--|--test/ |--|--|--java/ |--|--|--|-- ...junit packages... |--|--|--data/ |--|--|--|-- ...test data ... |--lib/ |--|-- ...3rd party libraries (temporary hack)... |--target/   (or build/) |--|--generated-src/ |--|--|-- ...Java code generated by JFlex |--|...

4.3 C++ Multi-project Build

C and C++ pose a challenge for the build systems that want to provide defaults for simple projects.

A collection of .o files can only be linked together if they have no more than one main() function.
- If you want a project to produce more than one executable, you have to tell the build manager how to divide up the .o files for each one.
Most IDE build managers and most default rules for non-IDE build managers don't allow that.
- One project – one executable
- But unit testing adds additional executables.

Unit testing and C/C++ Builds

Even simple projects with unit tests wind up being divided into multiple sub-projects.

One or more sub-projects for groups of ADTs with unit tests.
A subproject for each "real" executable.
If not all developers will have the C/C++ unit test framework pre-installed, that may be another subproject.

4.3.1 Source Layout

          project root/ |--lib/       (ADTs with unit tests) |--|--src/ |--|--|--main/ |--|--|--|--cpp/ |--|--|--|--|-- .cpp and local .h files for ADTs |--|--|--|--headers/ |--|--|--|--|-- Header files "exported" to other subprojects |--|--|--mainTest/ |--|--|--|--cpp/ |--|--|--|--|-- .cpp unit tests (GTest) |--exe/     (may be named for executable, particularly if more than one) |--|--src/ |--|--|--main/ |--|--|--|--cpp/ |--|--|--|--|-- .cpp main function for one executable |--gtest/       (Google Test framework) |--|--include/ |--|--|-- Header files exported to unit tests |--|--|--src/ |--|--|--|-- .cpp and local .h files for framewwork

The gtest/ subproject has a slightly different layout because it's a third party project and it's not worth trying to repackage their sourcecode.

4.3.2 Output layout

The lib/ and gtest/ subprojects each produce a static library.
The exe subproject produces an executable
project root/ |–lib/ (ADTs with unit tests) |–|–build/ |–|–|–objs/ holding area for .o files |–|–|–libs/ generated library goes here |–|–|–test-results/ reports from unit test |–exe/ (may be named for executable, particularly if more than one) |–|–build/ |–|–|–objs/ holding area for .o files |–|–|–exe/ generated executables go here |–gtest/ (Google Test framework) |–|–build/ |–|–|–objs/ holding area for .o files |–|–|–libs/ generated library goes here

5 Case Studies – Makefiles

5.1 Simple Java Build – Makefiles

          TARGET=codeAnnotation.jar  SRC=src/main/java                                 ➀ CLASSDEST=build/classes JARDEST=build TESTSRC=src/test/java TESTCLASSDEST=build/testclasses  JAVA=$(shell find $(SRC) -type f -name '*.java')  ➁ TESTJAVA=$(shell find $(TESTSRC) -type f -name '*.java') CLASSES=$(patsubst $(SRC)/%, $(CLASSDEST)/%, $(JAVA:%.java=%.class)) TESTCLASSES=$(patsubst $(TESTSRC)/%, $(TESTCLASSDEST)/%, $(TESTJAVA:%.java=%.class)) TESTCLASSNAMES=$(subst /,.,$(subst $(TESTSRC)/, ,$(TESTJAVA:%.java=%)))    .SUFFIXES:   #  # Targets: #  all: test $(JARDEST)/$(TARGET)  build/setup:                                  ➂     -mkdir -p $(JARDEST)     -mkdir -p $(CLASSDEST)     -mkdir -p $(TESTCLASSDEST)     date > $@   $(JARDEST)/$(TARGET): $(CLASSES)               ➃     cd $(CLASSDEST); jar cf temp.jar *     mv $(CLASSDEST)/temp.jar $@     test: $(TESTCLASSES) $(CLASSES)                ➄     java -cp lib/junit-4.10.jar:lib/junit/hamcrest-core-1.1.jar:$(CLASSDEST):$(TESTCLASSDEST) org.junit.runner.JUnitCore $(TESTCLASSNAMES)    $(CLASSDEST)/%.class: $(SRC)/%.java build/setup    ➅     javac -g -cp $(CLASSDEST) -d $(CLASSDEST) -sourcepath $(SRC) $(SRC)/$*.java  $(TESTCLASSDEST)/%.class: $(TESTSRC)/%.java build/setup $(CLASSES)     javac -g -cp $(CLASSDEST):lib/junit-4.10.jar:lib/junit/hamcrest-core-1.1.jar -d $(TESTCLASSDEST)  -sourcepath $(TESTSRC) $(TESTSRC)/$*.java    clean:     -rm -f build

➀ Symbolic names for directories make it easier to rearrange layout.
➁ Various functions for manipulating and rewriting lists of files
➂ Create output directories. Note that this appears as a dependency in most of the later compilation rules (to guarantee it's done before compiling).
➃ Build a Jar file
➄ Run the unit tests
➅ Compile .cpp files.

5.2 Java Build with Code Generation – Makefile

This project adds a stage before compilation to generate some of the source code that then needs to be compiled.

          TARGET=codeAnnotation.jar  SRC=src/main/java CLASSDEST=build/classes JARDEST=build TESTSRC=src/test/java TESTCLASSDEST=build/testclasses            GENSRC=build/gen/java GENDEST=build/classes FLEXSRC=src/main/jflex            JAVA=$(shell find $(SRC) -type f -name '*.java') TESTJAVA=$(shell find $(TESTSRC) -type f -name '*.java') CLASSES=$(patsubst $(SRC)/%, $(CLASSDEST)/%, $(JAVA:%.java=%.class)) TESTCLASSES=$(patsubst $(TESTSRC)/%, $(TESTCLASSDEST)/%, $(TESTJAVA:%.java=%.class)) TESTCLASSNAMES=$(subst /,.,$(subst $(TESTSRC)/, ,$(TESTJAVA:%.java=%)))            GENJAVA=$(GENSRC)/CppJavaScanner.java $(GENSRC)/CppJavaTeXScanner.java \    $(GENSRC)/ListingScanner.java $(GENSRC)/ListingTeXScanner.java GENCLASSES=$(GENDEST)/CppJavaScanner.class $(GENDEST)/CppJavaTeXScanner.class \    $(GENDEST)/ListingScanner.class $(GENDEST)/ListingTeXScanner.class            .SUFFIXES:   #  # Targets: #  all: test $(JARDEST)/$(TARGET)  build/setup:     -mkdir -p $(JARDEST)            -mkdir -p $(GENSRC)            -mkdir -p $(CLASSDEST)     -mkdir -p $(TESTCLASSDEST)     date > $@   $(JARDEST)/$(TARGET): $(CLASSES)     cd $(CLASSDEST); jar cf temp.jar *     mv $(CLASSDEST)/temp.jar $@     test: $(TESTCLASSES) $(CLASSES)     java -cp lib/junit-4.10.jar:lib/junit/hamcrest-core-1.1.jar:$(CLASSDEST):$(TESTCLASSDEST) org.junit.runner.JUnitCore $(TESTCLASSNAMES)            $(GENSRC)/CppJavaScanner.java: $(FLEXSRC)/code2html.flex build/setup     java -jar lib/jflex-1.4.3.jar -d $(GENSRC) $<  $(GENSRC)/CppJavaTeXScanner.java: $(FLEXSRC)/code2tex.flex build/setup     java -jar lib/jflex-1.4.3.jar -d $(GENSRC) $<   $(GENSRC)/ListingScanner.java: $(FLEXSRC)/list2html.flex build/setup     java -jar lib/jflex-1.4.3.jar -d $(GENSRC) $<   $(GENSRC)/ListingTeXScanner.java: $(FLEXSRC)/list2tex.flex build/setup     java -jar lib/jflex-1.4.3.jar -d $(GENSRC) $<   $(GENDEST)/%.class: $(GENSRC)/%.java build/setup     javac -g -d $(GENDEST) -sourcepath $(GENSRC)  $(GENSRC)/$*.java            $(CLASSDEST)/%.class: $(SRC)/%.java build/setup            $(GENCLASSES)            javac -g -cp $(CLASSDEST) -d $(CLASSDEST) -sourcepath $(SRC) $(SRC)/$*.java  $(TESTCLASSDEST)/%.class: $(TESTSRC)/%.java build/setup     javac -g -cp $(CLASSDEST):lib/junit-4.10.jar:lib/junit/hamcrest-core-1.1.jar -d $(TESTCLASSDEST)  -sourcepath $(TESTSRC) $(TESTSRC)/$*.java    clean:     -rm -f build

5.3 C++ Multi-project Build – Makefiles

Because this project is divided into multiple subprojects, we will have multiple makefiles:

One per subproject
And a master to launch them all.

5.3.1 The master file

In the project root directory:

          all:     $(MAKE) -C gtest     $(MAKE) -C lib     $(MAKE) -C exe  clean:     $(MAKE) -C gtest     $(MAKE) -C lib     $(MAKE) -C exe

5.3.2 The lib subproject

          TARGET=libmain.a   ➀ CXX=g++ CXXFLAGS=-g -pthread -I ../gtest/include -I src/main/headers -std=c++11 LINK=$(CXX) LFLAGS=../gtest/build/libs/libgtest.a -lpthread  SRC=src/main/cpp OBJDEST=build/objs/main DEST=build/libs EXEDEST=build/exe  TESTSRC=src/mainTest/cpp TESTOBJDEST=build/objs/test  CPP=$(wildcard $(SRC)/*.cpp) OBJS=$(patsubst $(SRC)/%, $(OBJDEST)/%, $(CPP:%.cpp=%.o)) TESTCPP=$(wildcard $(TESTSRC)/*.cpp) TESTOBJS=$(patsubst $(TESTSRC)/%, $(TESTOBJDEST)/%, $(TESTCPP:%.cpp=%.o))   .SUFFIXES:   #  # Targets: #  all: $(DEST)/$(TARGET) test   build/setup:                          ➁     -mkdir -p $(DEST)     -mkdir -p $(EXEDEST)     -mkdir -p $(OBJDEST)     -mkdir -p $(TESTOBJDEST)     date > $@   test: $(EXEDEST)/runtest              ➂     $(EXEDEST)/runtest    $(DEST)/$(TARGET): $(OBJS) test build/setup   ➃     ar rcs $@ $(OBJS)     ranlib $@  $(EXEDEST)/runtest: $(TESTOBJS) $(OBJS)       ➄     $(LINK) -o $@ $(CPPFLAGS) $(TESTOBJS) $(OBJS) $(LFLAGS)   $(OBJDEST)/%.o: $(SRC)/%.cpp build/setup        ➅     $(CXX) $(CXXFLAGS) -o $@ -c $<  $(TESTOBJDEST)/%.o: $(TESTSRC)/%.cpp build/setup    ➆     $(CXX) $(CXXFLAGS) -o $@ -c $<   clean:     -rm -f build

➀ Symbolic names for directories, programs, and lists of files.
➁ Set up output directories
➂ Run the unit tests.
➃ Create the library from the .o files generated by compilation
➄ Generate the executable for running unit tests.
➅ Compile the ADTs source code
➆ Comile the unit test source code.

How to Write a Makefile That Generates a .jar

Source: https://www.cs.odu.edu/~tkennedy/cs350/f19/Public/make/index.html

How to Write a Makefile That Generates a .jar

File Dependencies: make

1 The make Command

2 makefiles

2.1 Rules

2.2 Variables

2.3 Implicit Rules and Patterns

3 Working with Make

3.1 Touching Files

3.2 Artificial Targets

3.3 Dependency Analysis

3.4 Managing Subproject Builds

4 Case Studies – Introduction

4.1 Simple Java Build

4.2 Java Build with Code Generation

4.3 C++ Multi-project Build

4.3.1 Source Layout

4.3.2 Output layout

5 Case Studies – Makefiles

5.1 Simple Java Build – Makefiles

5.2 Java Build with Code Generation – Makefile

5.3 C++ Multi-project Build – Makefiles

5.3.1 The master file

5.3.2 The lib subproject

0 Response to "How to Write a Makefile That Generates a .jar"

Post a Comment

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel