Incremental project builders

An incremental project builder is an object that manipulates the resources in a project in a particular way. Incremental project builders are often used to apply a transformation on a resource to produce a resource or artifact of another kind. Resources created by a builder are typically marked as derived resources.

Plug-ins contribute incremental project builders to the platform in order to implement specialized resource transformations. For example, the Java development tools (JDT)define an incremental project builder that compiles a Java source file into a class file any time a file is added or modified in a Java project. It also keeps track of dependent files and recompiles them when necessary.

From an API point of view, the platform defines two basic types of builds:

Incremental builds are seeded with a resource change delta. The delta reflects the net effect of all resource changes since the builder last built the project. This delta is similar to the one used inside resource change events.

Projects can be periodically cleaned by the user in order to force a rebuild of a complete project the next time an incremental build is performed on that project. Cleaning a project removes build information such as problem markers and class files.

Projects support multiple build configurations which allow Incremental Project Builders to build a project in more than one way while providing a resource delta for each build configuration.

Builders are best understood by example. The JDT Java compiler is driven by a Java incremental project builder which recompiles the files in a project that are affected by changes. When a full build is triggered, (or an incremental build after a clean), all of the .java files in the project are compiled. Any compile problems encountered are added as problem markers on the affected .java files. When an incremental build is triggered, the builder selectively recompiles the added, changed, or otherwise affected .java files that are described in the resource delta and updates the problem markers as necessary. Any .class files or markers that are no longer appropriate are removed.

Incremental building has obvious performance benefits for projects with hundreds or thousands of resources, most of which are unchanging at any given point in time.

The technical challenge for incremental building is to determine exactly what needs to be rebuilt. For example, the internal state maintained by the Java builder includes things like a dependency graph and a list of compilation problems reported. This information is used during an incremental build to identify which classes need to be recompiled in response to a change in a Java resource.

Although the basic structure for building is defined in the platform, the real work is done in the builder code. Patterns for implementing complex incremental builders are beyond the scope of this discussion, since the implementation is dependent on the specific builder design.

Invoking a build

A builder can be invoked explicitly in one of the following ways:

In practice, the workbench user triggers a build by selecting corresponding commands in the resource navigator menu.

Incremental project builders are also invoked implicitly by the platform during an auto-build. If enabled, auto-builds run whenever the workspace is changed.

Defining an incremental project builder

The org.eclipse.core.resources.builders extension point is used to contribute an incremental project builder to the platform. The following markup shows how the hypothetical plug-in com.example.builders could contribute an incremental project builder.

   <extension
      id="mybuilder" name="My Sample Builder" point="org.eclipse.core.resources.builders">
      <builder
         <run 
            class="com.example.builders.BuilderExample">
            <parameter name="optimize" value="true" />
            <parameter name="comment" value="Builder comment" />
         </run>
      </builder>
   </extension>

The class identified in the extension point must extend the platform class IncrementalProjectBuilder.

   public class BuilderExample extends IncrementalProjectBuilder {
      IProject[] build(int kind, Map args, IProgressMonitor monitor)
            throws CoreException {
         // add your build logic here
         return null;
      }
      protected void startupOnInitialize() {
         // add builder init logic here
      }
      protected void clean(IProgressMonitor monitor) {
         // add builder clean logic here
      }
   }

Build processing begins with the build() method, which includes information about the kind of build that has been requested. The build is one of the following values:

If an incremental build or workspace auto-build have been requested, a resource delta is provided to describe the changes in the resources since the last build.

NOTE: If no delta is available, the build kind will always be FULL_BUILD. See Advanced Build Kinds for more on build kinds.

The following snippet further refines the build() method.

   protected IProject[] build(int kind, Map args, IProgressMonitor monitor
         throws CoreException {
      if (kind == IncrementalProjectBuilder.FULL_BUILD) {
         fullBuild(monitor);
      } else {
         IResourceDelta delta = getDelta(getProject());
         if (delta == null) {
            fullBuild(monitor);
         } else {
            incrementalBuild(delta, monitor);
         }
      }
      return null;
   }

It sometimes happens that when building project "X," a builder needs information about changes in some other project "Y."  (For example, if a Java class in X implements an interface provided in Y.)  While building X, a delta for Y is available by calling getDelta(Y).  To ensure that the platform can provide such deltas, X's builder must have declared the dependency between X and Y by returning an array containing Y from a previous build() call.  If a builder has no dependencies, it can simply return null. See IncrementalProjectBuilder for further information.

Full build

The logic required to process a full build request is specific to the plug-in. It may involve visiting every resource in the project or even examining other projects if there are dependencies between projects. The following snippet suggests how a full build might be implemented.

   protected void fullBuild(final IProgressMonitor monitor) throws CoreException {
      try {
         getProject().accept(new MyBuildVisitor());
      } catch (CoreException e) { }
   }

The build visitor would perform the build for the specific resource (and answer true to continue visiting all child resources).

   class MyBuildVisitor implements IResourceVisitor {
      public boolean visit(IResource res) {
         //build the specified resource.
         //return true to continue visiting children.
         return true;
      }
   }

The visit process continues until the full resource tree has been traveled.

Incremental build

When performing an incremental build, the builder works with a resource change delta instead of a complete resource tree.

   protected void incrementalBuild(IResourceDelta delta, 
         IProgressMonitor monitor) throws CoreException {
      // the visitor does the work.
      delta.accept(new MyBuildDeltaVisitor());
   }

The visit process continues until the complete resource delta tree has been traveled. The specific nature of changes is similar to that described in Implementing a resource change listener.  One important difference is that with incremental project builders, you are working with a resource delta based on a particular project, not the entire workspace.

Cleaning before a build

The workbench allows users to clean a project or set of projects before initiating a build. This feature allows the user to force a rebuild from scratch on only certain projects. Builders should implement this method to clean up any problem markers and derived resources in the project.

Build Locking

By default, builders run with the Workspace Root scheduling rule. This prevents other threads from modifying the Eclipse workspace while the build is in progress. Long running builders, or builders which do not care about concurrent changes in parts of the resource tree, may wish to allow modification while the build is in progress. A builder may relax the scheduling rule its run with by overriding the #getRule method:

  public ISchedulingRule getRule(int kind, Map<String,String> args) {
    // Allow any resource to be modified concurrently with this buidler's invocation.
    return null;
  }

Discovering what's being built

It's sometimes useful for a builder to discover why its being built. Builders have access to a build context by calling
IBuildContext buildContext = getContext();
When building a project and its references, projects may be implicitly built if they are reachable in the reference graph of the top-level project the user requested built. With this API a builder can discover which build configurations have been built before it (IBuildContext#getAllReferencedBuildConfigs()), will be built after it (IBuildContext#getAllReferencingBuildConfigs()), as well as which build configurations the user asked to be built (IBuildContext#getRequestedConfigs()).

Associating an incremental project builder with a project

To make a builder available for a given project, it must be included in the build spec for the project. A project's build spec is a list of commands to run, in sequence, when the project is built. Each command names a single incremental project builder.

NOTE: The builder name in a build command is the fully qualified id of the builder extension. The fully qualified id of an extension is created by combining the plug-in id with the simple extension id in the plugin.xml file. For example, a builder with simple extension id "mybuilder" in the plug-in "com.example.builders" would have the name "com.example.builders.mybuilder"

The following snippet adds a new builder as the first builder in the existing list of builders.

   final String BUILDER_ID = "com.example.builders.mybuilder";
   IProjectDescription desc = project.getDescription();
   ICommand[] commands = desc.getBuildSpec();
   boolean found = false;

   for (int i = 0; i < commands.length; ++i) {
      if (commands[i].getBuilderName().equals(BUILDER_ID)) {
         found = true;
         break;
      }
   }
   if (!found) { 
      //add builder to project
      ICommand command = desc.newCommand();
      command.setBuilderName(BUILDER_ID);
      ICommand[] newCommands = new ICommand[commands.length + 1];

      // Add it before other builders.
      System.arraycopy(commands, 0, newCommands, 1, commands.length);
      newCommands[0] = command;
      desc.setBuildSpec(newCommands);
      project.setDescription(desc, null);
   }

Configuring a project's builder is done just once, usually as the project is being created. A common way to associate a builder with a project is by configuring a project nature.

Advanced Build Kinds

Triggers vs. Kinds

When a build is invoked on an IProject or IWorkspace, we call the passed in build kind the build trigger. This trigger is usually passed through to the IncrementalProjectBuilder as the build kind argument when IncrementalProjectBuilder#build is invoked.

The major exception to this is where no delta exists. In this case the build trigger is always promoted to FULL_BUILD.

The mapping between triggers and kinds looks like:
Trigger Kind
FULL_BUILD FULL_BUILD
INCREMENTAL_BUILD INCREMENTAL_BUILD or FULL_BUILD
AUTO_BUILD AUTO_BUILD or FULL_BUILD
CLEAN_BUILD IncrementalProjectBuilder#clean(...)

Ignoring Build Kinds

The platform provides API for configuring which build kinds a builder responds to. If the builder is marked as 'isConfigurable' in its extension point it can be configured at runtime to respond to, or ignore, certain build triggers.

For example a long running builder which doesn't want to be called during workspace AUTO_BUILD, can use:
  ICommand#setBuilding(AUTO_BUILD, false);
It is important to note that:

See ICommand for more information.