Skipping tests with Maven

The Maven Surefire page explains that -DskipTests and -Dmaven.test.skip=true can both be used to avoid running tests. If you use an old-ish version of Surefire (pre-2.4) or even if you use the latest version (2.4.2 as of February 21th, 2008), there are a couple of things you need to know.

Your version of Surefire

First, and most importantly, the -DskipTests option only works from Surefire 2.4. And (annoyingly), there are no error messages if you do try to use it with Surefire 2.3.
Therefore, your first task is to check your version of Surefire. For this, run “mvn -X test” and look for a line that mentions surefire. It should look like this: “maven-surefire-plugin: resolved to version 2.3 from repository central“.

If you do want to upgrade to a more recent version of Surefire, the safest option is to explicitly configure Surefire in your pom.xml:

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.4.2</version
</plugin>

This will force everybody on your team to use the same version of Surefire — not a bad thing.
Another option is to request an update on your own little repository with “mvn -U test”. The -U option ensures that all the elements that are used during the build are upgraded to the latest versions available. Obviously, if you use -U, then you might have different versions of tools compared to your colleagues (that was also probably the case before; you just didn’t know it).

Are you still with me?

If the version of surefire is 2.4 or above, everything below applies. Otherwise, remember that only -Dmaven.test.skip=true is available to you.

What skipping the tests really does

It is important to understand that skipping tests will not skip the test phase; it will only prevent your JUnit test from being run. So if you type “mvn test -DskipTests” or “mvn test -Dmaven.test.skip=true”, the normal JUnit tests will not be run, but plugins attached to the “test” phase will.
Take for example this configuration:

<plugins>
<plugin>
<groupId>myGroupId</groupId>
<artifactId>exoticTestTool</artifactId>
<executions>
<execution>
<id>more-tests</id>
<phase>test</phase>
<goals>
<goal>run</goal>
</goals>
</execution>
</executions>
</plugin
</plugins>

In this case, your exoticTestTool will be run, regardless of whether you wanted to skip tests.

This is not a big issue… except if you use -Dmaven.test.skip=true and not -DskipTests. In this case, the test classes will not be compiled either. And the interesting thing is that you might not notice that fact, if they have been compiled in the past. Meaning that you could be rather confused: changes in your source test classes will not be reflected in your compiled test classes, so your exotic tests will keep passing (or failing) no matter what you do.

The reason behind this is that both -DskipTests and -Dmaven.test.skip=true are in fact parameters supported by Surefire (the intermediate plugin in between Maven and JUnit/TestNG), and not by the Maven engine… while -Dmaven.test.skip=true is the only one of those two options to be supported by the compiler.

So, remember that you should always favor a -DskipTests in place of a -Dmaven.test.skip=true (except if you are pre-Surefire-2.4, of course).

It also means that, if your test classes do not compile and you have an exoticTestTool configured, there are only a few unsatisfactory options:

1. change the configuration of your pom.xml so that your exoticTestTool is not run
2. modify your exoticTestTool so that it supports -DskipTests and -Dmaven.test.skip=true (often next to impossible if this is a third-party tool).

But of course, your test classes should always run, right?