Io Language: adding slots to messages

Week 2 of the Seven Languages in Seven Weeks book is about the Io Language. Day 3 is more specifically on how the flow of control (how messages are passed to an object or to its parent) can be hijacked to create a DSL.

In this post, I am going to describe an interesting feature in Io, and how I extended it a bit to enhance an XML generator.

In the example provided, the following Io code:

Builder ul(
  li("Io"),
  li("Lua"),
  li("JavaScript"))

can generate the following output:

<ul>
<li>
Io
</li>
<li>
Lua
</li>
<li>
JavaScript
</li>
</ul>

This is made possible by redefining the forward method (“slot” in the terminology of Io), which is responsible for deciding who to apply the method calls (“message calls” in Io) to.
The other key thing is that it is possible to obtain the details on a message call, even when this message is not defined, a bit like what method_missing does in Ruby. This is done using call message.

For example, while this would be a normal method definition:

Builder myMethod := method("I have been called" println)

you could also do it awkwardly by redefining forward and using call message:

Builder forward := method(if(call message name == "myMethod", "I have been called" println))

This leads us to the following program (provided in the book) as a way to implement the Builder called in the example:

Builder := Object clone

Builder forward := method(
	writeln("<", call message name, ">")
	call message arguments foreach(
		arg,
		content := self doMessage(arg);
		if(content type == "Sequence",
			writeln(content)
		)
	)
	writeln("")
)

The exercise was to add indentation dependent on the depth of the tag. It turns out that slots (that is, members of classes or instances) can added to pretty much anything in Io, so I decided to add a depth slot to arguments. Here is the resulting Io code:

// create a sub-class ("prototype") of Object
Builder := Object clone

// redefine the forward method in order to intercept unknown method ("message") calls
Builder forward := method(
	// if the message does not have a "depth" attribute ("slot"), add it
	call message hasLocalSlot("depth") ifFalse(call message depth := 0)
	// indentation is "depth"-times a tab
	indent := "\t" repeated(call message depth)
	// open an XML tag, the name being the name of the message,
	// preceded by the indentation
	writeln(indent, "<", call message name, ">")
	// loop on each of the arguments of the message
	call message arguments foreach(
		arg,
		// increase the depth level for the sub-message call
		arg depth := call message depth + 1;
		// call the message represented by the argument on the current instance
		// (or on the current prototype, as it is the case here)
		content := self doMessage(arg);
		// check if the argument turns out to be a normal String
		if(content type == "Sequence",
			// prints the value of the argument, preceded by the indentation
			writeln(indent, content)
		)
	)
	// closes the XML tag
	writeln(indent, "")
)

With this, the folllowing code

Builder p (
	ul(
		li("Io"),
		li("Lua"),
		li("JavaScript")
	)
)

produces this result:

<p>
	<ul>
		<li>
		Io
		</li>
		<li>
		Lua
		</li>
		<li>
		JavaScript
		</li>
	</ul>
</p>

To sum up, we have created a DSL by redefining the way methods are called, introspecting the method call, and finally added a parameter to the method call (!).

If you want to read more about this technique, check out this other post on Io in the 7 Languages book. Curiously, Ben Nadel uses a variable on the Builder to store the current depth. That works, but it seems to me that it behaves just like a global variable, which wouldn’t be thread-safe. Also, I rather like associating a tag to its corresponding depth.

Posted in misc | Leave a comment

Create hidden files in Apache VFS with the RAM filesystem type

Apache VFS is a great way to access different file systems in the same way. I particularly like the custom RAM filesystem in my unit tests in order to check code that eventually accesses the actual file system.

For example:

StandardFileSystemManager manager = new StandardFileSystemManager();
manager.init();
manager.resolveFile("ram://root/file.txt").createFile(); // prepare test harness
...
manager.resolveFile("ram://root/file.txt").getName(); // do something useful

Annoyingly, Apache VFS API provides a way to check if a file/folder is hidden, but, although it is always possible to manually create hidden files on your system, there is no way to mark a file as hidden in the RAM filesystem.

manager.resolveFile("ram://root/file.txt").isHidden(); // returns false
manager.resolveFile("ram://root/.file.txt").isHidden(); // also returns false

The solution is to modify the RAM filesystem and customize its checking of the hidden attribute. This means that we need a new RamFileProvider, a new RamFileSystem and finally a new RamFileObject. In this example, all files and folders whose names start with a dot will be considered hidden:

public class RamFileProvider extends org.apache.commons.vfs2.provider.ram.RamFileProvider {
  @Override
  protected FileSystem doCreateFileSystem(FileName name, FileSystemOptions fileSystemOptions) throws FileSystemException {
    return new RamFileSystem(name, fileSystemOptions);
  }

  private static class RamFileSystem extends org.apache.commons.vfs2.provider.ram.RamFileSystem {
    public RamFileSystem(FileName rootName, FileSystemOptions fileSystemOptions) {
      super(rootName, fileSystemOptions);
    }

    @Override
    protected FileObject createFile(AbstractFileName name) throws Exception {
      return new RamFileObject(name, this);
    }
  }

  private static class RamFileObject extends org.apache.commons.vfs2.provider.ram.RamFileObject {
    public RamFileObject(AbstractFileName name, RamFileSystem ramFileSystemWithHiddenSupport) {
      super(name, ramFileSystemWithHiddenSupport);
    }

    @Override
    public boolean isHidden() throws FileSystemException {
      // this is the important part
      if (getName().getBaseName().startsWith(".")) {
        return true;
      }
      return super.isHidden(); // in practice, this always return false
    }
  }
}

Finally, this new RamFileProvider must be registered with your existing FileSystemManager. In my case, I get something like that:

StandardFileSystemManager manager = new StandardFileSystemManager();
manager.init();
manager.addProvider("ram-ext", new RamFileProvider());
manager.resolveFile("ram-ext:///root/file.txt").createFile();
manager.resolveFile("ram-ext:///root/.file.txt").createFile();
...
manager.resolveFile("ram-ext://root/file.txt").isHidden(); // returns false
manager.resolveFile("ram-ext://root/.file.txt").isHidden(); // returns true

Note that I did not replace the “ram” filesystem entirely. Although that would have been my preferred option, that would have taken too much time in my little experiment.

Posted in java, test | Leave a comment

Cleaning up test code

My former colleague David just posted an example of verbose test code on his blog (the parts in French have been translated to English by myself):

/**
 *
 */
@Test
public void testGetCustomerOK() {

  LOGGER.info("======= testGetCustomerOK starting...");

  try {
    CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

    // Check.
    Assert.assertNotNull("Extra not found.", targetDTO);
    Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());

  } catch (CustomerBusinessException exception) {
    LOGGER.error("CustomerBusinessException : {}",
	    exception.getCause());
    Assert.fail(exception.getMessage());
  } catch (UnavailableResourceException exception) {
    LOGGER.error("UnavailableResourceException : {}",
	    exception.getMessage());
    Assert.fail(exception.getMessage());
  } catch (UnexpectedException exception) {
    LOGGER.error("UnexpectedException : {}" +
	    exception.getMessage());
    Assert.fail(exception.getMessage());
  } catch (Exception exception) {
    LOGGER.error("CRASH : " + exception.getMessage());
    Assert.fail(exception.getMessage());
  }

  LOGGER.info("======= testGetCustomerOK done.");
}

Facing this code, the first thing I would do is remove the logs in the catch clauses, which are clearly obscuring the code the most. They do not provide actual value, since the exception message is used in Assert.fail() calls.

/**
 *
 */
@Test
public void testGetCustomerOK() {

  LOGGER.info("======= testGetCustomerOK starting...");

  try {
    CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

    // Check.
    Assert.assertNotNull("Extra not found.", targetDTO);
    Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());

  } catch (CustomerBusinessException exception) {
    Assert.fail(exception.getMessage());
  } catch (UnavailableResourceException exception) {
    Assert.fail(exception.getMessage());
  } catch (UnexpectedException exception) {
    Assert.fail(exception.getMessage());
  } catch (Exception exception) {
    Assert.fail(exception.getMessage());
  }

  LOGGER.info("======= testGetCustomerOK done.");
}

Next, the catching of exceptions and calling Assert.fail() is in fact the same work that the test runner actually does. We can let the test runner handle those cases:

/**
 *
 */
@Test
public void testGetCustomerOK() throws Exception {

  LOGGER.info("======= testGetCustomerOK starting...");

  CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

  // Check.
  Assert.assertNotNull("Extra not found.", targetDTO);
  Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());

  LOGGER.info("======= testGetCustomerOK done.");
}

The remaining logs are actually information that can already be known from the test runner, since test results are presented by methods. My guess is that they were added to make the previous logs more readable. They can go now:

/**
 *
 */
@Test
public void testGetCustomerOK() throws Exception {
  CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

  // Check.
  Assert.assertNotNull("Extra not found.", targetDTO);
  Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());
}

The “check” comment serves no useful purpose. Also, there is an empty comment that obviously adds no value:

@Test
public void testGetCustomerOK() throws Exception {
  CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

  Assert.assertNotNull("Extra not found.", targetDTO);
  Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());
}

We are down to 6 lines of code. The first assert does a check that is also done implicitely by the following one. That seems redundant. Also, it is clearly a situation that should not happen in a normal test run. That can go:

@Test
public void testGetCustomerOK() throws Exception {
  CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

  Assert.assertEquals("accountIds must be the same.", "ABC99", targetDTO.getAccountId());
}

The remaining assertion has a comment that says what the test does. In some way, there is some value in this. However, in most (all?) cases, the person seeing the error would open the test class anyway and read the code. The test comment seems redundant. At the very least, its value is far from clear. In those situations, I remove the comments:

@Test
public void testGetCustomerOK() throws Exception {
  CustomerDTO targetDTO = this.serviceImpl.getCustomer("ABC99");

  Assert.assertEquals("ABC99", targetDTO.getAccountId());
}

OK, a little inlining can now be done:

@Test
public void testGetCustomerOK() throws Exception {
  Assert.assertEquals("ABC99", this.serviceImpl.getCustomer("ABC99").getAccountId());
}

The Assert.assertEquals method can also be statically imported:

@Test
public void testGetCustomerOK() throws Exception {
  assertEquals("ABC99", this.serviceImpl.getCustomer("ABC99").getAccountId());
}

And the test could use a renaming:

@Test
public void can_obtain_a_customer_by_account_id() throws Exception {
  assertEquals("ABC99", this.serviceImpl.getCustomer("ABC99").getAccountId());
}

Hm… the “this.” clause is not really necessary:

@Test
public void can_obtain_a_customer_by_account_id() throws Exception {
  assertEquals("ABC99", serviceImpl.getCustomer("ABC99").getAccountId());
}

And… that’s it. I’m down to 4 lines of code. There might be fewer comments, but I believe the result is a lot more readable.

Posted in java | 3 Comments

More data on productivity of small teams

“Adding manpower to a late software project makes it later”. The so-called Brooks’s Law on productivity of software project is well known, since Fred Brooks’ seminal work The Mythical Man-Month, first published in 1975.

That is one of the reasons I’m wince a little when a company talks about plans to hire lots of software developers.

This recent study by QSM seems to prove me right. Its conclusion seems to be that Brooks’s Law can be extended to projects that are not late. At any rate, code is not produced at a faster pace with more manpower.

The idea was to compare the time require to build a 100,000 lines projects, depending on the size of the teal. Results are :

  • 32-people teams : 8.9 months
  • 4-people teams : 9.1 months

That’s a one-week difference. 2-3%!

Costs are obviously much in favor of small teams (roughly 8 times cheaper for the same result!). One of the reasons suggested could be that large teams produce many more bugs.

Unfortunately, there are no discussions on other possible explanations, such as complexity in coordinating, communication, the number of managers, location over multiple rooms or sites, skills, turnover, size of the company… I would have also liked to see other sizes compared. Is there a productivity peak at 2 people ? 6 ? 4 ?

A few years ago, I got into a discussion where one of the persons remarked: “managing large teams is hard!” As far as as I’m concerned, the answer seems to be: “well, don’t do it, then.”

Posted in java | 3 Comments

Worse than static methods or final classes?

Do you know what’s worse that static methods or classes marked as final? I’ll tell you what’s worse: static methods that return final classes. That only provides private constructors.

Here I was, merrily testing my way through a piece of software that sends emails. According to the Java Mail documentation, you are supposed to first create an email session as follows:

Session mailSession = Session.getInstance(properties);

(it is worth noting that all what the getInstance() method does is call the private constructor for Session: new Session(props, null)).
Had Session been a more normal class, I could have mocked it like that:

Session mockSession = mock(Session.class);

and probably checked how it is being passed around like this:

verify(someService).startEmailSession(mockSession);

What follows is what I have to do instead.

The case of static methods

If getInstance() had been an instance method, I would have been able to mock it:

Session mockSessionProvider = mock(Session.class);
Session mockSessionInstance = mock(Session.class);
when(mockSessionProvider.getInstance()).thenReturn(mockSessionInstance);

Unfortunately, a static method cannot easily be mocked with Mockito (other mocking frameworks support that, but I believe they make the tests too obscure). So, first step: create a wrapper class just for the builder method.

public class SessionProvider {
	public Session getInstance(Properties properties) {
		return Session.getInstance(properties);
	}
}

The case of final classes

So far, so (relatively) good. However, since Session is a final class, the usual mocking mecanism under Mockito does not work:

SessionProvider mockSessionProvider = mock(SessionProvider.class);
Session mockSessionInstance = mock(Session.class); // fails because Session is final
when(mockSessionProvider.getInstance()).thenReturn(mockSessionInstance);

The case of private constructors

Another option would have been to instantiate the session instance, but that fails too, since the constructor is private:

SessionProvider mockSessionProvider = mock(SessionProvider.class);
Session sessionInstance = new Session(properties, null); // fails because constructor is private
when(mockSessionProvider.getInstance()).thenReturn(sessionInstance);

A third way might have been to call the original Session.getInstance() to create the test object:

SessionProvider mockSessionProvider = mock(SessionProvider.class);
Session sessionInstance = Session.getInstance(properties);
when(mockSessionProvider.getInstance()).thenReturn(sessionInstance);

That would work, but that also opens a whole new can of worms. Session does not implement the equals() method, so if I want to check the value of the Session instance passed around, I must either:

  • compare with the pointer to the sessionInstance — the issue is that there is no way to check that the session has not been changed (any call to a setter method on the session instance will have no effect on the comparison of pointers)
  • implement a generic deep equal matcher for Mockito — something we did at my company, but that I am reluctant to do myself
  • use an argument captor to catch the instance passed and test — which produces a fair amount of awkward lines of test
  • create a custom argument matcher — which hides the comparison in a separate class or method
  • yet another is to wrap the instance of Session in yet another class, such as SessionInstance — which would require us to also wrap any service that takes an instance of Session

So we also need a wrapper for the Session instance:

public class SessionInstance {
	private final Session session;

	public SessionInstance(Session session) {
		this.session = session;
	}

	public Session getSession() {
		return session;
	}
}

Which forces us to change the SessionProvider:

public class SessionProvider {
	public SessionInstance getInstance(Properties properties) {
		return new SessionInstance(Session.getInstance(properties));
	}
}

At last, we can mock this way:

SessionProvider mockSessionProvider = mock(SessionProvider.class);
SessionInstance mockSessionInstance = mock(SessionInstance.class);
when(mockSessionProvider.getInstance()).thenReturn(mockSessionInstance);

Done? Hardly. All other, possibly mockable, classes from third-parties that rely on instances of Session must now also by wrapper. For example, a third-party JavaMailTransport that requires an instance of Session now needs its own wrapper if I want to test it. Sigh…

Posted in java, tdd | 3 Comments

I avoid method variables in my test methods

Here is a typical example of a test method

@Test
public void should_search_by_path() {
	Searcher searcher = new Searcher();
	Path location = new Path("somewhere");
	String data = "data";

	searcher.putAt(data, location);

	assertThat(searcher.findAt(location), is(data));
}

It seems that many developers consider this good code. I don’t. My main gripe here is that the presence of variables does not add much useful information. Instead, they make the code too verbose.

A first rewrite would produce something like that:

@Test
public void should_search_by_path() {
	Searcher searcher = new Searcher();
	searcher.putAt("data", new Path("somewhere"));

	assertThat(searcher.findAt(new Path("somewhere")), is("data"));
}

Shorter, which is good. I also like the fact it makes clear that it is the value of the parameters that matter, and not their pointers.
We can go further by leveraging local static methods.

@Test
public void should_search_by_path() {
	Searcher searcher = new Searcher();
	searcher.putAt("data", path("somewhere"));

	assertThat(searcher.findAt(path("somewhere")), is("data"));
}

private static Path path(String path) {
	return new Path(path);
}

(a static method is not mandatory, but I think it makes its intention of being a utility method clearer)
And, finally, if this test class is dedicated to the Searcher class, I would probably make the searcher variable a field, as it would be used in most test methods.

private Searcher searcher = new Searcher();

@Test
public void should_search_by_path() {
	searcher.putAt("data", path("somewhere"));

	assertThat(searcher.findAt(path("somewhere")), is("data"));
}

private static Path path(String path) {
	return new Path(path);
}

This is more lines than the original example. However, the searcher variable is on top of the class and the path() method is at the bottom, far out of the way of the test methods. Indeed, I would often have several such static methods at the bottom of my classes (and, rarely, one or two more instance variables at the top). I even tend to do so when I have such a single method using them.

Beside the fact that the method has fewer lines, I also like that it makes for more fluent code. It almost reads like a story “when the searcher is being put a String of value ‘data’ at the path of name ‘somewhere’, then it should be able to assert that the String found at the path of name ‘somewhere’ has value ‘data’”.

Another example, so that my position is clear:

@Test
public void should_find_by_address() {
	String aValidAddress = "10 Downing Street";
	Person aPerson = new Person("David Cameron");
	contacts.put(aValidAddress, aPerson);

	assertThat(contacts.get(aValidAddress), is(aPerson));
}

I’d very much rewrite this as follows:

@Test
public void should_be_able_to_find__a_person_when_storing_with_a_valid_address() {
	contacts.put("10 Downing Street", person("David Cameron"));

	assertThat(contacts.get("10 Downing Street"), is(person("David Cameron")));
}

Any thoughts?

Posted in java | 5 Comments

Java’s varargs are for unit tests

At Devoxx last week, Joshua Bloch argued during his talk “The Evolution of Java: Past, Present, and Future” that varargs are only “somewhat useful”. I think he is overlooking some usages, particularly in tests. Here is my case.

A reminder on how varargs work: essentially, they allow the last parameter of a method to be made of zero to many values of the same type.
For example, a method like this:

int max(int... values) {...};

is used by code like that:

int maximum = max(1, 2, 7, 0);
int maximum = max(1);

In truth, that form is only moderately useful in production code. It turns out that, on my projects at least, it is not so common for methods to be called with varying number of parameters. And in those cases, it is often acceptable to simply overload a method with more parameters.

There are however at least two cases where varargs shine.

One is in utilities classes. For example, Math.max() only takes exactly two parameters. Which comparisons between 3 or more elements very awkward. If a varargs had been used, Math.max(3, Math.max(1, 4)) would be written Math.max(3, 1, 4). Agreed, that does not happen that often. But it does occasionally.
(I must admit that I do not understand exactly why Math.max() has not been modified to take a varargs nowadays)

However, the biggest benefit, I believe, is in the writing of tests.
Tests tend to contain lots of variations in the values passed to the code under test. And it is critical to keep the clutter to a minimum.
For example, here is how a typical series of tests might look like

@Test
public void should_find_the_longest_name_for_a_single_user() {
	List users = new ArrayList();
	users.add(new User("eric"));
	assertThat(findLongestName(users), is("eric"));
}
@Test
public void should_find_the_longest_name_when_shorted_is_first() {
	List users = new ArrayList();
	users.add(new User("eric"));
	users.add(new User("cecile"));
	assertThat(findLongestName(users), is("cecile"));
}
@Test
public void should_find_the_longest_name_when_longest_is_first() {
	List users = new ArrayList();
	users.add(new User("cecile"));
	users.add(new User("eric"));
	assertThat(findLongestName(users), is("cecile"));
}
@Test(expected = RuntimeException.class)
public void should_fail_when_search_for_the_longest_name_with_no_users() {
	assertThat(findLongestName(new ArrayList()), is("eric"));
}

Of course, they can be made a bit nicer using the varargs in Arrays.asList():

@Test
public void should_find_the_longest_name_for_a_single_user() {
	assertThat(findLongestName(asList(new User("eric"))), is("eric"));
}

@Test
public void should_find_the_longest_name_when_shorted_is_first() {
	assertThat(findLongestName(asList(new User("eric"),
		new User("cecile"))), is("cecile"));
}

@Test
public void should_find_the_longest_name_when_longest_is_first() {
	assertThat(findLongestName(asList(new User("cecile"),
		new User("eric"))), is("cecile"));
}

@Test(expected = RuntimeException.class)
public void should_fail_when_search_for_the_longest_name_with_no_users() {
	findLongestName(Arrays.asList());
}

However, the real goodness comes when writing our own builder method:

private static List users(String... names) {
	List users = new ArrayList();
	for (String name : names) {
		users.add(new User(name));
	}
	return users;
}

(side note: I like this type of methods to be private -so that I get notified when they are not used anymore- and static -mostly because they look nicer in italic, which makes it clearer that they are not part of the code being tested-)

which allow our tests to become:

@Test
public void should_find_the_longest_name_for_a_single_user() {
	assertThat(findLongestName(users("eric")), is("eric"));
}

@Test
public void should_find_the_longest_name_when_shorted_is_first() {
	assertThat(findLongestName(users("eric", "cecile")), is("cecile"));
}

@Test
public void should_find_the_longest_name_when_longest_is_first() {
	assertThat(findLongestName(users("cecile", "eric")), is("cecile"));
}

@Test(expected = RuntimeException.class)
public void should_fail_when_search_for_the_longest_name_with_no_users() {
	findLongestName(users());
}

Tests become a lot shorter, consistent, and easier to read. In fact, at this point, it is worth grouping the tests (at least those that do not throw an exception) into a single one:

@Test
public void should_find_the_longest_name_for_a_single_user() {
	assertThat(findLongestName(users("eric")), is("eric"));
	assertThat(findLongestName(users("eric", "cecile")), is("cecile"));
	assertThat(findLongestName(users("cecile", "eric")), is("cecile"));
}

@Test(expected = RuntimeException.class)
public void should_fail_when_search_for_the_longest_name_with_no_users() {
	findLongestName(users());
}

Final note: I prefer not to group those builder methods into a general utilities class — I tend to duplicate them in each test class. One reason is that I prefer keeping much of the test code close together. Another is that I tend to change the name of the method from test class to test class to make the code more fluent. Also, I think that DRY principles are not as important in the tests as they are in the production code.

Posted in conferences, java, tdd | 2 Comments

Play Framework and Guice: use providers in Guice modules

Play Framework has a Guice module. Unfortunately, its use is fairly limited compared to what Guice can do. In this post, I describe how it is configured on my current personal project.

Spring in Paris, when days last longer and pianists play in the street

In general, when I test classes that have dependencies, my favorite approach is to pass those dependencies to the constructor:

public class MyService {
	private final MyDependency myDependency;

	public MyService(MyDependency myDependency) {
		this.myDependency = myDependency;
	}
}

Which makes creating test harnesses with Mockito reasonably easy:

public class MyServiceTest extends UnitTest {
	@Test
	public void shouldComputeAResult() {
		MyDependency mockMyDependency = mock(MyDependency.class);
		when(mockMyDependency.findSomeValue()).thenReturn("some value");

		MyService service = new MyService(mockMyDependency);

		assertThat(service.computeSomething(), equalTo("result"));
	}
}

However, it appears that Guice, at least under version 1.1.1 of its Play module, can only be used with Play to inject into static members:

@InjectSupport
public class MyService {
	@Inject
	static MyDependency myDependency;
	// no constructor (a default constructor will be
	// generated automatically by Play Framework)
}

The @InjectSupport is what makes the Guice module detect that the class requires dependencies (a class that extends either com.google.inject.AbstractModule or play.modules.guice.GuiceSupport must also be present in the classpath).

This makes it hard to instrument classes under test, as the value for a mock instance of MyDependency is shared amongst all tests that run currently.

public class MyServiceTest extends UnitTest {
	@Test
	public void shouldComputeAResult() {
		MyDependency mockMyDependency = mock(MyDependency.class);
		when(mockMyDependency.findSomeValue()).thenReturn("some value");

		MyService service = new MyService();
		// static values are brittle;
		// they should not be touched in unit tests
		service.myDependency = mockMyDependency; // yuck!

		assertThat(service.computeSomething(), equalTo("result"));
	}
}

Many subtle problems may occur when injecting dependencies in this way. For example, all instances of MyServices used concurrently will point to the same dependencies instances, so it is possible that tests will behave in inconsistent ways. Also, it makes it harder to default to sensible (null) dependencies, since tests running in the same JVM will by default use dependencies set by the first test.

The solution I’m using is to add providers in your Guice module:

public class GuicyModule extends AbstractModule {
	@Override
	public void configure() {
		// no code needed
	}

	@Provides
	public MyService getMyService(MyDependency myDependency) {
		return new MyService(myDependency);
	}
}

In this way, you are making sure that the dependencies are passed as you wish. Also, it will not be necessary to add any sort of Guice-related code in my services:

public class MyService {
	private final MyDependency myDependency;

	public MyService(MyDependency myDependency) {
		this.myDependency = myDependency;
	}
}

The downside is that you do have a few more lines to maintain, which is never fun. Better than the alternative, though.

My thanks to David Gageot who told me about providers in Guice.

Posted in java, test | Tagged , , | 1 Comment

CITCON London 2010

I’m returning from CITCON London 2010. What a great conference (and I’m not just saying that just because I helped organize it)!

Break Sponsors

In fact, I feel it has been the best CITCON so far. I was a bit afraid of the large crowd (150 people registered, a similar number to Paris last year; I’m not sure how many showed up. 120, maybe?), but it turned out easier than expected to discuss with other participants. Also, and most importantly, there was a feeling of a higher level of experience than usual. Few talks about the basics of tests or Continuous Integration (and no “what’s the best CI server” session at all, thank God). Instead, it was “Advanced TDD”, “Share Pair Programming experience”, “Mobile Testing”, etc. All good stuff and, as usual, I just couldn’t attend all the sessions I wanted.

As a side note, there were also less talk related to competing programming environment. Only a few people introduced themselves as working in Java, .NET or another programming environment. I can think of a couple of reasons:

  • Java is so overwhelmingly everywhere that there is just no point bragging about it anymore
  • the .NET crowd (and possibly others) have given up on trying to make their work environments better. I didn’t attend the one “CI in .NET” session (there were no Java-specific sessions), but I was told that it was a slightly discouraging share of thoughts such as “it wasn’t my choice, but I want to make the best of it”, “I like .NET but the environment is just lacking” or “seriously, what do *you* guys do to implement CI in .NET?”.

In some ways, the Java world is also calcifying around some tools such as Maven (some contenders are barking at the door, though). But the feeling is that, by and large, things get done is a fairly productive way. (I’ll admit that I can be overly optimistic about this, considering that I am in a Java shop (Algodeal) where the incredible freedom helps us being particularly efficient)

Now, I was surprized that few mentioned up-and-coming languages such as Scala and Clojure. Are they just being lumped into the Java category? Was CITCON London the wrong area to find experts in them?

Other highlights of the conference include the venue. Thank you ever so much, Wendy, for opening the doors of SkillsMatter. You went out of your way. The location was fantastic, and the venue perfect for us. I especially appreciated the powerful wifi internet access, a rarity even in fancy hotels.
I also appreciated the fact that this area London has large pubs that can host all willing participants after the conference is over (this was frustrating in Paris last year, where bars can be small and especially crowded on Friday evenings). Too bad music can be so loud there, making conversations sometimes difficult.

A few things to remember from CITCON London:

  • Narrative, by the good folks at youDevise is a new test framework/helper. They present it as a BDD framework, but I rather see it as a way to enforce readability in your test classes. I like the modest, KISS, approach too. Plus, it seems easy to extend. I wrote a quick sample here.
  • Continuous Deployment was a big topic, as expected. There were few mentions on specific tools or techniques, but the feeling was that the practice was getting mainstream.
  • Apparently, a big framework in the JavaScript TDD space is QUnit. The session had left a lot to be desired (I left mid-way, but I’m told the end was good), but it at least drove the point that TDD was getting common in JS.
  • Again, I left with the feeling that we are not doing enough to generalize Pair Programming at Algodeal.

For CITCON 2011, there were many votes for Berlin. Sounds like a good destination. Again, it will depend on whether we can get a cheap enough venue there.

Continuous Deployment

Posted in citcon, java | Leave a comment

Why aren’t there more Agile luminaries developing and selling software?

Have you noticed that few Agile luminaries earn a living from writing and selling software? Many do write code as consultants. Other are respected authors of non-commercial open-source development tools. Some do work for software companies such as RallyDev or ThoughtWork Studios, though it seems that most visible presenters coming from there are consultants or at least business-facing types. But almost none actually make money directly by doing what they teach others to do.
James Shore mentioned working on his own startup with Arlo Belshee and Kim Wallmark, but that was more than a year ago and we haven’t heard much since. Ward Cunningham is CTO of a website, which come reasonably close to being a software house. In fact, Kent Beck is the only example I know of someone who actually tries to make a living out of writing and selling software (with mixed results). Tellingly, Ward and Kent are not very visible on the conference circuit anymore (though they are certainly interviewed regularly).

Programmer on "Vacation"

This lead to an interesting discussion yesterday on Twitter with Deborah Hartmann Preuss, Alexandru Bolboaca, Willem van den Ende, Brian Marick, and Jeffrey Fredrick (see transcript at the end of this post).

I think most Agile personalities have become addicted to the relatively easy money of consulting. Why would they risk get a software product out in the highly competitive software market? It is so easy to just found your own consultancy. It is probably related to age, too, as you do need to have lots of spare energy for late night coding (Paul Graham once wrote that the ideal age to start a startup was between 23 and 38).

In fact, as I argue below, I do not believe that “being agile” is viewed as a desirable trait for a startup and I’m sure it might even deter some of the most likely candidates to create one, as it is now viewed as a process for medium to large companies. What is viewed as needed is raw hacking powers, even if that means making things hang together with duct tape. Agile techniques might be preferable, but their ROI will become mostly apparent after two years, an eternity for a startup.

So, Agile luminaries are not starting software development ventures, and the founders of successful startups that use Agile techniques probably do not have the time or the will to tell the rest of the Agile world how they did it.

This is regrettable. I wish there was more cross-pollination, like 37signals has done with Getting Real. Where are the others?

Transcript of our conversation on twitter:

  • elefevre Wondering why so few Agile luminaries are into the commercial software business.
  • alexboly @elefevre I think Kent Beck said it best: “As a business man, I’m a very good software developer” :)
  • elefevre @alexboly interestingly, he seems to be the only one actually trying to make a living selling software (seems hard)
  • mostalive @elefevre because they are too busy consulting or marketing? want to publish own commercial product? work happens in odd hours -> slow
  • alexboly @elefevre As a programmer I would rather live in Castalia (http://goo.gl/AzQn) As a business man, I need to live in the real world.
  • elefevre @mostalive sure they do consulting. But if agile dev is really better, then products they would make ought to be better too. => Profit?
  • DeborahH @elefevre I guess Mike Beedle (PatientKeeper) is an exception, then. Co-author of first Scrum book.
  • mostalive @elefevre Profit yes, and, consulting & product development don’t really mix, as the business drivers point in opposite directions.
  • DeborahH @elefevre ken Schwaber came from sw dev company iirc, but perhaps you can’t change the world AND refactor your codebase at once. Or?
  • mostalive @elefevre agile tool vendors may be the exception, however their consultants can not recommend the best tool for the context (post-its ;))
  • mostalive @elefevre and if a product is successful, it generates so much revenue, that consulting revenue is irrelevant.
  • DeborahH @elefevre … Then they wouldn’t be luminaries in the sense of “visible as a teacher, writer” imo: these things are jobs in themselves.
  • DeborahH @elefevre though I respect more those who take time for practice as well, it grounds one’s teaching. But usually on s/one else’ project?
  • elefevre @DeborahH yes. Maybe if you’re busy developing (not consulting), then you have less time & inclination helping others.
  • elefevre @mostalive also, I suspect majority of (or the best known) consultants from Agile Tool vendors are more business-facing type than developers
  • DeborahH @elefevre until recently, Jeff was CTO @ PatientKeeper.http://bit.ly/ctQJRb Once yr own house is in order, you gotta get a new challenge?
  • elefevre @DeborahH @mostalive @alexboly I think my point could be: are successful/rich developers/hackers really applying Agile techniques?
  • elefevre @DeborahH @mostalive @alexboly or is Agile orthogonal unnecessary for a successful startup? (maybe I’ve been reading too much Paul Graham)
  • elefevre @DeborahH @mostalive @alexboly I think that, in a startup, techniques might be “agile”, but do not need to be identified as such
  • elefevre @DeborahH @mostalive @alexboly while, in a bigger biz, it helps that people can relate to a consensual definition of development process
  • elefevre @DeborahH @mostalive @alexboly in our startup, we are 3 former Agile consultants (out of 5 devs), but we take care not to say we’re agile
  • alexboly @elefevre @DeborahH @mostalive Agile doesn’t exist. Agile tools exist. Do you need agile tools for a succesful startup? I think you do.
  • alexboly @DeborahH @mostalive @elefevre Do you need to call the agile tools “agile” in order do succeed? Of course not. :)
  • DeborahH @elefevre “Agile” is a means to an end. You cannot afford to primarily “be agile”n you must be primarily busines owners!
  • alexboly @DeborahH @mostalive @elefevre The point was never to “be agile”, it always was to build software in a way that increases the success rate.
  • elefevre @DeborahH @mostalive @alexboly there might even be a stigma to calling things “agile” in a startup. Not hip (anymore)
  • DeborahH @elefevre surely “hipness” is not more important than profitability? Like “agility”, “hipness” sounds like a red herring, to me.
  • alexboly @elefevre @DeborahH @mostalive I’d like to see that people don’t talk about agile anymore but do the right thing. Human nature loves labels.
  • elefevre @DeborahH certainly is a red herring. But I do think it’s important for startups, often packed with smug hackers. #overgeneralization
  • marick @elefevre I don’t have any good product ideas. I don’t have skills to do it all & it’s a big step from 1-person company to N-person company.
  • elefevre @marick my thinking is that the mobile market is more tolerant of seemingly mediocre ideas than the desktop/enterprise space
  • DeborahH @elefevre @marick perhaps because in mobile market short release cycles allow products to “grow up” in public?
  • elefevre @DeborahH @marick I think it’s because there are fewer high-standard apps, so users tolerate mediocre ones. eg. quizzes, website wrappers
  • Jtf @elefevre I think people who are successful consultants are addicted to fast feedback from helping. Products are delayed gratification.
Posted in agile | 13 Comments

How to use LogMeIn under Linux

For my Remote Pair Programming session with Alexandru Bolboaca, I wanted to work on our actual code, not toy programs. It was hard finding a technical solution to allow this (despite the many suggestions I received on Twitter; the biggest issue is sharing the entire development environment), but I finally settled on LogMeIn. LogMeIn basically lets you create an ad hoc VPN with them serving as a middle man. The great thing with it is that all the configuration is done on the client machines. There is nothing to change on firewalls (especially important for the other people that you are working with).

LogMeIn has a download that seems very simple to use… as long as you are under Windows. It also has a Mac OS X version and a Linux version, but they hardly come with any documentation. What’s worse, it is hard to find additional information on the support site.

So, for your eyes only, here are some instructions on how to get LogMeIn to run under Linux. (this applies only to the client machine; setting up the network can be done entirely on LogMeIn’s website)

The worlds network

My configuration: Ubuntu 10.04 Lucid Lynx 64 bits with LogMeIn Hamachi 2.0.0.11-1. (Hamachi is a protocal that creates a VPN that goes through their servers)

  1. First, create a login on http://www.logmein.com/
  2. Install their Linux client. Just double-clicking it after download should be enough.
  3. Configure the client in the command line :
    1. hamachi login
    2. hamachi attach <your email on logmein>
    3. hamachi set-nick <a human-readable user name for you; any should do>
    4. hamachi do-join <id of the VPN previously created on LogMeIn>
      • The password is the one specified by the domain creator. This is not the password for your login.
  4. Wait for domain creator to approve your machine on the virtual network (you might need to send an email to remind her of that)

Done! From that point, you can use VNC or anything else to connect to a remote computer. Use something like ifconfig on the remote computer and use the IP address under the ham0 entry (ham is for Hamachi, obviously). The IP address has an unusual value such as 5.18.76.84.

Posted in pair programming | 7 Comments

How we use Git at Algodeal

I’ve talked recently with the CTO of a small-but-successful company, trying to explain how we do software development. I realized that many things are difficult for them to copy from us, mostly because we have a different approach to implementing features (in particular, we try to limit GUI-intensive features, while they have a very rich Javascript interface).

One thing, however, that I believe they can adopt without changing their code is the source code management tool Git. However, they had already considered it (they are currently using Subversion) and figured that it does not solve problems they already have.

The Gits

I’d agree that Git doesn’t fix obvious problems. However, Git is powerful enough (once the complexity is mastered) to make lots of little things easier. Here is what it does for us.

  • It makes merges much less painful, even when no branched are involved, for example when two developers modify the same file. With Git, it is possible for someone to move a file to a different package, while another developer renames it simultaneously. And removes half of the content. Git (almost always) magically resolves the conflicts and maintains the history of the file, all without any special command. No need for “svn move” or “svn rename”. And no code freeze when someone renames a package, so we tend to do refactorings more often.
  • When a regression is detected late, Git can help find the faulty commit. That can be done automatically if you can write a script that detect the bug (for example if there was no automated failing test in the faulty commit), but that’s not necessary. For example, it helped us find at what point “mvn eclipse:eclipse” stopped working (it was because we enforced maven 3 usage in the POMs). The command that does this is “git bisect.”
  • The entire source repository is on the developer’s machine (this is configurable). This gives us free backup across all our machines, and instantaneous access to logs. In the same spirit, I frequently use git blame to find out who wrote what on a particular piece of code, so that I can ask them for clarifications.
  • There are no access rights to manage (though users’ accounts on the server machine, if you choose to have one, as many do, are still necessary).
  • Suppose you’re working on something, but must suspend that work temporarily to fix a bug. In SVN, either you cancel all your changes, or re-download the entire code base in a separate directory. Or count on the fact that changes won’t overlap (maybe). In Git, you can temporarily put your changes on the side, do your fix, then retrieve them back again. This is known as “git stash“.
  • If by any chance it is necessary to roll a fix in production, it is possible to hand-pick changes with “git cherry-pick“. My colleagues often use this.
  • Generally speaking, there is much less mangling of the code base (none of the infamous “svn cleanup“)

None of these features are absolutely necessary. But all together, they make life easier for us. It even let do serverless CI.

Sebastien Douche has said during a presentation at a recent Paris JUG evening that DVCS are the one thing that all developers should learn in 2010. I think he’s right.

Posted in algodeal, source control | Leave a comment

Bob Martin on TDD in Clojure

Robert “Uncle Bob” Martin has just blogged on the differences in TDD styles using Clojure, as compared to more traditional languages such as Java. Though I am a Clojure-newbie, I mostly disagree with his conclusions.

His main point is that, because Clojure is a functional language, functions have no side-effects and therefore can be used directly in the tests.

For example, the production code

(defn update-all [os]
  (update os))

would be tested with something like

(testing "update-all"
  (let [
    o1 (make-object ...)
    o2 (make-object ...)
    os [o1 o2]
    us (update-all os)
    ]
    (is (= (nth us 0) (update o1)))
    (is (= (nth us 1) (update o2)))
    )
  )

There is no reason to believe that the (update) function is side-effect-free

Changing internal values is only one way of creating side-effect. I admit that Clojure encourages coders to write code that does not change variables (if I got it right, it is definitely possible to do so, but with some additional work). However, that effect only stops at the boundaries of the language. At some point, it might access the file system or a database. Clearly, the state might change there.

Correct implementation of the (update-all) function depends on the correct implementation of (update)

Bob Martin says: ”this test simply checks that the appropriate three functions are getting called on each element of the list”.
Suppose that the (update) function does not do anything or maybe does something that does not return a value, such as printing out to the console. Then, calling it will have the same effect as not calling it at all. The test above will pass even if the (update-all) function does not provide any implementation at all. When, later, the bug is found, it will be harder to fix.

The test could be clearer (with a more powerful test framework)

One of my biggest concerns is that the test looks a lot like the code itself. Looks like duplication of information to the reader.
If there was a mock framework for Clojure, I would expect to see something like

(testing "update-all"
  (let [
    pre-conditions (
      (should-return (update 1) 1.5)
      (should-return (update 3) 3.0) )
    o1 (make-object 1)
    o2 (make-object 3)
    os [o1 o2]
    us (update-all os)
    ]
    (is (= (nth us 0) (1.5)
    (is (= (nth us 1) (3.0)
    )
  )

Bob Martin is right to conclude that “Clojure without TDD is just as much a nightmare as Java or Ruby without TDD.”

But he should also make it clearer that it is lacking a mock framework (he does point to Brian Marick’s work on this).

It should be noted that it is possible to get a similar implementation style in Java as in Clojure, though it is significant work. In fact, that’s often how we use it here at Algodeal. That means mostly relying on immutable objects and state-less methods. Immutable collections from Google Collections help a lot, too. Still, we like to use mocks in our tests (too much for some, probably).

In the end, Uncle Bob’s post is another aspect of the (almost) age-old debate described by Martin Fowler: classicists vs. mockists. If you haven’t already, read Fowler’s article, it’s worth it.

Posted in tdd | Leave a comment

AppArmor: how to fix the Create New User issue with logprof

We have started to use AppArmor as a way to strengthen the security on our platform. A reasonably good tool for which you can find rather straightforward tutorials.

Portrait of a young woman dressed as Boadecia or Mother England
Portrait of a young woman dressed as Boadecia or Mother England by Powerhouse Museum

AppArmor is a tool that can explicitly allow or deny actions by some applications. Those actions are recorded in a profile. Many profiles are already available, for such tools as Firefox, but sometimes it is necessary to create your own. This was the case for us: we wanted to make sure using Mono was not too much of a risky endeavor (we run investment strategies written by our users in .NET).

Creating a profile for AppArmor can be done is a couple of ways. One is to run the application you want to lock down in record (“complain”) mode, check the logs produced by AppArmor and select the corresponding rules you want to enforce. The tool that checks the logs and produces the profile containing the rules is logprof, generally run with the aa-logprof command.

Last morning, when I merrily tried to run logprof, it prompted me at the end of the process to create a user.

Create New User?
(Y)es / [(N)o]

I didn’t know what “Creating a user” meant here. And, at this point, there is nothing useful to do. Whether you reply Yes or No, you are always prompted for a username and password, then asked whether you want to save the configuration, which inevitably ends with a Login failure, and you are back to the Create a User question (Ctrl-C to get out). Here is the whole trace:

Updating AppArmor profiles in /etc/apparmor.d.
Create New User?

(Y)es / [(N)o]
Username: noideawhatmyusernameis
Password: noideawhatthepasswordis

Save Configuration?

[(Y)es] / (N)o

Login failure
Please check username and password and try again.
RPC::XML::Client::send_request: HTTP server error: Not Found

Create New User?

It took me a while to understand what was going on, so I’m writing this post in the hope that it will help someone (possibly me, in the not-so-distant future).

The user here refers in fact to a user on the central, public repository for AppArmor profiles. You do not normally need a login for downloading profiles, but logins are required to upload them. Now, I obviously do not want to upload my profiles, so what’s the deal?
In all likelihood, I must have enabled the upload of profiles at some point, possibly when I was trying to figure out what AppArmor was doing. There is a way to undo that, but very little documentation and few discussions about it on the internet.
I finally found it on Novell’s site.

In the end, here is what you need to do:

  1. move to /etc/apparmor (and not /etc/apparmor.d, which is the directory where the profiles are saved)
  2. edit repository.conf
  3. in the [repository] section, replace upload = yes with upload = no

All done!

Footnote: the status of AppArmor is not clear to me. Wikipedia indicated that Novell has fired the original team that developed it and indeed Novell’s site only points to AppArmor v2.1 and earlier. A Google search returns many links to Ubuntu and indeed Karmic Koala comes with v2.3.1 (the latest, AFAIK), but Ubuntu pages do not offer very advanced documentation. Novell does have the best documentation but is strangely not well referenced on Google and the documentation only goes until v2.1, which is not impressive. The official development site is hosted by Novell, but it only mentions v2.3beta and has not seen any release seen mid-2008. Finally, a similar tool, Tomoyo, has been merged into the Linux kernel v2.6.30 since mid-2009. So I think that, once we will have move all our servers to Karmic, we’ll dump AppArmor.

Posted in unix | 6 Comments

Predictions for CITCON Europe 2009

Last year, at CITCON Amsterdam 2008, a few of us stayed late into the night, drinking beer and discussing the state of the world.

And what to do when you have 21 geeks with time on their hands? Why, predictions, of course! (I want to do it again this year, check out the Google Moderator page I’ve started)

Bar at the Marriott Hotel

We decided to come up with a number of predictions (and bet on them), some serious, some not, that would be verified at CITCON Europe 2009, the price being beer points. And, the losers will be named and shamed, while the winners will be glorified (at least until new predictions are made, and for no more than a year, whichever is earliest).

Here are the predictions, and the actual outcome (a couple of them were settled by votes at the closing session):

Prediction Votes Actual
CITCON Europe has more than 120 attendees (I had voted against!!) YES YES
more .NET developers than Java developers NO NO
CITCON will take place in Paris YES YES
at least 5% of attendees are female (I personally did vote in favor) yes NO
at least 20% of participants do Ruby draw NO
Java closures are considered too complex NO NO
IBM buys ThoughtWorks NO NO
IBM buys Valtech NO NO
there is a Maven.NET coded in Java, with MS Tools integration NO NO
Ivan Moore gives up on build-o-matic NO NO
McCain wins the election draw NO
CITCON Europe takes place in Frankfurt NO NO
Jeffrey Fredrick XOR Tom Sulston (that is, either Jeffrey or Tom, but not both) have short hair YES YES
Fewer Agile Consultancies NO NO

So, out of 14 predictions, we got 11 right, 1 wrong, and 2 undecided.
Now, you may think that the answers were straightforward. But you need to realize that, for each one of them, someone was willing to bet a beer against the consensus. In other words, at the time when the predictions were made, it was not clear cut.

In the interest of the bets, I shall now reveal the names.
Winners (Glory to Them All!)

  • Andrew Parker (8 rights, 1 wrong)
  • Eric Lefevre (that’s me) (10 rights, 2 wrongs)
  • Guillaume Tardif (6 rights, no wrongs)
  • Jean-Michel Bea (8 rights, 2 wrongs)
  • Pekka Pietikäinen (7 rights, 2 wrongs)

Losers (Boo to Them All!)

  • Julian Simpson (3 rights, 4 wrongs)
  • Jeffrey Fredrick (3 rights, 6 wrongs)
  • Paul Julius (5 rights, 2 wrongs) — PJ is still a loser, ‘cos he has been right on bets with small payoffs
  • Tom Sulston (4 rights, 4 wrongs)

I have started a new series of predictions for CITCON Europe 2010. There are two steps:

  1. suggest predictions & vote for the best ones
  2. when predictions have been selected, vote

To actually win your beers, you’ll have to come to CITCON Europe 2010 (still unannounced).
Please check out the Google Moderator page to propose your own predictions.

Sorting out the bets from 2008If you want to the gritty details, I have a picture of the full spreadsheet.

Posted in citcon | Leave a comment