Nate Kohari informed me that he fixed the performance issue that I discovered in my last test, so I thought I rerun the benchmark against the trunk version (revision 62) of Ninject.

I have also included another new container named Autofac, written by Nicholas Blumhardt and Rinat Abdullin.

This container has nice list of features, here are a few:

  • Autowiring (with out any intrusive attributes)
  • XML Configuration Support
  • Nice C# registration API (including possibility to create components with expressions)
  • Module system (nice way to structure parts of your application)
  • Nested containers

The registration API is at first glance like any other IoC container except it also provides the ability to override the autowiring by defining how components are created by using lambda expressions. Here is an example:

// using autowiring
builder.Register<UserController>().FactoryScoped();
// using expressions
builder.Register(c => new UserController(c.Resolve<IUserRepository>(), c.Resolve<IAuthentificationService>()))
  .FactoryScoped();

The second method could potentially be a lot faster since in it you create a anonymous method that creates the object directly. There is a lot more code to write and if you change the constructor you need to update the registration code, so I am not sure you would want to use it to for most components. To be fair to the other containers in the test I tested Autofac with both autowiring registration and with expression based registration. For more detail on how the benchmark works please view the first benchmark post.

Another interesting feature of Autofac is it's support for nested containers with predicable component cleanup. 

var container = // ...
using (var context = container.CreateInnerContainer())
{
  var controller = context.Resolve<IController>();
  controller.Execute(); // use controller..
}

In the above example the controller and all it's dependencies that implements IDisposable will be disposed.

Here are the results:

IoCSingleton_Autofac

IoCTransient_Autofac

It is nice to see that the Ninject problem has been solved. When using Autofacs expression (lambda) based registration API the results are, not surprising, a lot quicker than the other containers. I would have almost through that the performance difference was going be bigger, seeing how fast the new operator was in my first benchmark (where I made a crude comparison with using the new operator directly to create all the instances).  

My conclusion is the same as after the first test, the performance difference is not significant to warrant consideration when choosing a IoC container. Unless you create a incredibly large amount of transient components (not recommended), then maybe use Autofac :)

It was fun to try out Autofac, it is looking like an interesting contender, and might even replace Castle Windsor as my favorite. I doubt it though, Castle Windsor probably has the largest user base and I am pretty familiar with it's codebase / extension points. But who knows, you shouldn't always stick to what you know :)

For the benchmark code: IoCBenchmark_ReRevisted.zip

11 comments:

Anonymous said...

Thanks for this interesting comparison, which pointed me to Autofac. I just love this littel thing :).

Torkel Ödegaard said...

no problem :)

Luis said...

Thanks for this nice comparison. I'm really curious about the degree of the accuracy.

You can try a couple of things to have the same feeling:

1. In BenchEngine.cs, reduce the loop from 1,000,000 to 100.(You know what we expect to see)

2. Change the execution order of the containers in Program.cs.(especially between AutoFacLambda and AutoFac)

It's strongly indicating there is a sort of startup cost for these IoC containers involved in the test,which affects the result dramatically.

Also, some observations in my test:

1.For BenchUnitContainer.cs, change RegisterType T to RegisterType(typeof(..... would make big difference, which made AutofacLambda not that shine. I made another BenchUnitContainer2.cs for for that comparison.

2. Unity 1.2 is fastest

Test I used:
Unity 1.2
StructureMap 2.53
(Reset() to ResetDefaults())
Autofac 1.4.2.530
Ninject 1.0.0.082

Thanks. Look forward to your next benchmark.

Mike Gleason jr Couturier said...
This comment has been removed by the author.
Mike Gleason jr Couturier said...

Hi,

Thanks for sharing all thoses IoC containers with us...

For curiosity, I switched from Castle Windsor to Autofac with lambda expressions.

I went from 115 pps to 145 pps (pages served per second) on my .NET MVC web site. (Athlon X2 2GHZ 1 gig RAM, SQL on same machine)

I stress test the site with jakarta jmeter and navigate to every possible page without any caching and almost always a database connection.

I know that numbers like you posted are to be taken with a grain of salt but if one day my site become extremely popular, I think it matters and can be a cost effective decision.

I thought I'd give you a real world scenario, proving that you didn't wasted you time and helped somebody pick an IoC container :D

Mike

Nick said...

Any chance you want to run this test against the latest Ninject 2 sources? know it's still in beta, but it'd be interesting to see if speed improvements were made since it's essentially a top to bottom rewrite.

Nick said...

I went ahead and modified your test project to use a new build of Ninject 2. Surprised to see the results jump up, but not sure what that means other than it looks slower than the other containers at first glance.

Can I send you a zip of the updated solution?

outoftouch said...

Can you add LinFu to your benchmark tests? You pretty much covered all of the ones I have been looking at but that one.

Martin said...

I have uploaded some updated results to www.IOCBattle.com

Please let me know if i have missed something.

Dynamo included is my own little project I created to see if I could create a simple IOC container that is faster than all the popular ones. I use it primarly with MVC where it works fine together with the DependencyResolver.

icanhasjonas said...

Hello World..

This is one of the fastest containers I've seen. It's new though - don't know much about the feature set. But seems cool.

All test driven developed so there's perfect code coverage. Includes a very simple/naive performance comparison.

Andres said...

Hi, nice work!
I have a small comment though, is there a reason why you haven't used the last version of Unity (v2.1)?
This latest version also supports lambda injection.