Benchmarking qmail on Linux Filesystems

The Tests

Message Generation

For tests run without a specific message size, the message files were generated by fake-msg, which generates a randomly-sized message with an inverse exponential distributed size, with the mean set to 2000 as determined from the statistics.

Loop

In this test, an alias was set up that would generate new messages that would be delivered back to itself, causing a deliberate mail loop. By doing so, both the injection process and the queue processing is stressed.

Variables:

Procedure:

  1. Create the loop alias, using either a static message or fake-msg above.
  2. Inject N (load parameter) messages to the loop alias.
  3. Start qmail-send.
  4. Sleep T seconds.
  5. Replace the alias with an empty delivery ("#").
  6. Stop qmail-send.
  7. Count the number of deliveries made.

Discarded Tests

One of the first tests I ran was to measure the raw injection speed. The ratio of numbers from this test correlate roughly to the loop test, but their absolute value is meaningless. Also, this scenario does not take into account the beneficial effect that qmail-send/clean deleting messages has on ext3 with full data journalling.

Another test I ran was to stuff the queue full of messages and then let qmail-send blast them away to a null recipient. This is also unrealistic for some of the same reasons as the above.