Skip Navigation LinksWindows Azure Service Bus Developer Guide > Brokered Messaging > Brokered Messaging Features & Patterns > Handling Duplicate Messages > Configuring Duplicate Message Detection

Train with Alan
Alan Smith
Email: cloudcasts.net@gmail.com Blog: geekswithblogs.net/asmith Twitter: @alansmith Linked In: Alan Smith

Configuring Duplicate Message Detection

Duplicate message detection on a queue or topic is configured when the messaging entity is created by specifying properties in the appropriate description class. Once the messaging entity has been created these properties cannot be modified.

RequiresDuplicateDetection

When set to true, the RequiresDuplicateDetection property will ensure that all enqueued messages with a duplicate value of the MessageId property within the DuplicateDetectionHistoryTimeWindow will be ignored.

DuplicateDetectionHistoryTimeWindow

If duplicate detection is enabled for a queue the DuplicateDetectionHistoryTimeWindow value will specify how long the MessageId values for the received messages will be retained in order to check for duplicate messages. The property is a TimeSpan with a default value of 10 minutes and a maximum allowed value of 7 days.

The following code will create a queue with duplicate detection enabled and the duplicate detection history time window set to one hour.

// with a detection history window of one hour.

QueueDescription orderQueueDescription =

    new QueueDescription("orderqueue")

    {

        RequiresDuplicateDetection = true,

        DuplicateDetectionHistoryTimeWindow = new TimeSpan(1, 0, 0)

    };

 

// Create a queue that supports duplicate detection.

namespaceMgr.CreateQueue(orderQueueDescription);

 

 

Affects of Duplicate Detection on Queue or Topic Size

Enabling duplicate detection on a queue will mean that the unique MessageId values of all messages received by queue will have to be stored for the duration of the duplicate detection history time window. The storage of these values will result in an increase on the queue size.

In order to test this a simple application was created to send and receive 100 messages and examine the queue size in bytes. The code to send the messages and output the queue size is shown below.

 

Console.Write("Sending 100 messages");

for (int i = 0; i < 100; i++)

{

    dupeQueueClient.Send(new BrokeredMessage("Hello, " + i));

    Console.Write(".");

}

Console.WriteLine("Done!");

 

Console.WriteLine("Queue size: {0} bytes",

    namespaceMgr.GetQueue("dupetestqueue").SizeInBytes);

 

 

When the application is run using a queue without duplicate detection the results are as follows.

The application was then modified to use duplicate detection with a history window of one minute and to display the queue size after the history window for the messages has expired.

 

// Create a queue without duplicate detection.

QueueDescription dupeQueueDescription =

    new QueueDescription("dupetestqueue")

    {

        DuplicateDetectionHistoryTimeWindow = TimeSpan.FromMinutes(1),

        RequiresDuplicateDetection = true

    };

namespaceMgr.CreateQueue(dupeQueueDescription);

 

// Send and receive messages...

 

Console.ForegroundColor = ConsoleColor.Cyan;

Console.WriteLine("Queue size: {0} bytes",

    namespaceMgr.GetQueue("dupetestqueue").SizeInBytes);

Console.ResetColor();

 

Thread.Sleep(70000);

Console.WriteLine("DuplicateDetectionHistoryTimeWindow expired");

 

 

Console.ForegroundColor = ConsoleColor.Cyan;

Console.WriteLine("Queue size: {0} bytes",

    namespaceMgr.GetQueue("dupetestqueue").SizeInBytes);

Console.ResetColor();

 

 

 

The results show that there is an overhead of 6,400 bytes when sending 100 messages on the queue for the duration of the duplicate history time window. This works out at 64 bytes per message, which is the size required to store the MessageId values for the messages.

Bear in mind that setting a large value for the duplicate history time window on a queue that has a high message throughput. If the duplicate history time window for a queue is set to the maximum of 7 days for a queue that processes an average of 28 messages per second over 1 GB of storage will be required to store the MessageId values.