InfiniTec - Henning Krauses Blog

Don't adjust your mind - it's reality that is malfunctioning

Processing items in an Exchange folder using EWS Managed API

A requirement I often hear is something like this: “We have a mailbox that receives emails of some kind that need to be processed somehow”. What options are there to fulfill the requirement? As always, it depends. When the items need to be processed as soon as possible and the client processing the items can be directly reached by the Exchange server, push notifications are certainly a good choice. If, on the other hand, item processing is not time critical a suitable method of processing those items is polling. It’s much simpler to use than push notifications. A simple example might look like this:

   1: private static void ProcessItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset);
   5:  
  10:  
  11:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  12:  
  13:         foreach (var item in result)
  14:         {
  15:             ProcessItem(item);
  16:         }
  17:         offset += pageSize;
  18:     } while (result.MoreAvailable);
  19: }

This is a very naïve implementation as it always returns every item from the inbox folder of the mailbox. But at least it uses paging. This breaks the processing down from one very large request into many small requests (in this case 100 items are returned per request).

This method is suitable if you delete the processed items from a mailbox after they are processed. Of course, the items should not be removed until the every item has been processed. Otherwise items may be skipped since the offsets of the individual items change when an item is removed from the folder.

If the items are not removed from the store, the calling application must distinguish new items from items already processed. An obvious way to do this is to mark each item once it has been processed by marking it as read. This changes the requirement for the ProcessItems method: It should only process unread items. This modifications has been incorporated into the following example:

   1: private static void ProcessItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:         {
  11:             SearchFilter = new SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, false)
  12:         };
  13:  
  14:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  15:  
  16:         foreach (var item in result)
  17:         {
  18:             ProcessItem(item);
  19:         }
  20:         offset += pageSize;
  21:     } while (result.MoreAvailable);
  22: }

The only change in this example is in line 11: A search filter has been added to filter for unread items. What’s missing here is the modification of the Unread status of each message. And that is a considerable drawback of the whole solution. To touch each processed item, the UpdateItems method must be called. It is sufficient to call the UpdateItems method and pass all item ids from the last resultset to the method. But this adds a significant burden on the Exchange server and slows overall processing down. Furthermore, if someone accesses the mailbox (either with Outlook or Outlook Web Access) and accidently marks one or more items as read, those items will not be processed by the client application.

Next idea: Find all items that were received after the last time the mailbox was checked. A ProcessItems method that implements this behavior might look like this:

   1: private static void ProcessItems(ExchangeService exchangeService, DateTime lastCheck)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:                        {
  11:                            SearchFilter = new SearchFilter.IsGreaterThanOrEqualTo(ItemSchema.DateTimeReceived, lastCheck)
  12:                        };
  13:  
  14:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  15:  
  16:         foreach (var item in result)
  17:         {
  18:             ProcessItem(item);
  19:         }
  20:         offset += pageSize;
  21:     } while (result.MoreAvailable);
  22: }

As with the second example, the difference to the first one is line 11. A search filter has been added that restricts the FindItems call to those items received after a specific time. This method removes the requirement to mark each processed item on the server. But it adds another caveat: A mail that is received during a ProcessItems call will be missed if no additional checks are performed. This can be a little tricky.

Luckily, the Exchange WebServices offer a more suitable solution for the whole requirement: The SyncFolderItems method. This method not only solves the problems mentioned above but also returns deleted items, if such a processing is necessary. A ProcessItems method that uses this API now looks like this:

   1: private static string SyncItems(ExchangeService exchangeService, string syncState)
   2: {
   3:     const int pageSize = 100;
   4:  
   5:     ChangeCollection<ItemChange> changeCollection;
   6:     do
   7:     {
   8:         changeCollection = exchangeService.SyncFolderItems(new FolderId(WellKnownFolderName.Inbox),
   9:                                                            new PropertySet(BasePropertySet.FirstClassProperties), null, pageSize,
  10:                                                            SyncFolderItemsScope.NormalItems, syncState);
  11:  
  12:         foreach (var change in changeCollection)
  13:         {
  14:             if (change.ChangeType == ChangeType.Create)
  15:             {
  16:                 ProcessItem(change.Item);
  17:             }
  18:         }
  19:         syncState = changeCollection.SyncState;
  20:  
  21:     } while (changeCollection.MoreChangesAvailable);
  22:     return syncState;
  23: }

This method only processes newly created items and ignores all other item changes or deletions. The application calling this method needs to store the synchronization state between each call. If an empty sycnState is provided, Exchange will return every item from the mailbox as “Created” item. This makes it possible to process all existing items once and then, with the same logic, every changed item. The only drawback with this method is the fact that the synchronization state can become quite big (in a folder with ~4500 items, the syncstate has a size of approx. 60kb).


Posted by Henning Krause on Sunday, June 7, 2009 12:54 PM, last modified on Monday, November 29, 2010 9:21 PM
Permalink | Post RSSRSS comment feed

Comments (5) -

On 8/11/2009 9:37:21 PM Duke United States wrote:

Duke

Great writeup Henning. Keep up the good work. I&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;ve always found your articles and replies on technet very useful.

On 1/25/2012 5:47:32 AM asdfads wrote:

asdfads

asdfasdf

On 3/8/2012 3:20:17 PM Mark wrote:

Mark

I would like to know how to get the attachments for the items in your processiem for example, because when I try, the attachments are 0, while the message does have attachments.

On 3/8/2012 3:21:17 PM Mark wrote:

Mark

Here&amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;#39;s the code I use:

if (item.Attachments.Count &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;gt; 0)
{
  foreach (Attachment att in item.Attachments)
  {
    txtResult.Text += &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;quot;  &amp;amp;amp;amp;amp;amp;amp;amp;amp;amp;quot; + att.Name + Environment.NewLine;
  }
}

On 9/9/2012 12:29:31 AM Sathesh wrote:

Sathesh

Great. Really helpful.

BTW. Is there any workaround to reduce the initial number of sync of all the items for the SyncFolderItems method. Am writing up a mobile appln and I dont want to load all the thousands of messages to be cached.

 +Pingbacks and trackbacks (3)