InfiniTec - Henning Krauses Blog

Don't adjust your mind - it's reality that is malfunctioning

Getting the body of an Email with a FindItems request

The FindItem operation (or the corresponding ExchangeService.FindItems method) does not return the body of an email by default. And when trying to explicitly request them via a custom propertyset, the call fails. Consider this method, which uses the EWS Managed API to execute a FindItems method to get every message from the inbox folder of a mailbox, fetching only the item id and the body:

   1: private static void GetAllItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:                        {
  11:                            PropertySet = new PropertySet(BasePropertySet.IdOnly)
  12:                                              {
  13:                                                  ItemSchema.Body
  14:                                              }
  15:                        };
  16:  
  17:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  18:  
  19:         foreach (var item in result)
  20:         {
  21:             ProcessItem(item);
  22:         }
  23:         offset += pageSize;
  24:     } while (result.MoreAvailable);
  25: }

When executed, the ExchangeService instance throws a ServiceValidationException stating “The property Body cannot be used in FindItem requests”. The official workaround proposed by Microsoft is to use a FindItem request to get the item ids of the items in a folder and afterward issue a GetItem request containing all the item ids and request the body property. However, there is another solution: The body properties can be fetched by requesting the MAPI properties containing the body:

   1: private static ExtendedPropertyDefinition TextBodyProperty = new ExtendedPropertyDefinition(0x1000, MapiPropertyType.String);
   2: private static ExtendedPropertyDefinition HtmlBodyProperty = new ExtendedPropertyDefinition(0x1013, MapiPropertyType.Binary);

The first property definition can be used to fetch the text body of a mail. The second one fetches the Html body. The new GetAllItemsMethod now looks like this:

   1: private static void GetAllItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:                        {
  11:                            PropertySet = new PropertySet(BasePropertySet.IdOnly)
  12:                                              {
  13:                                                  HtmlBodyProperty
  14:                                              }
  15:                        };
  16:  
  17:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  18:  
  19:         foreach (var item in result)
  20:         {
  21:             object body;
  22:             if (item.ExtendedProperties.TryGetValue(HtmlBodyProperty, out body))
  23:             {
  24:                 Console.Out.WriteLine("item.Body = {0}", Encoding.UTF8.GetString(Convert.FromBase64String((string)body)));
  25:             }
  26:         }
  27:         offset += pageSize;
  28:     } while (result.MoreAvailable);
  29: }

The HtmlBody is requested with the addtion of the HtmlBodyProperty to the ItemView in line 13. Since the Html body is stored in binary form and returned in Base64 encoded format, it needs to be decoded before it can be displayed. This is done in line 24. If the plaintext body is requested, the value of the body property in line 24 can simply be converted to a string.


Posted by Henning Krause on Tuesday, June 9, 2009 9:00 PM, last modified on Wednesday, April 20, 2011 6:19 PM
Permalink | Post RSSRSS comment feed

Processing items in an Exchange folder using EWS Managed API

A requirement I often hear is something like this: “We have a mailbox that receives emails of some kind that need to be processed somehow”. What options are there to fulfill the requirement? As always, it depends. When the items need to be processed as soon as possible and the client processing the items can be directly reached by the Exchange server, push notifications are certainly a good choice. If, on the other hand, item processing is not time critical a suitable method of processing those items is polling. It’s much simpler to use than push notifications. A simple example might look like this:

   1: private static void ProcessItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset);
   5:  
  10:  
  11:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  12:  
  13:         foreach (var item in result)
  14:         {
  15:             ProcessItem(item);
  16:         }
  17:         offset += pageSize;
  18:     } while (result.MoreAvailable);
  19: }

This is a very naïve implementation as it always returns every item from the inbox folder of the mailbox. But at least it uses paging. This breaks the processing down from one very large request into many small requests (in this case 100 items are returned per request).

This method is suitable if you delete the processed items from a mailbox after they are processed. Of course, the items should not be removed until the every item has been processed. Otherwise items may be skipped since the offsets of the individual items change when an item is removed from the folder.

If the items are not removed from the store, the calling application must distinguish new items from items already processed. An obvious way to do this is to mark each item once it has been processed by marking it as read. This changes the requirement for the ProcessItems method: It should only process unread items. This modifications has been incorporated into the following example:

   1: private static void ProcessItems(ExchangeService exchangeService)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:         {
  11:             SearchFilter = new SearchFilter.IsEqualTo(EmailMessageSchema.IsRead, false)
  12:         };
  13:  
  14:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  15:  
  16:         foreach (var item in result)
  17:         {
  18:             ProcessItem(item);
  19:         }
  20:         offset += pageSize;
  21:     } while (result.MoreAvailable);
  22: }

The only change in this example is in line 11: A search filter has been added to filter for unread items. What’s missing here is the modification of the Unread status of each message. And that is a considerable drawback of the whole solution. To touch each processed item, the UpdateItems method must be called. It is sufficient to call the UpdateItems method and pass all item ids from the last resultset to the method. But this adds a significant burden on the Exchange server and slows overall processing down. Furthermore, if someone accesses the mailbox (either with Outlook or Outlook Web Access) and accidently marks one or more items as read, those items will not be processed by the client application.

Next idea: Find all items that were received after the last time the mailbox was checked. A ProcessItems method that implements this behavior might look like this:

   1: private static void ProcessItems(ExchangeService exchangeService, DateTime lastCheck)
   2: {
   3:     var offset = 0;
   4:     const int pageSize = 100;
   5:  
   6:     FindItemsResults<Item> result;
   7:     do
   8:     {
   9:         var view = new ItemView(pageSize, offset)
  10:                        {
  11:                            SearchFilter = new SearchFilter.IsGreaterThanOrEqualTo(ItemSchema.DateTimeReceived, lastCheck)
  12:                        };
  13:  
  14:         result = exchangeService.FindItems(WellKnownFolderName.Inbox, view);
  15:  
  16:         foreach (var item in result)
  17:         {
  18:             ProcessItem(item);
  19:         }
  20:         offset += pageSize;
  21:     } while (result.MoreAvailable);
  22: }

As with the second example, the difference to the first one is line 11. A search filter has been added that restricts the FindItems call to those items received after a specific time. This method removes the requirement to mark each processed item on the server. But it adds another caveat: A mail that is received during a ProcessItems call will be missed if no additional checks are performed. This can be a little tricky.

Luckily, the Exchange WebServices offer a more suitable solution for the whole requirement: The SyncFolderItems method. This method not only solves the problems mentioned above but also returns deleted items, if such a processing is necessary. A ProcessItems method that uses this API now looks like this:

   1: private static string SyncItems(ExchangeService exchangeService, string syncState)
   2: {
   3:     const int pageSize = 100;
   4:  
   5:     ChangeCollection<ItemChange> changeCollection;
   6:     do
   7:     {
   8:         changeCollection = exchangeService.SyncFolderItems(new FolderId(WellKnownFolderName.Inbox),
   9:                                                            new PropertySet(BasePropertySet.FirstClassProperties), null, pageSize,
  10:                                                            SyncFolderItemsScope.NormalItems, syncState);
  11:  
  12:         foreach (var change in changeCollection)
  13:         {
  14:             if (change.ChangeType == ChangeType.Create)
  15:             {
  16:                 ProcessItem(change.Item);
  17:             }
  18:         }
  19:         syncState = changeCollection.SyncState;
  20:  
  21:     } while (changeCollection.MoreChangesAvailable);
  22:     return syncState;
  23: }

This method only processes newly created items and ignores all other item changes or deletions. The application calling this method needs to store the synchronization state between each call. If an empty sycnState is provided, Exchange will return every item from the mailbox as “Created” item. This makes it possible to process all existing items once and then, with the same logic, every changed item. The only drawback with this method is the fact that the synchronization state can become quite big (in a folder with ~4500 items, the syncstate has a size of approx. 60kb).


Posted by Henning Krause on Sunday, June 7, 2009 12:54 PM, last modified on Monday, November 29, 2010 9:21 PM
Permalink | Post RSSRSS comment feed

InfiniTec.Exchange.Notifications updated to 1.5.0.0

I have just published a new version of the my notification component on CodePlex. The new version has some breaking changes to the previous version but it should be simpler to use. Additionally, I have added a small sample application that shows how to use the component. It’s a small WPF application that allows a user to subscribe to calendars of multiple users. The application uses the new Managed API which made the whole thing a whole lot easier to write. Here is a class diagram of the component:

ClassDiagram

If you have worked with the component before you’ll notice that the Subscription class has lost some of its members. I have removed those methods to avoid confusion on how the component is utilized. The only way to create a new subscription now is to use the SubscriptionCollection class. Here is an example on how to use the component:

   1: using System;
   2: using System.ComponentModel;
   3: using System.IO;
   4: using System.Net;
   5: using InfiniTec.Exchange.Notifications;
   6: using InfiniTec.Threading;
   7:  
   8: namespace NewMailNotificationExample
   9: {
  10:     internal class Program
  11:     {
  12:         private const string StateSaverFilename = "notificationstate.bin";
  13:  
  14:         private static void Main()
  15:         {
  16:             // Ignore any certificate errors
  17:             ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => true;
  18:  
  19:             // Setup the adapter which will be used to call into the Exchange WebService  
  20:             using (var adapter = new ExchangeWebServicesAdapter(new Uri("https://w2k3x64/ews/exchange.asmx"), 
  21:                 new NetworkCredential("administrator", "password", "contoso")))
  22:             using (var listener = new PushNotificationListener())
  23:             {
  24:                 Console.Out.WriteLine("Starting Notification Service...");
  25:                 listener.Start();
  26:  
  27:                 SubscriptionCollection subscriptionCollection;
  28:  
  29:                 if (File.Exists(StateSaverFilename))
  30:                 {
  31:                     using (var stream = File.OpenRead(StateSaverFilename))
  32:                     {
  33:                         subscriptionCollection = SubscriptionCollection.Deserialize(stream, adapter, listener);
  34:                         subscriptionCollection.SubscriptionRestartCompleted +=
  35:                             SubscriptionCollection_OnSubscriptionRestartCompleted;
  36:                         subscriptionCollection.SubscriptionRestartProgressChanged +=
  37:                             SubscriptionCollection_OnSubscriptionRestartProgressChanged;
  38:  
  39:                         subscriptionCollection.RestartAsync();
  40:                     }
  41:                 }
  42:                 else
  43:                 {
  44:                     // Create a new subscription collection to manage all the subscriptions  
  45:                     // Register for a NewMail notification on the inbox of the administrator
  46:                     subscriptionCollection = new SubscriptionCollection(adapter, listener)
  47:                                                  {
  48:                                                      {new[] {new FolderReference(WellKnownFolderId.Inbox)}, EventTypes.All}
  49:                                                  };
  50:                 }
  51:  
  52:                 Console.Out.WriteLine("Creating subscription");
  53:                 foreach (var subscription in subscriptionCollection)
  54:                 {
  55:                     // Write a line to the console for each new mail received  38:  
  56:                     subscription.NewMail += (sender, e) =>
  57:                                             Console.Out.WriteLine(string.Format("{0}: New Mail arrived in your inbox", e.Timestamp));
  58:                     subscription.Start();
  59:                 }
  60:                 Console.Out.WriteLine("Waiting for notifications... Hit [Enter] to quit...");
  61:                 Console.ReadLine();
  62:  
  63:                 Console.Out.WriteLine("Saving the current state of the notification listener...");
  64:                 using (var stream = File.OpenWrite(StateSaverFilename))
  65:                 {
  66:                     subscriptionCollection.Serialize(stream);
  67:                 }
  68:                 Console.Out.WriteLine("State saved to {0}", Path.GetFullPath(StateSaverFilename));
  69:             }
  70:         }
  71:  
  72:  
  73:         private static void SubscriptionCollection_OnSubscriptionRestartProgressChanged(object sender, ProgressChangedEventArgs args)
  74:         {
  75:             Console.Out.WriteLine("Subscription restart {0}% complete.", args.ProgressPercentage);
  76:         }
  77:  
  78:         private static void SubscriptionCollection_OnSubscriptionRestartCompleted(object sender, AsyncCompletedEventArgs<SubscriptionRestartErrorSummary> args)
  79:         {
  80:             Console.Out.WriteLine("Subscription restart is complete. {0} subscriptions could not be restarted.", args.Result.Errors.Count);
  81:         }
  82:     }
  83: }

Posted by Henning Krause on Saturday, May 23, 2009 12:38 PM, last modified on Saturday, May 23, 2009 12:40 PM
Permalink | Post RSSRSS comment feed