InfiniTec - Henning Krauses Blog

Don't adjust your mind - it's reality that is malfunctioning

ExchangeWebServices / WebDAV and untrusted server certificates

Exchange 2007 has requires SSL for its WebServices, and event for Exchange 2003 some administrators have enabled this requirement on the IIS. If you are dealing with a self-signed certificate on the server and want to use .NET, you will stumble across this error message:

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

or

The remote certificate is invalid according to the validation procedure.

By default, .NET checks whether SSL certificates are signed by a certificate from the Trusted Root Certificate store. To override this behavior, use the System.Net.ServicePointManager.ServerCertificateValidationCallback property:

   1: ServicePointManager.ServerCertificateValidationCallback = RemoteCertificateValidationCallback;

The callback looks like this:

   1: private static bool RemoteCertificateValidationCallback(object sender, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors)
   2: {
   3:     return true;
   4: }

This will accept all certificates, regardless of why they are invalid. One option here is to display a warning similar to the Internet Explorer one.

Using C# 3.0, this can even be written with less code:

   1: ServicePointManager.ServerCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) => true;

Posted by Henning Krause on Wednesday, November 26, 2008 5:57 PM, last modified on Wednesday, November 26, 2008 5:57 PM
Permalink | Post RSSRSS comment feed

Retrieving the size of a mailbox via WebDAV

A common question is how to retrieve the size of a given mailbox or the size of a folder structure via WebDAV. There is an example for this task on MSDN (Getting the Size of a Mailbox (WebDAV)). But that example is rather difficult to understand because the WebDAV requests are built using simple string concatenation, which is rather ugly. It is far better to create those requests using the XmlReader and XmlWriter classes (I’ve written about it in the article PROPPATCH requests using XmlReader and XmlWriter, Part 1).

The procedure to retrieve the cumulative folder size is this:

  • Issue a SEARCH request on the root folder of interest (for example the mailbox) and retrieve the size of all messages in that folder
  • Repeat this step for each sub folder.

Here is a C# program which demonstrates this:

   1: using System;
   2: using System.IO;
   3: using System.Net;
   4: using System.Text;
   5: using System.Xml;
   6: using System.Xml.XPath;
   7:  
   8: namespace InfiniTec.Exchange.Examples
   9: {
  10:     internal class Program
  11:     {
  12:         private const string DavNamespace = "DAV:";
  13:         private const string ProptagNamespace = "http://schemas.microsoft.com/mapi/proptag/";
  14:  
  15:  
  16:         public static byte[] GetFolderSizeRequest(string url)
  17:         {
  18:             var settings = new XmlWriterSettings {Encoding = Encoding.UTF8};
  19:  
  20:             using (var stream = new MemoryStream())
  21:             using (XmlWriter writer = XmlWriter.Create(stream, settings))
  22:             {
  23:                 writer.WriteStartElement("searchrequest", DavNamespace);
  24:                 var searchRequest = new StringBuilder();
  25:  
  26:                 searchRequest.AppendFormat("SELECT \"http://schemas.microsoft.com/mapi/proptag/x0e080014\", \"DAV:hassubs\" FROM SCOPE ('HIERARCHICAL TRAVERSAL OF \"{0}\"')", url);
  27:  
  28:                 writer.WriteElementString("sql", searchRequest.ToString());
  29:                 writer.WriteEndElement();
  30:                 writer.WriteEndDocument();
  31:  
  32:                 writer.Flush();
  33:                 return stream.ToArray();
  34:             }
  35:         }
  36:     
  37:         private static long GetMailboxSize(string url, ICredentials credentials)
  38:         {
  39:             XmlReader reader;
  40:  
  41:             byte[] buffer = GetFolderSizeRequest(url);
  42:  
  43:             var request = (HttpWebRequest) WebRequest.Create(url);
  44:             request.Method = "SEARCH";
  45:             request.ContentType = "text/xml";
  46:             request.Credentials = credentials;
  47:             request.Headers.Add("Translate", "f");
  48:             request.Headers.Add("Depth", "1");
  49:             
  50:             using (Stream stream = request.GetRequestStream())
  51:             {
  52:                 stream.Write(buffer, 0, buffer.Length);
  53:             }
  54:  
  55:             using (WebResponse response = request.GetResponse())
  56:             {
  57:                 string content = new StreamReader(response.GetResponseStream()).ReadToEnd();
  58:  
  59:                 reader = XmlReader.Create(new StringReader(content));
  60:  
  61:                 var nsmgr = new XmlNamespaceManager(reader.NameTable);
  62:                 nsmgr.AddNamespace("dav", DavNamespace);
  63:                 nsmgr.AddNamespace("e", ProptagNamespace);
  64:  
  65:                 var doc = new XPathDocument(reader);
  66:                 long result = 0;
  67:                 
  68:                 foreach (XPathNavigator element in doc.CreateNavigator().Select("//dav:response[dav:propstat/dav:status = 'HTTP/1.1 200 OK']", nsmgr))
  69:                 {
  70:                     var size = element.SelectSingleNode("dav:propstat/dav:prop/e:x0e080014", nsmgr).ValueAsLong;
  71:                     string folderUrl = element.SelectSingleNode("dav:href", nsmgr).Value;
  72:  
  73:                     Console.WriteLine("Folder size of {0}: {1:0.00} MB", folderUrl, (double)size / 1048576);
  74:  
  75:                     result += size;
  76:                     bool hasSubs = element.SelectSingleNode("dav:propstat/dav:prop/dav:hassubs", nsmgr).ValueAsBoolean;
  77:  
  78:                     if (hasSubs)
  79:                     {    
  80:                         result += GetMailboxSize(folderUrl, credentials);
  81:                     }
  82:                 }
  83:  
  84:                 return result;
  85:             }
  86:         }
  87:  
  88:         private static void Main()
  89:         {
  90:             long size = GetMailboxSize("http://w2k3srv.contoso.local/exchange/administrator/", new NetworkCredential("administrator", "password"));
  91:  
  92:             Console.Out.WriteLine("Mailboxsize = {0:0.00} MB",(double) size/1048576);
  93:  
  94:             Console.Out.WriteLine("Finished");
  95:             Console.ReadLine();
  96:         }
  97:     }
  98: }

Posted by Henning Krause on Wednesday, June 11, 2008 7:19 PM, last modified on Wednesday, June 11, 2008 7:20 PM
Permalink | Post RSSRSS comment feed

Exchange Developer Roadmap - WebDAV and StoreEvents gone

Microsoft just released a post about the technologies being removed from the next version of Exchange. I'm ok with WebDAV being removed, given that EWS will be extended to support access to hidden messages and providing strong-typed access to mailbox settings. I can also live with the fact that CdoEx and ExOleDB will be remove. But store events are another thing.

Sure, they are not really easy to implement and the whole access using ADO is a terrible mess. CdoEx makes it not exactly better as there is no support for tasks.

The proposed replacement for store event sinks are transport agents and EWS notification. While transport agents are fine when dealing with email messages, they are useless with respect to appointments, contacts and tasks. This leaves the EWS notifications as the sole option here. Why is this bad?

  • Synchronous execution (access to old data as well): Synchronous event sinks (OnSyncSave, OnSyncDelete) provide direct access to the item being modified (the record is directly available). And during the Begin phase, the event sink can even open the old record and execute actions based on how fields where changed. This feature will be lost completely with EWS notifications.
  • Register once, even for new users (storewide eventsinks): Store wide event sinks are registered once on a store and it will be triggered for every mailbox in the store - even new users. EWS notifications must be registered for each mailbox and the application receiving the mails is required to monitor Exchange for new mailboxes.
  • Access to all properties, even during deletion of an object: With a synchronous OnSyncDelete event sink, all properties of an item can be examined before it gets deleted. With notifications I merely get a notification that and item with a specific ItemId has been deleted. The client application is responsible to track the deleted item - whether is was soft-deleted or moved to the recycle bin. The properties can then be accessed from there. But if the item was hard-deleted (in case of the dumpster being disabled on public folders, for example), one is out of luck. The real problem is this: The ItemId is based on the email address of the mailbox owner as well as the MAPI EntryId of the item (see Exchange 2007 SP1 Item ids). Both, the mailbox address as well as the MAPI entry id are not guaranteed to be stable (Since Exchange 2003 SP2, Exchange recreates meeting items under certain circumstances: CDOEX Calendaring Differences Between Exchange 2003 and Exchange 2003 SP2). This has the effect that the ItemId is not suitable as a long term identifier which should be stored in a database. In scenarios where Exchange data is being replicated to a relational databases, this can become a problem.
  • Order of execution: Synchronous event sinks are called in the order the items were changed. With asynchronous notifications, this cannot be guaranteed.
    To clarify this: The order in which notifications are sent to the client application cannot guaranteed to reflect the order in which the changes were made. To work around this issue, the each notification carries a watermark and a previous watermark. This way a client application can restore the correct order. But with synchronous store events, this comes for free.
  • Silent updates: Due to the possibility to modify an item while it's being changed, the synchronous store events allow some sort of silent update. This works like this:
    1. Along with all the other modifications, a client program sets a special user defined field to an arbitrary value.
    2. The event sink checks this field during the Begin phase, and if it's set it won't process the item. Instead it just removes the property from the element.
  • Modify changes made by a user / Block modifications: A synchronous event sink can modify changes while they are being saved. It can even abort the change, thus preventing a user from deleting an item for example.
    Note: Canceling updates/deletions can break synchronization with ActiveSync or Outlook. So don't do this!
  • Performance: In most scenarios, the WebService receiving the push notifications will not reside on the Exchange server (I know a bunch of administrators who even hesitate to install the .NET Framework on the Exchange server let alone software which does not come from MS*). In this case, the push notifications work like this:
    image
    This makes three network hops to get the properties of a changed item. What makes things worse is the fact that the notifications are send for each modified element. With store event sinks, one could specify a filter so that the sink was only triggered for a subset of elements.

So, while store event sinks are certainly no unconfined pleasure to use, they are far more suited for certain scenarios. I would rather see them being replaced by a managed solution (like transport agents) than the stuff that is coming now.

By the way, the OnTimer, OnMdbShutdown, OnMdbStartup event sinks are being removed without any replacement.

Enough ranting for one day...

* Now, one could argue that a store event sink has far more impact on the Exchange server and those administrators would also hesitate to install them... while that is technically true, there is no other option yet, so they have to swallow that pill.


Posted by Henning Krause on Friday, May 23, 2008 4:41 PM, last modified on Tuesday, July 19, 2011 8:58 AM
Permalink | Post RSSRSS comment feed

A search on a calendar folder can not span more than 732 days

More Information

A SEARCH on an Exchange calendar folder with a constraint of a begin and an end date is called an expansion query. This means, that all recurring appointments are expanded, and each instance is returned in the query. (See Searching Calendar Folders with WebDAV on MSDN for more information. The article specifically targets the WebDAV  protocol, but the issue explained here applies to the ExOleDB provider as well).

An example of such a query is displayed below.

If you run this query and use values for the start and end properties where the difference between start and end is greater than 732 days, you get the following error:

Error while processing WHERE clause in the SQL statement.

Example

An example of such an expansion query might look like this:

    1 SEARCH /exchange/~username/calendar/ HTTP/1.1

    2 Host: www.example.com

    3 Content-Type: text/xml

    4 

    5 <?xmlversion="1.0"?>

    6 <g:searchrequestxmlns:g="DAV:">

    7   <g:sql>

    8     Select "urn:schemas:calendar:location", "urn:schemas:httpmail:subject",

    9     "urn:schemas:calendar:dtstart", "urn:schemas:calendar:dtend",

   10     "urn:schemas:calendar:busystatus", "urn:schemas:calendar:instancetype"

   11     FROM Scope('SHALLOW TRAVERSAL OF ""') WHERE

   12     "DAV:contentclass" = 'urn:content-classes:appointment'

   13     AND "urn:schemas:calendar:dtstart" &gt; CAST("2005-01-01T00:00:00Z" AS 'dateTime.tz')

   14     AND "urn:schemas:calendar:dtend" &lt; CAST("2006-01-01T00:00:00Z" AS 'dateTime.tz')

   15     ORDER BY "urn:schemas:calendar:dtstart" ASC

   16   </g:sql>

   17 </g:searchrequest>

Workaround

To work around this problem, do your search in multiple steps, with a time span of two years or less.

Status

This is a known limitation in the Exchange Server: Expansion queries are limited to a time span of two leap years.

Posted by Henning Krause on Thursday, April 13, 2006 12:00 AM, last modified on Thursday, April 13, 2006 12:00 PM
Permalink | Post RSSRSS comment feed

Workaround: A status of 409 - conflict is received when adding an attachment to a mail via WebDAV

More Information

The WebDAV protocol can be used to send emails. This can be done by creating the email in a mailbox folder (i.e. the Drafts folder) and moving this item to the ##DavMailSubmissionURI## folder. This is described in this MSDN article: Sending a Message (WebDAV).

Attachments can be either added to this mail using the Outlook Web Access method, or by issuing a PUT, using the address of the mail as folder: If the address of the item is

http://myserver/exchange/jdoe/drafts/new_mail.eml

the url for the PUT command would be something like http://myserver/exchange/jdoe/drafts/new_mail.eml/attachment.zip.

If you created the mail item via the PUT method, instead of a PROPPATCH, you might get a 409  - conflict error when adding attachments to the item.

Solution

A workaround seems to be to issue a PROPPATCH on the mail item itself. The PROPPATCH does not necessarily have to change the item. It's more like a touch command. After this PROPPATCH command, the PUT command for the attachment will be successful.

Status

The status of this problem is unknown.

Posted by Henning Krause on Thursday, April 13, 2006 12:00 AM, last modified on Thursday, April 13, 2006 12:00 PM
Permalink | Post RSSRSS comment feed

Iterating through all mailboxes in an Exchange 2000/2003 organization

Description [Updated]

Updates

  • 2008-07-23: Corrected intra-site links.

One question that came up lately in the newsgroups is: How can I find messages containing certain keywords in the mailboxes of all users in my organization?

This scenario is not directly supported by Exchange. All one can do is to search each mailbox individually.

You must follow these steps to do the search over all mailboxes:

  1. Enumerate the users which have a mailbox. Essentially, these are the users appearing on the global address list. See How to get the Global Address List programatically for more information on how to do this.
  2. Build the mailbox url which can be used to access the mailbox via WebDAV or ExOleDB. See Get the WebDAV url for an Exchange 2000/2003 mailbox on how to do this. If you are using the ExOleDB provider or want to use the administrative virtual root instead, see the remarks section for more information.
  3. Once you have the url for the mailbox you can start accessing it. If you must support different languages, see Getting Well-Known Mailbox Folder URLs on MSDN to get the url of the default folders.
  4. If you are using WebDAV and have Form-based-authentication enabled on your server, you must do a manual logon to the mailbox. See Access the Exchange store via WebDAV with Form-Based-Authentication turned on.

Remarks

Permissions

Depending on how you want to access the mailboxes, you need different permissions:

  • If you are using the normal urls (e.g. http://myserver/exchange/username), you need access permissions to all mailboxes on the MAPI level. See HOWTO: Grant access to all mailboxes on a mailbox store to a special account on how to do this. If you have more than one mailbox store, you should grant the necessary permissions on each mailbox store. To simplify this process, you could also grant the "Send as" and "Receive as" permission on the Administrative Groups container via ADSIEdit.msc instead of each mailbox store.
  • You can also use the administrative virtual root. This method is used by the Exchange Systems manager, and it is available via WebDAV and ExOleDB. The normal MAPI permissions are completely ignored when using this method, but an administrative account is required to use this method (See Working with Store Permissions in Microsoft Exchange 2000 and 2003 on Technet for more information on this topic).

Mailbox urls

  • If you are using WebDAV to access the store, you can simply build the mailbox url based on the article Get the WebDAV url for an Exchange 2000/2003 mailbox. To use the administrative root instead, modify the url from http://myserver/exchange/mailboxname to http://myserver/exadmin/admin/<dsndomainname>/mbx/<mailboxname>. You must replace the <dnsdomainname> with the primary smtp domain name of your organization.
  • If you are using ExOleDB, you must modify the address from http://myserver/exchange/mailboxname to file://./backofficestorage/<dnsdomainname>/mbx/<mailboxname>. To use the administrative virtual root, change this url to file://./backofficestorage/admin/<dnsdomainname>/mbx/<mailboxname>.

Posted by Henning Krause on Thursday, April 13, 2006 12:00 AM, last modified on Wednesday, July 23, 2008 10:59 PM
Permalink | Post RSSRSS comment feed

WebDAVLayer 1.0.1735.12520

Posted under .NET Tools | Comments (2)

More Information

The database that is used by Exchange 2000/2003 is called WebStorageSystem. Each element within this store is accessible via an URL using the WebDAV protocol. This protocol extends the HTTP protocol by exposing several new verbs like PROPFIND, PROPPATCH or SEARCH to name a few. Additionally, the Exchange implementation of the WebDAV protocol has some proprietary commands for batch operations: BMOVE and BCOPY. The parameters of these commands are expressed within the body of the HTTP/WebDAV request as XML.

Features

  • Encapsulation of most of the WebDAV commands.
  • Easy access to folders, elements and properties within the store
  • Support for various authentication schemas: Basic, Digest, NTLM, Kerberos and even the Form-based authentication of Exchange 2003
Unfortunately, there is currently no documentation available. But on the other hand: You can use it free of charge!

Downloads


Posted by Henning Krause on Monday, February 7, 2005 9:04 PM, last modified on Monday, February 7, 2005 9:25 PM
Permalink | Post RSSRSS comment feed

Get the WebDAV url for an Exchange 2000/2003 mailbox

More Information

For Exchange 2000 and Exchange 2003 prior Service Pack 1, the building of an URL suitable for WebDAV requests against a users mailbox is a rather complicated thing. A solution for this is descrbied under the section Solution.

From Exchange 2003 SP 1 onwards, you can just use his SMTP address to get access to his mailbox. For example, to access the mailbox of John Doe (jdoe@contoso.com) you would use the url

http://<myserver>/exchange/jdoe@contoso.com

Solution

To build the webaddress, you must do the following:

  1. If you only have a domain\username, not the distinguished name of the user, you must first get the latter one. See this aricle how to retrieve the name.
  2. Get the directory entry with the distinguished name of the user
  3. From that object, retrieve the property homemdb. This property contains the distinguished name of the mailbox store.
  4. Retrieve the directory entry of the mailbox store.
  5. From this object, retrieve the property named msExchOwningServer. This is the distinguished name of the Exchange server that hosts the mailbox.
  6. Get the root of the http virtual server on the exchange server. The distinguished name is CN=http, CN=Protocols, <distinguished name of the exchange server>.
  7. Search for all http virtual server (LDAP filter: (objectClass=protocolCfgHttpServer), Scope: OneLevel, retrieve these properties: msExchServerBindings, msExchDefaultDomain).
  8. From this list, retrieve the default SMTP server. It's the one without the attribute msExchDefaultDomain.
  9. Get the default SMTP domain of the organization:
    1. Open the node CN=Default Policy, CN=Recipient Policies, CN=<Name of your organization>, CN=Microsoft Exchange, CN=Services,<Distinguished name of the configuration naming context>.
    2. Retrieve the property gatewayProxy. This is a multi-valued property that contains the default addresses of the organization.
    3. Check each of the entries in that list. If it starts with SMTP:, you have found it.
  10. Get the property proxyAddresses from the directory object of the user. This is a multi-valued properties that contains all addresses that are assigned to the user. Each entry has the following consists of a protocol identifier and an address entry. For example:
    SMTP:john.doe@example.com
    If the protocol identifier is all uppercase, it is the default entry for that protocol.
  11. Iterate through the list of assigned addresses and search for the SMTP address which domain part matches with the default SMTP retrieved above.
  12. Select the correct HTTP Server:
    • If you found an email address corresponding to the default SMTP policy, extract the alias from the found address. For the above example the alias would be john.doe.  The correct HTTP server to use is the default HTTP Server
    • If there is no smtp address matching the default policy, iterate through the list of proxy addresses again, and do the following, if it is an SMTP address:
      1. Extract the alias and the domain from the address.

      2. Iterate through the list of virtual HTTP server and check the property msExchDefaultDomain of each entry. If it matches the domain of the users address domain, you have found the correct http server.

  13. Get the property msExchServerBindings from the selected HTTP server node. This entry has the following structure:
    IPAddress:Port:Servername
    The first and the last part of the string can be empty, like in this example:
    :80:
  14. Depending on the content of this field, you can now build the desired url:
    • If the servername of the msExchServerBindings property is not empty, build the OWA url like this:
      http://<Servername>:<Port>/exchange/<Alias>
    • If the servername is empty, but an ipaddress is present, build the OWA url like this:
      http://<IPAddress>:<Port>/exchange/<Alias>
    • If both, the servername and the ipaddress fields are empty, get the ipaddress for the exchange server:
      1. From the node of the exchange server, retrieve the property networkaddress.
      2. Iterate through this property and retrieve the value which starts with ncacn_ip_tcp:.
      3. Now, build the url with the above scheme, using the just found ip address.

Tags: , , , , ,

Technorati: , , , , ,

Posted by Henning Krause on Friday, December 31, 2004 12:00 AM, last modified on Tuesday, July 26, 2011 9:57 PM
Permalink | Post RSSRSS comment feed