Sending Emails to Content Lists

From Sense/Net Wiki
Jump to: navigation, search
  •  
  •  
  •  
  •  
  • 100%
  • 6.0.8
  • Enterprise
  • Community
  • Planned

Overview

Sending emails to content lists
Sense/Net 6.0 lists and libraries can be configured to receive email notifications and retrieve emails from a Microsoft Exchange Server (2007 or 2010) or (from Sense/Net 6.3) any other email server that enables connecting through POP3 protocol. An email address can be bound to lists and libraries, thus emails can be sent directly to them (e.g. to document libraries and memo lists), with the attachments being saved as files. Optionally the emails themselves can be saved in different formats.

Details

In this article you can learn how to configure content lists and libraries to be able to receive attachments sent by email.

This article focuses mainly on Exchange integration, however from Sense/Net 6.3 it is possible to use the well-known POP3 protocol for receiving emails through any email server that is capable of publishing emails through POP3 (e.g. GMail, Hotmail).

Features

The feature includes the following basic functionality:

  • lists can be configured to receive emails from a defined - and existing - mailbox;
  • processing of incoming emails can be configured on a simple UI available in List Settings;
  • processing of incoming emails can be fine adjusted by editing a built-in Workflow or creating a custom workflow;
  • built-in Workflow activities are available for custom Workflows to connect to a mail server and receive mails;
  • simple API can be used by developers for retrieving mails;
  • built-in in framework and services are available for Exchange push/pull-notifications and subscription handling.

Connecting to an Exchange server

Sense/Net can be connected to a Microsoft Exchange server in two modes:

  • push notifications: an Exchange server can send notifications about incoming emails to a web service defined in Sense/Net CMS. A custom workflow can be defined to process the incoming emails.
  • polling unread mails: a custom workflow can be defined to retrieve unread emails from an Exchange mailbox with a given time interval.

The difference between the two methods is summerized in the following table:

Difference between push notifications and email polling
Push notifications Email polling

Ico-ok.gif Nearly instantaneous notifications
Ico-ok.gif No wasted traffic
Ico-na.gif Listener must be addressable by the Exchange server
Ico-na.gif Difficult to configure and troubleshoot

Ico-ok.gif Client does not need to be addressable (can be behind a proxy or firewall)
Ico-ok.gif Easy to configure
Ico-na.gif Receive notifications as frequently as client polls
Ico-na.gif Wasted traffic
Ico-na.gif Fine tuning required to get optimal polling interval

Sense/Net CMS supports both notification types when an email address is bound to a Content List in the Sense/Net Content Repository. The default method is pull notifications, but this can easily be changed by changing the workflow that processes incoming emails.

Configuring pull notifications

The mailbox can be defined by giving the email address in the Incoming Email Settings of a Content List. A workflow is executed after the mailbox address is given that runs constantly and connects to the Exchange Server periodically to retrieve unread mails. This workflow then processes the incoming emails and creates new Content in the Content List according to the Incoming Email Settings.

To configure the portal to use email polling, you simply have to give your lists and email address (see below), and optionally set the ExchangeAddress element in the settings content (prior to version 6.3: in the Web.config) to use a specific mail server address instead of using autodiscover.

Make sure the user running the app pool of the portal site has necessary permissions to access the mailbox.

Configuring push notifications

To enable push notifications the CMS needs to subscribe to a mailbox. This mailbox can be defined by giving the email address in the Incoming Email Settings of a Content List. Then, whenever a new email is sent to the defined mailbox in Exchange, the Exchange server sends a notification to the web service - whose url address was given with the subscription automatically - with info about the new email. The web service runs in the portal context of the Sense/Net CMS and creates a content under the list in the IncomingEmails SystemFolder persisting the ID of the incoming email. The workflow that runs in the background on lists with a specific mailbox given processes the incoming emails and creates new Content in the Content List according to the Incoming Email Settings.

To configure the portal to use push notifications, from version 6.3, you will have to set the MailProcessingMode to ExchangePush, PushNotificationServicePath and optionally the ExchangeAddress elements in the settings content, and also set the PushNotification parameter of the Mail Processor Workflow's ExchangePoller activity to True to use push notifications. Then you can go on an set the email addresses of your lists.

Prior to version 6.3: you have to set the SubscriptionServiceEnabled, SubscribeToPushNotifications, PushNotificationServicePath and optionally the ExchangeAddress elements in the Web.config, and also set the PushNotification parameter of the Mail Processor Workflow's ExchangePoller activity to True to use push notifications.

Make sure the user running the app pool of the portal site has necessary permissions to access the mailbox.

Settings

To enable Exchange Integration for either push notification or email polling, or using POP3 protocol, you need to modify the settings in the following Setting file:

  • /Root/System/Settings/MailProcessor.settings
    <MailProcessor>
      <!-- Global setting for the way of email polling. Possible values: ExchangePush, ExchangePull, POP3 -->
      <add key="MailProcessingMode" value="ExchangePull" />
      <!-- Exchange polls the CMS with the given interval if at least one push notification subscription is active. -->
      <add key="StatusPollingIntervalInMinutes" value="120" />
      <!-- The address of the web service handling push notifications. {0} will be replaced with the Document Library's path. -->
      <!--<add key="PushNotificationServicePath" value="http://myhost.com{0}?action=ExchangeService.asmx" />-->
      <!-- Address of the exchange server EWS web service. Leave it uncommented for using autodiscover by email address. -->
      <!--<add key="ExchangeAddress" value="https://mysmtp.com/EWS/Exchange.asmx" />-->
      <!--POP3 settings: server, port and SSL-->
      <add key="POP3Server" value="pop3.example.com" />
      <add key="POP3Password" value="" />
      <add key="POP3Port" value="995" />
      <add key="POP3SSL" value="true" />
    </MailProcessor>

The following settings are available:

  • MailProcessingMode: controls how the system gets the emails sent to lists. Possible values are:
    • ExchangePull: the system polls the Exchange server for new mails periodically. You can modify the time period in the customizable workflow as it is described later in this article.
    • ExchangePush: a logic will be executed on every system startup that re-subscribes to all mailboxes for every Content List that is configured to receive mails. This can be useful when the portal has been unavailable for the Exchange server and therefore the server deleted the subscription. Also a subscription to the mailbox events in Exchange Server will occur after setting the email address for a List.
    • POP3: the system connects to the mail server using the POP3 protocol. This is a simple method for getting emails that needs a mailbox on the Incoming Email Settings page and the password in the settings file. All mails that were sent to that mailbox will be processed and deleted.
  • StatusPollingIntervalInMinutes: used with push notifications. For active subscriptions the Exchange server sends a status message to the webservice, to check if the receiver is still available. These messages are ignored - but otherwise served to indicate available service - by Sense/Net. Set it to a higher value if you want to receive these status checks less frequently. The polling interval will be in effect for new subscriptions (either manual or automatic subscriptions referred at the SubscriptionServiceEnabled setting).
  • PushNotificationServicePath: used with push notifications. The path of the web service handling push notifications. A built-in service is placed at /Root/(apps)/ContentList/ExchangeService.asmx, this will be addressed by the exchange server. The path here therefore should include the host of Sense/Net CMS, and the rest will be handled by the portal (the '{0}' pattern will be replaced with the path of the appropriate Content List, and the ?action=ExchangeService.asmx will instruct the portal to run the webservice defined for Content Lists among the global applications).
  • ExchangeAddress: the address of the exchange EWS web services. It is usually in the given format, only the host of the smtp server needs to be adjusted. If this key is commented out the autodiscover services of exchange will be used to resolve the address of the appropriate Exchange server.
  • POP3Server: POP address of the email server, e.g. pop.gmail.com.
  • POP3 password: if POP3 mode is configured, you have to provide the password for all the mailboxes here to let the system check for emails periodically.
  • POP3Port: port number, default is 995.
  • POP3SSL: True if SSL connection is needed.

Configuration before Sense/Net 6.3

In previous versions the configuration could be found in the sensenet/exchangeService section of the Web.config and the mail processing mode could be set by two values:

    <exchangeService>
      <!-- If true the portal subscribes to Exchange push notifications on repository startup for every doclib with email address. Helpful for automatic re-subscribing if Exchange deleted a subscription. -->
      <add key="SubscriptionServiceEnabled" value="false" />
      <!-- If true the portal subscribes to Exchange push notifications when List Email settings is changed/saved. Switch this off if you want to use pull notifications -->
      <add key="SubscribeToPushNotifications" value="false" />
    </exchangeService>
  • SubscriptionServiceEnabled: used with push notifications. When set to true, a logic will be executed on every system startup that re-subscribes to all mailboxes for every Content List that is configured to receive mails. This can be useful when the portal has been unavailable for the Exchange server and therefore the server deleted the subscription.
  • SubscribeToPushNotifications: if set to true, a subscription to the mailbox events in Exchange Server will occur after setting the email address for a List. Set it to true when using push notifications, and leave it false when using pull.

All the other configuration values are also in this section and look like the same as described above.

List Configuration - Incoming Email Settings

To bind an email address to a Content List, you need to access the Incoming Email Settings surface through the Settings menu of the List:

1: Accessing the Settings menu
2: Accessing the Incoming Email Settings
3: Incoming Email Settings of a Content List

On this interface the following settings are available:

  • Email address of Content List: the email address of the existing mailbox that will act as the email address of this List. If the corresponding mailbox does not exist, create it beforehand.
  • Group attachments: the way attachments of incoming emails should be grouped when put into the Content List. The following options are available by default:
    • Save all attachments as children of separate Email content: a content of Email Content Type will be created for each email, with the subject of the email as its name, attachments will be put under this content. The Email content will save the Sender, the Subject, the Body and the Sent date of the incoming email in its Fields.
    • Save all attachments in root: no container will be created and all attachments of every email will be put into the Content List directly.
    • Save all attachments in folders grouped by subject: a Folder with the name of the subject of the incoming email will be created, and attachments will be put into the corresponding Folder. Attachments of emails with the same subject will be put into the same Folder.
    • Save all attachments in folders grouped by sender: a Folder with the name of the sender of the incoming email will be created, and attachments will be put into the corresponding Folder. Attachments of emails from the same sender will be put into the same Folder.
  • Inbox folder: from version 6.3: a relative path of the folder in the list where you want to place emails and attachments. By default it is empty; you can set it to a relative path like emails or emails/subfolder. If the path does not exist, it will be created automatically by the workflow, using the defined container type (default: Folder) defined in the workflow.
  • Save original email: if this is set to true, a file of extension .eml will be created containing the incoming email in .eml format, and will be put into the same folder to which the attachments of the email go, according to the grouping selected above.
  • Overwrite files with same name: if this is checked, existing files with the same name will be overwritten. If left unchecked, filenames with incremental suffix will be used when a file with the same name already exists in the target folder or Content List.
  • Accept emails only from users in local groups: from version 6.3: if you check this, only users that are members of local groups will be able to send emails to the list. The Sender (or From) email address will be checked for this.
  • Incoming email workflow: by default the system launches the MailProcessorWorkflow workflow, that will create the necessary content for the incoming email and the attachments according to the above given configuration. This is the only built-in workflow for this job, however a custom workflow can be created for this, inheriting from the MailProcessorWorkflow, and then can be selected here. Also, the MailProcessorWorkflow can be configured for both push / pull notifications - by default it uses pull notifications.

Make sure that the referred mailbox exists in Exchange Server and the user running the app pool of Sense/Net CMS has the necessary permissions to access it. In case of push notifications if the subscription cannot be created an error message will appear at the top of the view.

Make sure that the selected Content Types can be created in the Content List, so Folders, Files or Email Content Types are listed in the Content List's Allowed Content Type field!

Mail Processor Workflow

Sense/Net CMS comes with a built-in workflow that is executed when setting a mailbox for a list and runs continously in the background. This can be exchanged to a user-defined workflow any time, in the Incoming Email Settings for a Content List, or if a global adjustment is necessary, this built-in workflow can also be modified. Read more about the workflow, how it works, and how to customize it:

Connecting through POP3

POP3 protocol is a widely used protocol for accessing mailboxes. It is easy to configure: you only have to set the POP3 address and port number published by the mail server, and the password in the MailProcessor Settings as described above. Than you can provide the necessary mailbox on the Inoming email settings page of the content list.

As the protocol does not know the concept of read emails, processed emails are deleted immediately. If you want to have a copy of the mail, you can choose the Save original email option on the settings page.

Managing push subscriptions

When subscribing to notifications for a mailbox the Microsoft Exchange Server will start sending status messages periodically with the configured time interval (see #web.config Configuration. It is possible to unsubscribe from a mailbox event (delete the subscription) by answering to a status message with the appropriate message. A subscription is automatically deleted if the Exchange server cannot access the webservice running on Sense/Net CMS for a definite time interval. Therefore it is quite cumbersome to determine if a subscription is still active or not. Sense/Net provides you the following means to manage subscriptions:

  • Deleting subscriptions: to delete an existing subscription, simply erase the contents of the List Email Field in the Incoming Email Settings and save the settings. This way the next status message (or new mail notification) sent by the Exchange server will not be processed and a return message will be sent to the Exchange server to delete the subscription.
  • Reactivating subscriptions: if the portal has been down for a given period of time it might happen that the Exchange server has deleted the subscription and will not send any notifications about incoming emails. If this happens there are two options to renew the subscriptions:
    • automatically: configure the subscription service in the #web.config Configuration so that when the portal is restarted it will automatically re-subscribe to every defined mailbox in the system. Re-subscription on active subscriptions does not cause problems since in this case the previous subscriptions will automatically be deleted with the next status message (or new mail message).
    • manually: go to the Incoming Email Settings of the Content List, and erase the contents of the List Email email Field. Save the settings and then once again open the settings and fill in the List Email. Upon saving the settings once again a new subscription will take place to the events of the given Exchange mailbox.

Notification on unprocessed mails

Receiving unread emails that arrived in the mailbox while the portal or the connection between the portal and the Exchange server was down is done by using watermark indications. Every time a new email is processed in the watermark provided by the Exchange server is saved on the IncomingEmails SystemFolder under the list. This watermark identifies the last processed email. When renewing subscriptions the watermark of the lastly created workflow is sent to the Exchange server, and according to this information the Exchange server will immediately send notifications of emails that arrived after the email that corresponds to the specified watermark - in other words Sense/Net is notified of all the yet unprocessed emails.

Please note, that the last watermark is persisted on the IncomingEmails SystemFolder (in its Description field) under the list, therefore if you delete this folder you will not get notifications of unprocessed mails upon the very next subscription.


Troubleshooting

To troubleshoot any problems emerging with Exchange Integration, read the following article:

Example/Tutorials

Example settings for push notifications

The following excerpt from the settings content configures the Exchange Integration so that it uses the automatic subscription service, receives status messages in every 60 minutes (besides immediate notifications of new mails), exposes the Exchange web service for incoming notifications on host smallserver.sn.hu and uses the smtp.sn.hu Exchange server:

    <MailProcessor>
      <add key="MailProcessingMode" value="ExchangePush" />
      <!-- Exchange polls the CMS with the given interval if at least one push notification subscription is active. -->
      <add key="StatusPollingIntervalInMinutes" value="120" />
      <!-- The address of the web service handling push notifications. {0} will be replaced with the Document Library's path. -->
      <add key="PushNotificationServicePath" value="http://smallserver.sn.hu{0}?action=ExchangeService.asmx" />
      <!-- Address of the exchange server EWS web service. Leave it uncommented for using autodiscover by email address. -->
      <add key="ExchangeAddress" value="https://smtp.sn.hu/EWS/Exchange.asmx" />
    </MailProcessor>

Related links

References