Email integration

Redmine email integration can be considered to consist of two components: email delivery (notifications) and email retrieval.

Email delivery

Most likely, if you open the Email notifications tab on a recently installed Redmine, you will get the following message:

Email delivery

This message clearly states that you can't fix this issue through the web interface. First, we need to modify the mentioned configuration file and then we can get back here. So, it's time to open the console.

The configuration.yml file

In the config subdirectory of the Redmine root directory, you will find the configuration.yml.example file. Copy (or just rename) it to configuration.yml:

$ cp configuration.yml.example configuration.yml

Now, open this file in your favorite console editor.

As you can see, it is divided into three blocks: default, production, and development. The production and development blocks are for environment-specific configuration and the default block combines the configuration options of all environments. As we are not going to have any specific configurations for the environments, we will use the default block.

The email configuration section starts under the email_delivery keyword. In this section you will find commented sample configurations, one of which you can use for your Redmine installation. The only option that is used in all the samples is delivery_method, which determines the type of delivery.

The delivery_method option accepts the following values: :sendmail, :smtp, :async_sendmail, and :async_smtp. The :async_sendmail and :async_smtp methods deliver emails in separate threads, thus not making users wait for the delivery to complete. Hence, asynchronous methods should be used on installations that involve the delivery of a large number of emails, when a mail (SMTP) server is slow or hardly accessible, or if you are experiencing slow loading of pages that send emails (for example, when you update an issue , when you create a wiki page, and so on).

From the aforementioned delivery methods, I personally recommend the use of :sendmail or :async_sendmail, as these methods will use the sendmail system tool. This tool is a standard part of Mail Transfer Agent (MTA) (software that performs mail delivery); that is, it ships with MTA and uses MTA to send emails. In other words, with the sendmail tool, you use the default delivery configuration of the operating system. So, if a system administrator modifies the email delivery configuration of the system, Redmine will automatically use it without any changes on its side. Additionally, such a configuration is easier to maintain as you don't have to care about separate email settings in Redmine:

default:email_delivery:
    delivery_method: :sendmail

You can also use a local or remote SMTP server by choosing the :smtp or :async_smtp delivery method. In this case, you will additionally need to specify smtp_settings to let Redmine know how to connect to the server.

Tip

Avoid using external SMTP servers as in the event of an Internet connection loss, email messages won't be sent at all. Additionally, this will slow down the performance of your Redmine.

So, let's review the options that can be specified inside the smtp_settings block:

  • The address option should contain the IP or hostname of the SMTP server. If you are going to put localhost here, consider using :sendmail instead.
  • The port option should contain the TCP port number of the SMTP server. Normally, it is 25 or 587 if the TLS secure protocol is used (for example, TLS is used by Gmail).
  • The value of the domain option is used for the HELO SMTP command. Normally, it's the domain part of the sender's email address. For Gmail (but not Google Apps), you should specify smtp.gmail.com here.
  • The authentication option accepts the :plain, :login or :cram_md5 values. You should ask which value to choose to the system administrator of the SMTP server that you are going to send emails through. For Gmail, this should be :login.
  • Values of the user_name and password options are going to be used to authenticate the Redmine mailer on the SMTP server.
  • Finally, the enable_starttls_auto option should be set to true if the SMTP server requires TLS (normally, this should be enabled if the port number is 587). Set it to true for Gmail.

Tip

Avoid using a personal account in smtp_settings. Some SMTP servers (for example, Gmail) may override the from address of the email message with the address of the authenticated account. Therefore, all Redmine notifications may appear to be coming from your personal email address. Consider creating a special account for this purpose instead.

When you finish editing the configuration.yml file, don't forget to save it and restart Redmine.

The Email notifications tab

Let's check out the Email notifications tab again. Now it should look as shown in the following screenshot:

The Email notifications tab

These settings are very important as email notifications can annoy users and multiple emails from a single source can be treated as spam. So, let's review them in detail.

The value of the Emission email address setting is going to be used as the sender address in email messages, which will be sent from your Redmine. So, its default value, which you can see in the preceding screenshot, should definitely be changed! Normally, people set it to something like [email protected], but I would recommend using a real email address here, for example, the email address of your support staff.

If the Blind carbon copy recipients (bcc) setting is disabled, notification recipients will be able to see who else has been sent a copy of this notification (what could be treated as a disclosure). So, it's normally a good idea to leave this setting enabled.

The Plain text mail (no HTML) setting can be used to make Redmine send all email notifications in the plain text format, that is, without rich text formatting, links, and so on.

However, the most important setting is perhaps the Default notification option. It determines which notification mode will be set for new users. As practice shows, users rarely change this setting in their profiles, so the option you choose here is most likely going to be used for most of your users. Let's review the supported modes in detail (also see Chapter 9, Personalization):

  • For any event on all my projects: This option should never be selected by anyone except the users themselves, as in this mode, they are going to receive notifications about all events in all projects that they are members of. From my experience, I can tell you that users (especially if they are not familiar with Redmine) will most likely configure their email clients to move such notifications to a separate folder, and will never read them rather than fix the appropriate notification setting in their profiles. That's why I believe that only the users themselves should select this option. However, if you still want this mode to be the default, be sure to check as few notification actions in the Select actions for which email notifications should be sent block as possible.
  • Only for things I watch or I'm involved in: This option is selected by default. In this mode, users will receive notifications about events in objects that they own (that is, of which they are the authors) or are/were assigned to (if the object is an issue).
  • Only for things I am assigned to: In this mode, users will be notified about events in issues that they are or were assigned to (this is the same as the previous one, except that authors are not notified).
  • Only for things I am the owner of: In this mode, users will be notified only about the events in objects that they are authors of.
  • No events: This mode disables email notifications for the user.

All modes, except No events, will still send notifications to users if they are explicitly watching the object that generates these notifications (for example, an issue).

The Select actions for which email notifications should be sent block contains the list of events about which Redmine will notify users. Only administrators (and only on this tab) can determine which actions will generate notifications. In other words, if you leave, for example, the News added event disabled here, users will never receive emails with any news about any projects (they will need to check it out using the project page or news feeds)! Therefore, I personally recommend that you enable all the available actions here. Otherwise, people may subscribe to some objects assuming that they will get emails on particular events and then get frustrated with the absence of such emails. So, let it be the users who control which notifications they can get.

The next two blocks are self-explanatory. They contain text that will be inserted into the email before and after the notification message. Be sure to change http://hostname/my/account to the actual URL here.

Once you have finished configuring the email notifications, you can click on the Send a test email link to check whether the email delivery works, and how it works. This link will send a test message to the email address that you have specified in your Redmine account.

Reminder emails

Redmine issues have an optional Due date attribute. So what about being notified of the issue due date in advance? Let's do it?

Redmine ships with so-called rake tasks—small Ruby tools intended for different specific tasks that can or should be performed from the console. It also comes with a rake task that can be used to generate notifications about upcoming issue due dates. The name of this task is redmine:send_reminders, and it accepts the following options:

  • days: The number of days before the due date. The value defaults to 7.
  • tracker: The numerical (internal) tracker ID. If the value is not specified all issue trackers will be checked.
  • project: The identifier (which is used in the URL) of the project. The task will check all projects if no identifier is specified.
  • users: The numerical IDs of users and/or groups separated by commas. If the value is omitted all users will be notified.
  • version: The project version in which the issues are to be resolved. All versions will be checked if the value is not specified.

The syntax of the command that runs the redmine:send_reminders task is as follows:

$ rake redmine:send_reminders days=7 tracker=1 project=book users=1,5 versions=1.0.0 RAILS_ENV=production

Now, let's configure our Redmine to remind all users about the issue due dates a day before they are due, but not on weekends. So, open crontab (the file that specifies commands that should be run periodically) using the following command:

$ crontab -e

Tip

You may need to specify a different user with the -u option, for example:

$ sudo crontab -u www-data -e

If you are not sure whether the current account can be used, just try running the rake task manually first.

The preceding command will open an editor. Add the following two lines there:

0 10 * * 1-4 /usr/local/bin/rake -f /usr/share/redmine/Rakefile redmine:send_reminders days=1 RAILS_ENV=production
0 10 * * 5 /usr/local/bin/rake -f /usr/share/redmine/Rakefile redmine:send_reminders days=3 RAILS_ENV=production

Run which rake to determine the path to the rake tool on your server. Be sure to also replace /usr/share/redmine/ with the correct path to Redmine if you are not using the Redmine package. When you're done, save the changes and exit the editor.

So what do these lines mean? The first line will be executed at 10:00 a.m. on Mondays, Tuesdays, Wednesdays, and Thursdays, and the second line will be executed at 10:00 a.m. on Fridays.

Also note that this rake task will generate reminder emails for issues, the due dates of which are within the specified number of days. So, if you choose 7 days, users will get notifications about issues that are due in 6 days and in 1 day. Running such a task (with days=7) every day can really annoy your users. That's why we chose one day in the preceding example. In other words, you should execute this task once in the specified number of days; that is, if you use days=7, execute it only, for example, on Mondays.

Email retrieval

While the invention of email predates the invention of the World Wide Web, email is still one of the primary electronic ways to communicate. Thus, while I have a public issue tracker for my projects, I still keep getting emails regarding issues from users. Perhaps that's why email integration is one of the essential features of modern issue trackers, and Redmine does have it.

If you have taken a look at the Redmine directory structure, you might have noticed the extra directory, the subdirectories of which contain some scripts and a sample plugin. We are going to review all of these tools in this chapter (except the plugin). Soon, we'll start with rdm-mailhandler.rb. As you might have guessed from its name, this script can help Redmine handle incoming email messages.

In addition to rdm-mailhandler.rb, Redmine comes with two rake tasks that can be used to retrieve emails, namely redmine:email:receive_imap and redmine:email:receive_pop3.

The difference between the script and the two rake tasks is in the method in which they are invoked. Thus, rdm-mailhandler.rb was designed to be run on the mail server by the Mail Transfer Agent (MTA), while rake tasks are to be used to fetch emails from a remote mail server. Certainly, the script that gets executed by the mail server when an email arrives is better for several reasons: firstly, it runs only when there is a job to do and secondly, Redmine retrieves the email right after it comes to the mail server (that is, the delay is minimal). However, the tool that fetches emails from a remote mail server can be useful as well, especially if the remote mail server is, for example, Gmail (so there is no way to install the rdm-mailhandler.rb tool on it).

For the aforementioned reasons, we are going to review both these solutions. But let's start with discussing what exactly Redmine can do with incoming emails.

Handling incoming emails

In incoming emails Redmine expects to see mainly new issues. In addition to issues, Redmine can import issue comments and forum board messages from emails. So, when you receive a notification about a new issue, you can just reply to it and the message that you'll send will be added as a note to this issue. In the same way, you can answer someone's note or other changes in an issue, or someone's message in forums.

Tip

If you want Redmine to receive replies from users via email, you should change the Emission email address setting to the address at which you plan to handle such emails. Thereby, this email address will be used automatically by users' email clients when they click on the Reply button.

To create a new issue, Redmine needs to know values of several required attributes, that is, the tracker, status, and priority. Luckily, the default values for these attributes can be specified in the tools that are going to be used to process incoming emails. But what if your Redmine also requires some custom fields? Besides, will it not be too limiting to have hardcoded values of such attributes for all issues created via email?

To mitigate the issues that were just mentioned, Redmine can recognize attributes in the message body and supports many more attributes than just the required ones. Check out the sample email body:

Hi!

I have an issue…

These are attributes:
Assigned to: Andriy Lesyuk
Tracker: Support
Status: New
Priority: Urgent
Category: Email queries
Target version: 1.0.0
Start: 2016-01-01
Due date: 2016-01-18
Estimated time: 20
Done ratio: 50
Custom field: Value

Additionally, values for the issue attributes can be specified in the replies to existing issues. Certainly, whether changes will be applied to the issue, as well as whether a new issue with the specified attribute values will be created, depends on the permissions of the user who authors the email (the user is identified by the from email address). Also, to allow changes to the project, tracker, status, priority, and issue category in the email body, you need to list all of these attributes in the allow_override or --allow-override argument for the appropriate tool (see Forwarding emails from mail server and Fetching emails from IMAP/POP3 subsections).

To prevent attributes and their values from appearing in issue descriptions, issue notes, and forum messages, Redmine allows you to configure which lines the message text should be truncated after. The corresponding lines should be specified under the Incoming emails tab of the Settings page, that is shown in the following screenshot:

Handling incoming emails

Thus, for our preceding example, we could specify These are attributes: here to remove attributes from the issue description or issue note. Of course, such delimiters are to be negotiated with users.

Tip

I believe that it's essential to have the double hyphen line (--) here, as it is often used to separate the message body from the signature.

Also, if the incoming email includes To and/or CC addresses that are registered in Redmine (that is, there exist user accounts with such emails), the corresponding users will automatically be added to the watchers list of the issue. However, this does not happen for replies.

Finally, if the email includes attachments, they will be attached to the corresponding issue or forum message. Of course, the attachment size limit that is specified on the General tab of the Settings page is applied to such attachments as well. Additionally, on the Incoming emails tab, which was discussed earlier, you can specify the list of wildcarded filenames to skip while importing attachments from emails.

Forwarding emails from mail server

The majority of popular Mail Transfer Agents (MTAs) can forward incoming email to a third-party script as soon as it arrives in the inbox. And the rdm-mailhandler.rb tool, which I have mentioned before, is a script that should be used in this way. When the MTA forwards the email to this tool, rdm-mailhandler.rb builds a special HTTP request containing the email message and transmits it to a special web service (WS) of Redmine. Then, this web service processes the message and performs the appropriate action (for example, creates an issue).

So first, we need to enable the mentioned web service in Redmine. This can be done on the Incoming emails tab of the Settings page, that is shown in the preceding screenshot. There, you should check the Enable WS for incoming emails checkbox and then click on the Generate a key link. This will make Redmine generate a special key that will be needed to access the web service.

When ready, copy the rdm-mailhandler.rb tool to your mail server and put it into /usr/local/sbin. If it's a different server (that is, not the one that Redmine runs on), make sure that you have everything needed to run a Ruby script. If not, install whatever is missing.

But before we proceed, we will check out what arguments are supported by this tool. Let's start with the mandatory ones:

  • The --url argument should be set to the base Redmine URL, for example, http://mastering-redmine.com
  • The --key argument should be set to the key that we have generated
  • Instead of the --key argument, you can use --key-file, which should be set to the path of the file where the key is stored

Now, let's see what optional arguments are available:

  • The --unknown-user option accepts values such as ignore (the default), accept, or create. If the default ignore value is used, all email messages that come from an unknown email address (that is, an email address that is not associated with any user account in Redmine) will be ignored. If the accept value is set, issues created with such email messages will be authored by the special Anonymous user. And finally, if the create value is specified, new user accounts will be created.
  • There are also a couple of other arguments that can be used to control the process of user creation. Certainly, they work only when the --unknown-user argument is set to create. Thus, --default-group can be used to add the newly created users to some special group. The --no-account-notice argument instructs Redmine not to send the account information to the new user. The --no-notification arguments disables email notifications for such users (that is, it enables the No events option).
  • The --project argument can be set to the identifier of the project in which issues are to be created. If this argument is omitted, the project identifier should be explicitly specified in the body of each email message.
  • The --project-from-subaddress argument can be used to tell Redmine which email address can contain the project identifier as a subaddress part of the address (for example, in [email protected] the book is a subaddress and a project identifier). In this way, users will be able to select the project right in the email address. Also note that Redmine will look for such addresses in to, cc, and bcc email headers.
  • The --fixed-version argument can be used to set a target version for new issues.
  • The --tracker argument can be set to the name of the tracker that will be used for newly created issues.
  • The --status argument can be specified to set a different initial status for issues that are created from incoming emails (the default issue status is configured per tracker what will be reviewed in the next chapter).
  • The --category argument can be used to set a special category for issues that are created from emails.
  • The --priority argument can be used to set a different priority for issues that are created from incoming emails.
  • If the --private argument is used, all created issues will be made private.
  • The --allow-override argument can be used to restrict the issue attributes, which can be specified in the message body of the incoming email. The issue attributes that are supported by this argument are the six aforementioned ones: project, tracker, status, category, priority, and private (multiple attributes should be separated by commas).
  • If the --no-permission-check argument is used, new issues, comments and forum messages will be created, even if the author of the email does not have appropriate permissions.
  • Finally, two special arguments, --no-check-certificate and --certificate-bundle, can be used to bypass SSL certificate errors.

The rdm-mailhandler.rb script is intended to be executed in the shell, that is, it's just a console script. So here is a sample of the command:

/usr/local/sbin/rdm-mailhandler.rb --unknown-user create --project book --url http://mastering-redmine.com --key mvF868NBavZZVWinIejC

The rest of configuration depends on which mail transfer agent are you using. As we can't review all the available MTAs here, let's assume that you are using Postfix, which is configured to use a plain-text file for aliases, such as /etc/aliases.

So, open the /etc/aliases file and add the following line:

issues: "|/usr/local/sbin/rdm-mailhandler.rb --unknown-user create --allow-override=project,tracker,status,category,priority --project book --url http://mastering-redmine.com --key mvF868NBavZZVWinIejC"

Here, issues is the username part of the email address (the full address can look like [email protected]). The string that starts with the pipe (|) instructs Postfix to forward messages that come to this email address to the standard input of the rdm-mailhandler.rb tool.

After this, your Redmine should be able to receive issues, notes, and forum messages via email.

Fetching emails from IMAP/POP3

Nowadays, many companies host their email on external mail servers that they do not own (for example, on Google). So, it's impossible for them to integrate the rdm-mailhandler.rb tool with the server's MTA. Especially for such cases, Redmine provides two additional rake tasks that can fetch emails from remote IMAP and POP3 servers. As the IMAP protocol is more advanced, the IMAP rake task has more features than the POP3 one, but which one to use depends on what protocol is supported by your mail server. Many servers (including Gmail) support both.

So let's see what the command for fetching emails from IMAP looks like:

$ rake redmine:email:receive_imap host=imap.gmail.com port=993 ssl=1 [email protected] password=LIIWLmedev6M9yAJ RAILS_ENV=production

Now, let's review the options that are available for the IMAP and POP3 rake tasks. Here they are:

  • The unknown_user option, like the --unknown-user argument of rdm-mailhandler.rb, supports values such as ignore, accept, and create. The meanings of these values are the same as that ones for rdm-mailhandler.rb.
  • Like rdm-mailhandler.rb, the rake tasks support special options: default_group and no_account_notice. These options can be used to control the process of user creation (to add the user to a group and to disable the account information notification correspondingly).
  • The project, status, tracker, category, priority, fixed_version, private, and allow_override options also accept the same values and have the same meanings as the corresponding arguments of rdm-mailhandler.rb.
  • The no_permission_check option can be used to allow creating issues, comments, and forum messages by users who do not have the appropriate permissions in Redmine.
  • The host option should be set to the IP or hostname of the mail server. Its default value is localhost.
  • The port option should contain the port number of the mail server. For IMAP the default value is 143, and for POP3 it is 110.
  • The ssl option should be set to 1 if the mail server supports SSL.
  • The starttls option is available only for IMAP. It should be set to 1 if the IMAP server supports this protocol extension.
  • The username and password options specify the credentials that should be used to connect to the IMAP or POP3 server.
  • The folder option, which is supported only for IMAP, specifies the name of the mail folder on the server to process. The default folder is INBOX.
  • The apop option can be used to select the APOP authentication for the POP3 server.
  • By default, the POP3 rake task keeps email messages that it failed to process on the mail server (because, for example, they can be intended for a human). So the special delete_unprocessed option can be used to instruct the task to remove such emails.

The IMAP protocol allows moving email messages between server folders. To use this feature, the IMAP rake task supports the following additional options:

  • The move_on_success option specifies the name of the folder to which the successfully processed emails should be moved.
  • The move_on_failure option specifies the name of the folder to which the ignored email messages should be moved.

Now that we know what options we can use, let's make the rake task check the IMAP server for new emails once in an hour. To do this we'll add it into cron using the command:

$ crontab -e

This command will open the editor. Add there a line that looks like this:

45 * * * * /usr/local/bin/rake -f /opt/redmine/redmine-3.2.0/Rakefile --silent redmine:email:receive_imap host=imap.gmail.com port=993 ssl=1 [email protected] password=LIIWLmedev6M9yAJ unknown_user=create allow_override=project,tracker,status,category,priority RAILS_ENV=production

This line tells cron to run the redmine:email:receive_imap rake task on the 45th minute of every hour. Here, /usr/local/bin/rake is the full path to the rake tool, which can be obtained using the which rake command, and /opt/redmine/redmine-3.2.0 is the root directory of the Redmine installation. Also, make sure you replace the credentials with your own ones.

The redmine:email:receive_pop3 rake task can be run in a very similar way.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset