Repository integration

Repository integration is an awesome feature of Redmine. I just can't imagine this application without it! So, I believe that you will like Redmine for having it. But as soon as you get to know how deep this integration can be, I'm sure you will love Redmine even more!

We will start by configuring repositories and ensuring that the basic integration works. Then we will discuss how to turn Redmine into a repository manager.

The Repositories tab

So, here we come to the last tab on the Settings page—the Repositories tab:

The Repositories tab

The exclamation mark near the name of an Source Control Management (SCM) command indicates that there is some issue with this SCM. The small text below the checklist in the Enabled SCM block advises that you may need to specify proper paths to the commands in config/configuration.yml. But you should do this in rare cases, for example, if you have installed SCMs from sources. And in most other cases, the exclamation mark just means that the SCM is not installed (as in this case).

So, let's shortly review how to install these SCMs on Ubuntu/Debian. Of course, you are not likely to need all of them (you should have decided which SCM to use after having read Chapter 1, Getting Familiar with Redmine).

SCM

Installation command

Subversion

$ sudo apt-get install subversion

Darcs

$ sudo apt-get install darcs

Mercurial

$ sudo apt-get install mercurial

CVS

$ sudo apt-get install cvs

Bazaar

$ sudo apt-get install bzr

Git

$ sudo apt-get install git

If you have installed any of these SCMs, do not forget to restart Redmine to make it pick them up. Afterwards, if you go to the Repositories tab, you should see check marks instead of exclamation marks near the commands and SCM version numbers in the Version column:

The Repositories tab

If any of the listed SCMs are not installed (or exclamation marks are still shown for them for some other reasons), you should disable them here by unchecking the corresponding checkboxes.

Tip

Before enabling the Filesystem SCM, you should set the scm_filesystem_path_regexp option in config/configuration.yml to a regular expression that will ensure that a proper path is used! Otherwise, users who have the Manage repository permission can get the ability to read the filesystem of the server that runs Redmine.

Other blocks of the Repositories tab are reviewed in the following subsections.

Fetching commits

The main feature of Redmine SCM integration is the repository browser, which is available under the Repository tab of a project. Here is the screenshot that shows how it looks on Redmine.org:

Fetching commits

This repository browser can be used to view or download files, browse directories and revisions, compare revisions, see what changes were made to each file, and more. But to be able to see new changes, Redmine needs to fetch them from the repository first. By default, it does this each time a user accesses anything under the projects/<project>/repository URL (that is, under the Repository tab of the project).

In other words, Redmine fetches new commits from the repository when you click on the Repository tab, on the trunk directory of the repository, on a file of such a directory, and so on! If Redmine does not have to serve many users at the same time and the repository is local, this should not be a big problem. But what if the Redmine installation is heavily loaded and uses remote repositories?

This default behavior is enabled by the Fetch commits automatically setting, which can be found on the Repositories tab of the Settings page. So, to make Redmine stop updating commits on each user visit, you just need to disable it. But how will Redmine fetch new data in this case? Without such updates, the Repository view of the project is going to become outdated sooner or later.

Luckily, Redmine offers several other solutions for fetching new commits from repositories. Which one to use depends on whether the repository is local or remote and from where the update can be initiated. Generally, there are two places where you can initiate this process—from cron or from an SCM hook.

Using cron

Cron can be used to trigger an update of repositories within Redmine. However, this can be done only on the same server where Redmine is running. Still, it's better than the default update-on-request option, as you can control how many times the update is executed.

To trigger the update, you need to use the following command:

$ rails runner "Repository.fetch_changesets" -e production

So, this command should be put into crontab as follows:

*/30 * * * cd /opt/redmine/redmine-3.2.0 && /usr/local/bin/rails runner "Repository.fetch_changesets" -e production

Don't forget to replace /opt/redmine/redmine-3.2.0 with the actual path to your Redmine installation.

This line will trigger the fetching of commits for all projects every 30 minutes.

Tip

This command can also be used to make Redmine load data from a huge repository that was just added to a project. Doing this from the browser (using the Fetch commits automatically mode) can be troublesome.

The second option is more advanced and flexible, and it is to be used on the server where the SCM is running.

Using an SCM hook

All SCMs—at least the ones that I used to work with—support hooks. Hooks are scripts that are located in a special directory or are listed in a configuration file of the repository. Thus, for Subversion this directory is hooks. For Mercurial hooks are to be specified under the [hooks] section in .hg/hgrc. For CVS, they can be specified in the commitinfo file. For Bazaar, they can be either put into the .bzr/hooks directory or listed in the bazaar.conf file. For Git, they should be put into the .git/hooks directory. These scripts are triggered when an event occurs, for example, when a user makes a commit to the repository. And this is exactly where we should initiate fetching the commits for Redmine!

To do this, we need to use another special web service (WS) that is provided by Redmine and that can be accessed using the following URI: sys/fetch_changesets. But first, we need to enable it on the Repositories tab of the Settings page:

Using an SCM hook

Check the Enable WS for repository management checkbox here and click on the Generate a key link. Then press Save.

Now, you need to create a hook script for your SCM (check out the SCM's documentation for the format, the name of the script, and the location) and put the following command there:

curl -o /dev/null "https://mastering-redmine.com/sys/fetch_changesets" -d key=oVVm1CmvXJnpRVmTZ5ii > /dev/null 2>&1

Replace https://mastering-redmine.com with the actual URL of your Redmine installation and oVVm1CmvXJnpRVmTZ5ii with the key that was generated for you.

Tip

Use the HTTPS protocol in the URL (if available) to encrypt the key when it is sent over the Internet.

Different projects usually use different repositories, which can be located on different servers, and even use different SCMs. For such cases, this web service supports an additional parameter: id. This parameter should contain the project identifier, like in the following example:

https://mastering-redmine.com/sys/fetch_changesets?key=oVVm1CmvXJnpRVmTZ5ii&id=book

Actually, this is the right approach—each repository should be configured to update only the project to which it belongs. But, this means that each repository is to be configured separately, which can be a headache (especially if your Redmine installation hosts many projects). However, Redmine will otherwise update all repositories in all projects when a commit is made to any of them.

Tip

The Git SCM supports only local repositories in Redmine, but it's possible to make a clone from GitHub and then keep the local copy up to date using Jakob Skjerning's GitHub Hook plugin:

https://github.com/koppen/redmine_github_hook

Do not forget to disable the Fetch commits automatically setting on the Repositories tab when you have configured all your repositories to trigger the update of commits through the web service.

Automatic creation of repositories

In the previous subsections of this section, we discussed how to configure Redmine as a repository browser, which is a well-known feature of this application that actually works out of the box (if the SCM is installed). Now let's speak how to turn Redmine into a repository manager.

Redmine comes with a special tool intended for creating repositories for projects that do not have a repository yet. The name of this tool is reposman.rb, and it's located in the extra/svn directory of Redmine. It is to be executed periodically by cron on the SCM server. When it is run, it connects to Redmine, fetches its projects list, checks whether a repository exists for each of the projects, and creates a repository if the project does not have a repository yet. But note that it expects the project's repository to have exactly the same name as the project itself (it uses the project identifier). Also note that to use it, you'll need access to the server that hosts SCM (or is going to host it).

To proceed, you need to know where the repository files are to be located. If the SCM server has already been configured, figure out what the path for repositories is and which SCM is to be used. If no SCM server exists yet, let's just create a directory for repositories for now; later on, in the next subsection, we'll configure the server.

It is assumed that you are going to use Subversion. To conform to the FHS, let's select /var/lib/svn as the directory for the Subversion repositories:

$ sudo mkdir /var/lib/svn

If you have not enabled the web service for repository management, as described in the Using an SCM hook subsection of this chapter, do it now. This web service is going to be used by the reposman.rb tool to check the repositories list.

Also, before creating a repository, we need a project that this repository will be added to. This can be either a test project or a real one. A new project can be created using the New project link, which becomes available when you click on the Projects item in the top-left menu. So please create it, if you don't have a project yet.

When ready, open the console on the SCM server and test the reposman.rb tool by running it in the following way:

$ sudo /opt/redmine/redmine-3.2.0/extra/svn/reposman.rb --owner=www-data --svn-dir=/var/lib/svn --url=file:///var/lib/svn --redmine-host=mastering-redmine.com --key=oVVm1CmvXJnpRVmTZ5ii --verbose –test

Change the paths, the hostname, and the API key to the correct ones.

Tip

If the SCM server is located on a different physical server, you'll need to copy reposman.rb there and possibly install Ruby.

Let's review the various options that are supported by this tool:

  • The --svn-dir option should point to the directory under which new repositories are to be created.
  • The --redmine-host option should hold the IP or hostname that will be used by the reposman.rb tool to connect to Redmine.
  • The --key option specifies the API key of the web service.
  • The --owner option specifies the name of the system user who will own the repository files on the SCM server.
  • The --group option can be used to specify the name of the group to which the repository files will belong.
  • The --url option specifies the URL that can be used to access repositories externally (for example, http://mastering-redmine.com/svn).
  • The --scm option should be specified if any other than the Subversion SCM is to be used (its possible values are Subversion, Darcs, Mercurial, Bazaar, Git, and Filesystem).
  • The --command option can be used to specify a custom command for repository creation, and it is required for SCMs other than Subversion and Git.
  • The --key-file option can be used as an alternative to --key as it allows you to store the API key in a file.
  • The --test option can be used to check what the tool is going to do.
  • By default, the reposman.rb tool creates a repository only if the corresponding project does not have any repository yet. This can be changed by the --force option, which instructs the tool to always create the repository with the project's identifier in its name, if such a repository is missing.
  • The --verbose option can be used to get more information about what the tool is doing.
  • The --quiet option is to be used if you do not want the tool to give any output to the console.

If the test run worked fine, we can use cron to make the tool be run periodically and create repositories, when new projects are registered. To do this, add the following line to crontab:

15 * * * * /opt/redmine/redmine-3.2.0/extra/svn/reposman.rb --owner=www-data --svn-dir=/var/lib/svn --url=file:///var/lib/svn --redmine-host=mastering-redmine.com --key=oVVm1CmvXJnpRVmTZ5ii --quiet

This tells cron to run reposman.rb on the 15th minute of every hour.

From now on, all your projects should have repositories created for them automatically.

Tip

The SCM Creator plugin

For easy repository creation, you can also use the SCM Creator plugin. This plugin currently supports Subversion, Git, Mercurial, and Bazaar. For all of these SCMs, except Git, it can create only local repositories (located at the same server on which Redmine is running). For Git, it can create local and remote GitHub repositories. You can check it out at:

http://projects.andriylesyuk.com/project/redmine/scm-creator

Advanced repository integration

If you check a role's permissions on the Roles and permissions page of the Administration menu, you will see the Commit access permission. This permission has nothing to do with Redmine as a web application, as no commit access is, in fact, possible within Redmine. Actually, it is for what is called advanced repository integration. Thus, this permission is going to be checked by an extra tool called Redmine.pm, if it has been properly integrated. And like other similar tools this one can be found in the extra/svn directory of Redmine.

Normally, a Subversion server uses the Apache web server with the WebDAV module. The same configuration can be used for a Git and a Mercurial server. And the Redmine.pm tool is actually another module for Apache that handles authorization.

So how does it work? When users request access to a repository, Apache asks Redmine.pm whether it should authorize or forbid them. To give the answer, Redmine.pm reads data from the Redmine's database. Access is granted or denied depending on the user's role and permissions. Thus, if the user's role in the project has the Commit access permission, the user is allowed to commit changes to the project's repository.

Tip

If you want to allow read-only access to the repository for anonymous users, make the corresponding project public and grant the Browse repository permission to the Anonymous role.

Let's now see how to configure advanced integration for Subversion:

  1. We'll start with installing and configuring the Subversion server. It is assumed that you are going to use Debian/Ubuntu to run the SCM server and you have already installed Apache. Execute the following command:
    $ sudo apt-get install libapache2-svn libapache2-mod-perl2 libapache-dbi-perl

    This command will install the WebDAV and ModPerl modules for the Apache and the Perl DBI library.

  2. Now, copy Redmine.pm into the Authn subdirectory of the /usr/lib/perl5/Apache directory:
    $ sudo mkdir /usr/lib/perl5/Apache/Authn
$ sudo cp /opt/redmine/redmine-3.2.0/extra/svn/Redmine.pm /usr/lib/perl5/Apache/Authn/
  3. Next, we need to know where the repositories are going to be located. If you have chosen the path in the Automatic creation of repositories subsection, use that path. Otherwise, let's use /var/lib/svn:
    $ sudo mkdir /var/lib/svn

    At the moment, we have all that we need to start configuring Subversion with Apache.

    There are two options for adding the Subversion configuration to Apache configuration files: use an existing <VirtualHost> directive if you want Subversion to be accessible under a path such as /svn, or add a new <VirtualHost> directive, if you want to use Subversion under a subdomain, (for example, svn.mastering-redmine.com). We will choose the first option (for the second option, you need to create a new virtual host first). Also, we will use the redmine.conf file that we created while installing Redmine on Ubuntu using the official tarball (see the Installing Redmine from sources section of Chapter 2, Installing Redmine).

  4. Open the configuration file of the virtual host and add the following lines before the closing </VirtualHost> directive:
    PerlLoadModule Apache::Authn::Redmine
<Location /svn>
        DAV svn
        SVNParentPath /var/lib/svn

        Order deny,allow
        Deny from all
        Satisfy any

        AuthType Basic
        AuthName "Mastering Redmine SVN Server"
        PerlAccessHandler Apache::Authn::Redmine::access_handler
        PerlAuthenHandler Apache::Authn::Redmine::authen_handler

        RedmineDSN "DBI:mysql:dbname=redmine;host=localhost"
        RedmineDbUser redmine
        RedmineDbPass your_password_here

        <Limit GET PROPFIND OPTIONS REPORT>
            Require valid-user
            Satisfy any
        </Limit>
        <LimitExcept GET PROPFIND OPTIONS REPORT>
            Require valid-user
        </LimitExcept>
</Location>

    Instead of the credentials that are used in this sample configuration (the database name, username, and password), use the real ones. If you are not sure about them, take them from the Redmine's database.yml file.

    Tip

    You will also need to replace localhost with the IP or hostname of your Redmine installation if you are using a separate server for Subversion. Additionally, you may need to modify the bind-address option of the MySQL server in this case.

  5. When ready, restart Apache:
    $ sudo service apache2 reload

This is it! You are now ready to go with advanced Subversion integration.

Advanced Git integration can be configured in a very similar way (with some Git-specific configuration steps). The same can also be done for Mercurial and Bazaar. However, it would be unfair to only additionally review Git in this section, and it would be too much to review all SCMs here. So for other SCMs, you should check out the official tutorials for advanced integration at http://www.redmine.org/projects/redmine/wiki/HowTos. See the Source Code Management (SCM) section there.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset