Name of the blog

Short description of the blog

Comment on our product roadmap!

Looking into 2010 from a PureCM point of view

Exciting times are ahead with the PureCM 2010-1 beta now available for download and the full release coming closer. As this release will mark a very substantial step forward regarding PureCM's feature set, it looks like this is a great moment to share our product roadmap with you.

Please feel free to comment; regardless of whether you want to praise or criticise! 

Feature

Target release

 

Release planning, task tracking

  • Agile release planning for multiple projects using releases and iterations
  • Support product backlogs and drag & drop scheduling
  • Link configuration management to release planning
  • Support real-time status updates
  • Support visual reporting with burndown charts
  • Support attachments and comments for tasks
  • Support for multiple task types

 

 

2010-1

 

2010-1

2010-1

2010-1

2010-2

2010-2

2010-2

Configuration management

  • Visually show release a change has been introduced, e.g. a change or bug fix
  • Improve resolve dialogue for easier copy & paste

 

 

2010-1

 

2010-2

Integrations

  • Support working on tasks in Visual Studio
  • Support working on tasks in Eclipse
  • Support changeset references to Gemini issues
  • Add full integration to Gemini to provide a web front end and helpdesk integration

 

 

2010-1

2010-2

2010-1

2010-2

DB Backend

  • Support SQLite as embedded and cross-platform database backend
  • Support SQL Server databases

 

 

2010-1

2010-1x

 

 

PureCM and Continuous Integration - Get Started!

Continuously rising...

Continuous Integration (CI) is one of the most frequently used practises in agile teams nowadays. According to the 3rd annual survey on “The state of agile development”, 65% of the respondents practise CI; up from 50% in 2007. This is little surprising given the advantages of CI.

All these companies are aiming to make integration a non-event in their projects. No big bang integration at the end of a project anymore. No more searching for a bug that was introduced months ago.

The role of source control

So what has PureCM got to do with it? A lot, as CI relies on a source control tool to provide information about changes made (and some more areas, which we will see later on). PureCM makes this particularly transparent, as any change made is a changeset and thus a finished task. To start with a simple example, let us look at the following (mainline development) setup:

Based on this setup, we want to introduce:

  • A mainline build for each submitted change
  • Build and unit test automation
  • Instant build result communication

While automation reduces the overhead of your CI process, team communication is definitely a key element to setup. Continuous Integration works best when a team instantly knows if a change broke the build. So you no longer waste time back-tracking to find out who broke the test doing which change.

With CI, the developer in question instantly needs to fix the change until it compiles and passes the tests. This will actually save time, as the change is still fresh in his mind. Just compare this to the situation where a bug fix is needed for a change introduced several months ago...

CruiseControl.NET server installation

I will base the following on CruiseControl.NET (CC.NET), as this tool is very popular and integrates nicely with PureCM. The first thing you need to do is to setup a clean build machine; you know why ;-). Then install CC.NET, download the PureCM plugin choosing your release and copy it into the CC.NET server directory. You can also find a quick overview about the installation process in our knowledge base.

The all powerful ccnet.config file

The single most important file to configure CC.NET is the... ccnet.config file. In this file, you can define source control integration, used build and test tools, as well as file deployment and notifications. For our example web project, we use the following example file:

<cruisecontrol>
<!-- This is your CruiseControl.NET Server Configuration file. Add your projects below! -->

  <project>
    <name>PureCM Test</name> 
    <webURL>http://ServerAddress/ccnet/</webURL>  

    <triggers> 
      <intervalTrigger name="continuous" seconds="30" initialSeconds="30"/>
    </triggers>

    <modificationDelaySeconds>2</modificationDelaySeconds>

    <sourcecontrol type="purecm">
      <workspace>C:\Documents and Settings\Administrator\My Documents\Example\Development\
          VSWebExample_CCNET</workspace>
    </sourcecontrol> 

    <tasks>
      <!-- Use MsBuild to build the .Net solution-->
      <msbuild>
        <executable>C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe</executable>
        <workingDirectory>C:\Documents and Settings\Administrator\My Documents\Example\Development\
            VSWebExample_CCNET\VSSampleWebSolution</workingDirectory>
        <projectFile>VSSampleWebSolution.sln</projectFile>
        <targets>Build</targets>
        <timeout>15</timeout>
      </msbuild>

      <!-- Use NUnit for unit testing-->
      <nunit path="C:\Program Files\NUnit 2.4.8\bin\nunit-console.exe">
        <assemblies>
        <!--Path to Visual Studio project executable-->
        <assembly>C:\Documents and Settings\Administrator\My Documents\Example\Development\
             VSWebExample_CCNET\VSSampleWebSolution\NUnitClassLibrary\bin\Debug\
             NUnitClassLibrary.dll</assembly>
        </assemblies>
      </nunit>

      <!-- Email notification to users -->
      <email from="xxx@xxx.com" mailhost="hostname" mailport="25" includeDetails="TRUE" 
           mailhostUsername="username" mailhostPassword="password" useSSL="FALSE" >
        <users>
          <!-- <user name="LeadDeveloper" group="buildmaster" address="mailto:aaa@aaa.com /> -->
          <!-- <user name="Developer1" group="developers" address="mailto:bbb@bbb.com /> -->
        </users>
        <groups>
          <!-- <group name="buildmaster" notification="always" /> -->
          <!-- <group name="developers" notification="change" /> -->
        </groups>    
      </email>

    </tasks>
  </project>
</cruisecontrol>


(Get this ccnet.config example file)

Let us have a look at the different blocks. We have just an example project, so the entry is pretty short. The <Trigger> tag allows you to define the rhythm with which CC.NET scans for changes, every 30 seconds in this example. Note that CC.NET does not support triggered builds natively, e.g. onSubmit. Now what or where exactly does CruiseControl scan?

In our mainline development example, there is just one PureCM stream where changes get submitted to. So we need to create a workspace in PureCM that allows CruiseControl to retrieve the latest revision, using the same credentials as the CC.NET service. Once set up, this workspace will get updated each time CruiseControl scans for changes, eventually triggering a build.

Obviously, you can move to the next level having a staged stream hierarchy, where CI is performed on each level. But let me talk about that in a later blog.

Setting up the build...

As CC.NET is not a build tool, you need to tell it how it can create builds. This will involve your existing build tool, which is MSBuild in our example. At this point, the config file is complete enough to get the latest changes, compile them and give us feedback about the build status. Using the web dashboard, this looks as follows (no worries, we will get back to unit testing and notifications in a minute).

You can see that each time a change is detected (“IfModificationExists”), CruiseControl.NET uses PureCM’s changeset ID to identify the build. This is where you can see why PureCM changesets are so useful, as each build gets linked to a change that anybody in your team can easily understand, find and potentially... fix.

...and unit testing

However, no CI process is complete without some testing. So we install NUnit (you can get the installers here), write a few test cases and add them to the Visual Studio solution.

Once this is done, we add the reference to the ccnet.config file to include them in the build/test cycle. As NUnit is natively supported in CC.NET, the test results will automatically appear in the web dashboard. This should look like this:

 

Email notification and the ccTray

As stated initially, you would like to communicate the results of a CI cycle back to your team, so anyone knows whether the mainline is stable or a developer has to fix a recent bug. You can do this via email as in our example, which includes a link to the web dashboard as specified in the <webURL> tag. Alternatively, you can install the ccTray that displays the status of your CI projects in the Windows Taskbar. This instant messenger like service runs real time, so you can get the current status of your CI projects with a glimpse to the lower right of your task bar.

 
ccTray, image copied from the CruiseControl Wiki

As this blog is getting longer and longer, let me just indicate the link to the CruiseControl online documentation (it is really easy!).

And finally...

So this is it – we are ready to go! I hope that this blog has facilitated your initial CI setup. Of course, the CI process might need some time to become as normal as submitting changes to PureCM for your team  ;-). But try to follow some agile principles like submitting code at least once per day, and you will quickly realise that integration in small and frequent steps not just increases codeline stability, but also productivity. And feel free to let me know about your experiences!

Better Merging - Part 2

Following on from Tim Rutter’s article ‘Better Merging – Part 1’, we have looked at how to improve the merging capability of the tool even further by introducing Merge Paths and a Merging perspective for the 2009-1 release (see Stephen Worthington’s blog on perspectives – ‘PureCM gets a complete makeover’). This is especially useful if you have a staging hierarchy, eg. Development, QA, Release. In such a situation, pushing changes to the next level becomes very transparent. There are many benefits from introducing these merging features, here are the key ones:

Centralized location

A new Merging Perspective provides a central location where users can easily see which changesets have been merged between streams as well being able to carry out required merges. From here users can create new merge paths and see all existing ones. A merge path consists of 4 nodes:

Merged Changesets – Here you can see which changes have been merged from the source stream to the destination stream.

Pending Changesets – Changes that have failed the quick merge process will end up here, allowing you to carry out a full merge, resolving any conflicts (see "Better Merging – Part 1" blog). Changesets can be moved here directly, from ‘Unchanged Changesets’ or all changesets could be moved here for certain types of merge paths. We’re going to introduce an option that would force all changes to become pending for the path. So for example if you have created a child stream and you know you want all the changes that will be submitted to the parent then it makes sense to make all the changesets pending.

Unmerged Changesets – All changes that have been submitted to the source stream, but have not yet been merged to the destination stream are shown here, with functionality to carry out the merge, move to pending, or move to excluded.

Excluded Changesets – When a changeset should not be merged to the destination stream, it can be placed here, helping other users to avoid performing unwanted merges.

Creating the Merge Path..

The display name can be edited to provide a friendly name.

Automatic Merging

Automatic merging optionally takes place when changes are submitted to a stream. So for example, if multiple streams exist that are based on one ‘main’ stream, and it is important that these streams are kept up to date with Main, then a merge path could be created for each of these, with the option to ‘automatically  merge’ switched on (ie Main -> Stream1, Main -> Stream2 etc). This would result in the submitted changes being quick merged to the other streams at the time of the submit. If the quick merge fails, due to conflicts, the changesets would then appear under the Merge Path node as a pending change.

When working with automatic merging, Continuous Integration becomes a very useful practice to control code quality, so look out for a future blog on Continuous Integration...

Further usability improvements

We’re well underway in the development of these new features but have plans for more usability features:

Allowing one off merges to take place from the Merge perspective is an idea we are considering – so rather than having to navigate to a stream, or Workspace, you should be able to use the Merging perspective to do any merge. A ‘My Pending Merges’ node will be useful for developers who have submitted a change to a stream and then need to merge to other streams. This will list only their changesets that need merging. An ‘All Pending Merges’ node will be useful for managers or administrators responsible for overall merging to quickly see what needs merging.

We’re always open to suggestions or ideas so if you have any regarding merge paths, or merging in general, please let us know.

 

Better Merging - Part 1

One of the great strengths of PureCM has always been it's support for parallel development. PureCM's merge functionality allows you to keep track of your changesets and which streams they have been submitted against while making it easy for you to merge these changes into other streams as desired. Now we are looking at making merging even more powerful and simple, but let's first take a look at how merging works today.

Merging Today

The original merge process required you to own a workspace for the stream you are merging into. This gives you a number of benefits such as the ability to easily test each change before submitting it to the server. This is shown as (B) in the lower half of the picture below.

However this process can be time consuming and is hard for manager who may not always have a workspace for each stream. This led to a solution that allows users to quickly merge changes between similar streams, which we called “quick merge”. This allows you to merge changes directly on the server removing the need for a workspace in these situations (A in the picture above).

So far, quick merge will only merge changes where there is no possibility of a conflict. This works well for merging last minute fixes into a release stream or early on in the development cycle when not much has changed in the new development stream but we felt there was a need to improve on this restriction.

Moving Forward

The next stage in this evolution is to allow you to resolve conflicts as part of quick merge without the need for a workspace. This will simply launch the familiar PureCM resolve tool when quick merge finds a conflict.

Combining this new functionality with automated test and build processes (such as PureCM's Cruise Control plug-in) provides a fast and simple way to keep your streams up to date while giving you the security that any errors are picked up immediately so you can assign someone to look at it.

This new advancement to quick merge opens up a realm of possible future features for the tool to help speed up your development process through merging. One example of this is the creation of automatic merge paths to control the flow of your changes through your development streams. Plenty of reasons to look forward to the 2009-1 release – and keep your eyes open for future blogs for information about where we go from here!

Towards Rebasing

Overview

In order to better support Agile development, we're looking to allow easier Workspace Rebasing.  Currently, when you finish working in a particular task stream, you have to create a new workspace when you start work on another.  The idea behind rebasing is that rather than having, for example :-

 

and 'rebase' to v1.2 and v1.3 as your workload requires.  Of course we'll always remain flexible to allow you to choose your own way of working.

Initial Steps

Before we can make Rebasing a reality we felt that we needed to address some of the Workspace infrastructure.  From inception, PureCM has always had a workspace database that sits alongside your code and holds the revision, checkout and file information about the workspace.  Since this holds the file data (in compressed and delta-ed form) it grows in size proportional to the size of the files in your workspace.  We've always liked this approach since it makes certain operations easy to implement offline (i.e. no server connection)

The first stage of the rebasing work involves minimising the size of the workspace database (since most of the information is thrown away as you rebase).  We're nearly complete now and the results are encouraging.  Some of our internal workspace databases can be reduced in size from roughly 500MB to 10MB.  Obviously the size of the files in the workspace doesn't change but it's quite a healthy saving nonetheless!  Early performance figures indicate roughly a 25% speedup when creating the workspace as well.  These new workspaces can also be used offline although certain functionality is slightly limited (e.g. you can't always view file differences offline).  You can always create a traditional workspace that holds full offline data if this is a concern.

Looking Forward

The next step is to actually implement the Rebasing support.  The initial plan is to provide the UI/Dialog for this from the PureCM GUI as well as the Command Line client.  We can't currently make this work from within Visual Studio - our SCC Provider doesn't have the ability to add custom commands/toolbars so we'll have to look at a true Developer Studio AddIn for added functionality such as this.  The rebasing functionality itself is scheduled to be shipped as part of our 2009/1 release in early 2009.