Given the modest success of that experimental session, I have planned a series of similar events (details can be found here).

Currently planned are:

• “What is a configuration Item?” – Friday 4th March 2011
• “Version control for beginners” – Friday 18th March 2011
• “Branching. What, when, and how” – Thursday 31st March 2011
• “Parallel development. What, when and how” – Friday 15th April 2011
• “Selling configuration management” – Friday 29th April 2011

All meetings are free to attend and are currently planned for 20:00GMT (There really is no good time that covers all time-zones. If enough people ask I may repeat meetings at other times to cover other timezones—better yet, why not organise one yourself .)

I’ll certainly be chatting for half an hour at least (if no one else steps up to contribute I can certainly talk about any of these topics for at least that long ) but I’ve allowed two hours for each session and am happy for sessions roll on beyond this if there is enough interest.

These are informal discussion groups intended for any level of expertise (although obviously some are aimed primarily at beginners I would welcome input from experienced users too). I will probably make some opening remarks to get things rolling, but I am hoping enough people are interested to make for some real discussion.

You can contribute through the text ‘chat’ on the site or better yet if you have a mic (and video camera if you like) you can step up and present your ideas that way.

Come along if you have questions, are just interested in the topic, or have something to say on the topic being discussed.

Let’s put this into concrete terms.

Suppose the original change request is something like, “correct the spelling mistakes in the poem” and the poem is held in two files.

file1.txt contains the following text.

Mary hard a litl lamb,
It's fles was wite as snow,

And file2.txt contains the following text.

And everywere that Mary went,
That lam was sure to go.

We correct these two file to the following using a change record CR1.

Mary hard a little lamb.
It's fleece was white as snow

And everywhere that Mary went,
That lamb was sure to go.

Here then is my first build (build1).

Initial build: conceptual and literal change equivalent

So, not bad, but the still have “hard” instead of “had” in the first line of file1.txt. This does not change the integrity of build1, but does make another build (build2) necessary to include a correction to the erroneous work submitted into build1.

There are different ways we might choose to track the correction of the defect arising from build1.

Each approach has its own merits but, from the perspective of build integrity, they share an the important feature; when dealing with the conceptual change “correct the spelling mistakes in the poem”, they are semantically equivalent. For build2 to have integrity it must include both the revisions associated with the original change and all new revisions associated with the correction to the change, regardless of how that correction is controlled.

We correct the defect in build1 with an additional change to file1.txt to produce the following.

Mary had a little lamb,
It's fleece was white as snow,

The following illustration shows build2 (in this case I have re-used CR1 to correct the defect). I should be obvious that missing any revision associated with CR1 has the potential to break the build’s integrity (we may have only part of CR1 in the build).

Fix Build: Conceptual change implemented with reworked literal change

Fix build: Conceptual change now implemented with two literal changes

And this is the point. Yes, the literal changes CR1 and D1 are, in one sense, changes. In practical terms though each on its own fails to achieve the objective of the original change request and consequently they are little more than sets of revisions. The two literal changes controlled by records CR1 and D1, taken together, achieve the original goal of the change request “correct spelling mistakes in the poem”. It is this latter type of change (what I have called conceptual change) that is more interesting to configuration management and build integrity is maintained through these conceptual changes rather than the less significant literal changes.

This distinction between the conceptual change and literal change becomes increasingly important as we consider configuration integrity, a subject I shall discuss in more detail in subsequent posts.

It is easy to become entangled in confused terminology when discussing items and item histories. To avoid such confusion I will define an Item as a specific version of something we control (most commonly in software configuration management these ‘things’ are files, so for the remainder of this post I will use files to illustrate, but remember that an Item could be anything we version control).

With this definition in mind we can define one type of history as a set of Items related to one another by a predecessor/successor relationship. This is the most common history one becomes familiar with, the revision history.

But there is another history worth considering. The process of managing an Item may involve the Item passing through a number of states. We commonly refer to these states as a lifecycle. An Item may be created into the NEW state, then be edited by a developer in the DEVELOP state. Once completed the developer may submit the item to the REVIEW state for a code review. The item may them pass through various states before finally reaching the RELEASED state. The lifecycle history is independent of the Item revision history.

To illustrate the difference between the lifecycle history and the revision history consider file1.txt shown below. This file has three revision Items; 1, 2, and 3.

Zooming in to the file1.txt revision 3 Item we see the following lifecycle history.

file1.txt revision 3 passed through all of these lifecycle states, passing through the DEV-TEST states twice as a defect was found and resolved.

Version control tools track the revision history but seldom the Item lifecycle history. Configuration management tools, such as Dimensions, capture both the revision history and the lifecycle history.

Any generalised tool for analysing and reporting configurations from as wide a variety of sources must deal with these different histories.

1. People are often more afraid of the perceived problems than the practical realities of parallel development.
2. People do not truly appreciate the problems and practical realities of version control and parallel development.
3. There is a need for more theoretical work on the topic and, perhaps more significantly, a need for more formal expression of the process and problems involved.
4. There is a quite substantial book that could be written on the subject.

The first two points are related to the last two insofar as people need a clear explanation of the discipline and, to really acquire a firm understanding of the subject, you need to commit to studying it. There is simply no easy way to explain the more complex material (and by simple I mean one side of A4 summaries demanded by people who think they need to know but who have no practical exposure to the technical nature of software development or version control — there is no ‘managers guide too..’).

As to the third point, I have expressed a concern about this before in various forums. Rather than continue to observe the problem I have decided to start doing something about it. So, over the coming weeks and months I will be expanding some ideas I’ve been mulling over for many years in a series of posts to this blog (special category set up here) and on my website. Be warned though, these articles will probably get very theoretical, often be speculative, and may raise as many new questions as they resolve existing ones. We shall see.

As for the final point, as I build up a sufficiently cogent set of material while writing on the blog (a process unlikely to be well structured), I will assemble more coherent articles on my website and who knows I might get round to pulling them together into a book one day. (Ha! Another of the many books I either intend to write or am in the process of writing but may never finish or publish — *sigh* so many things to do, so little time.) I may even pull material together and add it to the CM wiki or the ITSLM BoK. You never know, stranger things have happened.

Rather than start here, I’ll start an introductory post right now and hopefully have it up in the next day or two. In the meantime, you could check out some past posts on the subject; here and here

Simple Merges

The most basic merge is a very straightforward (although, as we shall see, can have hidden complexities).

Simple Merge

The most obvious limitation of these simple merges is that the file must be ‘mergeable’, which for most tools means the file must be record oriented. Text files are a collection of records, one for each line in the file. Binary files tend to be resistant to merge unless a special tool is provided (MSWord can merge two documents with reasonable accuracy).

Conflicting Merges

Sometimes a merge involves changes to the same records (the same lines in a text file) in the same file. The illustration below shows just such a merge.

Conflicting Merge

Sometimes these conflict resolutions will require more than a simple decision about which line from the two changes should be adopted and the developer will need to rework both changes to make them work together. Fortunately the majority of merge conflicts are relatively straightforward providing integrations are performed frequently.

Renamed and Moved Files

While merging a file’s content is commonly supported by version control tools, tracking renamed and moved files during a merge operation is less well supported but in todays software development world, where refactoring is common, it is increasingly essential.

The problem with renamed and moved files are summarised in the following illustration.

Merge with rename

Rename/move conflicts

Merges involving renames do not necessarily run smooth. Not only are they subject to the same content merge issues as the simple file merge but also the problem when the file is renamed/moved in both branches. This situation is illustrated below.

Rename conflict

Mitigation

Before moving on to some other issues you will encounter when using parallel development, we pause briefly to consider how to avoid, or at least mitigate, some of the issues mentioned so far.

Firstly, despite the impression the previous sections may have conjured, merging is not fraught with perils. After many years of managing parallel development on projects ranging from the simple to very complex, I can assure you that, with a little common sense and some forethought the issues can be minimised (but seldom completely eliminated).

1. Merge as early as you can, and then merge often to keep each merge as small and manageable as possible.
2. When you have input to a project’s design, ensure a sound architecture. This will minimise overlapping changes and consequently merge conflicts.
3. If you identify files that are merged often, try to decompose the file into smaller files to avoid high merge rates. (This is common with files used for configuration or files containing a lot of utility code.)
4. Keep refactoring on one branch. Renaming and moving files can cause tricky merge issues when performed on many branches.

Now, on to some issues that, while not unique to parallel development, can make your life difficult if not managed properly.

Versions and Delta

A simple linear development progresses by building one version on another. So, version one is updated to produce version two, version two is changed to make version three, and so on. This form of version progression is well understood whether the thing being controlled is a document, source file, or an entire system.

The diagram on the right shows a simple linear version line. Versions one and two are separated by the change (or delta) ‘a’. This can be read as version ’1′ with changes ‘a’ becomes version ’2′.

If all development work could be relied upon to be this linear then this would be a very short series. Fortunately for us, this simple linear model cannot be relied upon for anything but the simplest development work.

Bring on the branch

Suppose we have released version two of our product to formal user testing and have begun development of version three of the product. Formal user testing will last for at least six weeks and after that we expect that the product will be in use for some time before version three is ready.

During testing we expect to find some defects and the users will undoubtedly request some ‘fine tuning’ of version two.

This situation cannot be accommodated with the simple linear development process described above. What we need is some means to manage the corrections to version two while version three development is carried out.  Of course, any changes we make in version two must eventually be included in version three (otherwise defects corrected in version two will suddenly reappear when we release version three). However, we do not want changes made when developing version three to be included in version two maintenance (otherwise we would find partially implemented version three changes suddenly causing problems in the user test version one systems).

Basic Branch

The diagram above shows a simple branch. Our main development (the work developing version three of our product) continues, while maintenance work is branched off from version two of the product (shown above as version ‘a1′).

Any changes made between version ’2′ and ’3′ in the main line development have no effect on the branch.

The branch is used to make changes such as defect resolution during the testing of version two. Any changes on the branch (between version ‘a1′ and ‘a2′) must be integrated back into the main line development before version ’3′ is released for testing. This ensures that any defects fixed during version two testing are also fixed in version three.

Branching really is that simple. However (there’s always a ‘however’ in there somewhere), while this is schematically correct and simple, in real life things can get more complicated. I leave dealing with these complications for another day.

As always, any questions can be addressed to me directly or through the comments on this post.