Archive for January, 2010

h1

Common Interface

January 31, 2010

After much deliberation and soul searching I’ve finally decided it’s time to address one of my all time bugbears. I am going to develop a set of transferable libraries for analysing and operating a CMS. And because I am most familiar with Perl (and, as explained in my earlier post Glue Software, I find it the most useful language for my needs) I shall write them in that language.

As with most people who have worked long enough on the coalface of SCM I have a ragtag collection of routines, scripts and utilities already but now is the time to pull them all together, clean them up, give some thought to a coherent design and build a better mousetrap that I can use whenever I go to a new client.

This is not an effort to develop a be-all-and-end-all CM system, but more a matter of me realising that I develop the same sort of scripts time and again. So I would be better off developing a more robust architecture that will allow me to exploit report generators or integrity checkers written when using one tool when I go to a client who uses another tool but has the same problem to be solved.

A typical example of a script I have written several times is one to check the integrity of a build configuration. Answering questions such as, ‘does this build configuration contain all, and only, the completed change requests assigned to this release?’

It would also be nice to have a tool for federated reporting across tools. Often clients have several version control tools and it would be nice to have one reporting front-end that I could plug into their many VCSs and get information about them all.

There are, of course, many problems to overcome (not least all the different philosophies supported by these tools) but this is not a case of trying to solve all problems in one go, rather the idea is to write a facade that will allow me to access the implementation tools through a common interface. In this way, all I need do for each client is write a new facade that complies with my own interface definitions but interrogates their particular implementation. The hard parts of these scripts, the logic, will remain unchanged and useful in multiple applications, treating the tools as databases.

As with many efforts born of frustration, this one is not completely thought out yet (in fact some of the ideas are positively vague), so expect more to come on this side project…

h1

Glue Software

January 25, 2010

As someone who inevitably becomes involved in the technical implementation of configuration management systems (CMS) I am often called upon to create what I choose to call ‘glue software’. Glue software is not a full integration between two products (often there is simply no way to fully integrate products) but rather it is a more Heath Robinson collection of scripts, triggers and hooks used to provide some connection between applications that support the CMS.

There are a multitude of technologies that can be used to create glue software, including but by means limited to; SQL, various shell scripts, C, C++, C#, Java, PHP, Python, various web technologies (XSLT, ASP etc.) and so on. Over the years I have used all of these and more but by far and away my favourite glue software development language is PERL. And here’s why… Read the rest of this entry ?

h1

Holographic Configuration Management

January 14, 2010

The advent of the ‘cloud’ and the idea of massively distributed systems (think grid computing and SaaS) is the latest technology swing with the potential to impact configuration management practice. I say ‘potential’ because a properly designed and implemented configuration management system will be able to copy without too much difficulty. The main impact will be procedural and technical, the underlying configuration management process will remain unchanged.

A properly designed CM system will be able to handle any IT system environment. Problems only occur when you are married to the idea that centralised control of CM data and centralised change control are necessary for CM, or that a specific development process is integral to CM for it to function correctly. Read the rest of this entry ?

h1

The virtue of indolence

January 12, 2010

It has long appeared to me that people see virtue in making work. I see virtue in laziness. Or rather, I see virtue in eliminating work.

As a freelancers I see part if my role on an contract as eliminating as much of the purpose for my presence as possible. This means optimizing away my role, generally through process efficiency (ensuring there are no extraneous operations in the process), automating as much as possible, and ensuring that my role is not performing tasks that can be performed by others. Read the rest of this entry ?

h1

Subversion’s Ignore List

January 8, 2010

The idea behind the Subversion ignore list is very simple: when adding (using svn add or svn import) files into a Subversion repository, any file that matches a pattern on the ignore list is skipped.

The ignore list is constructed from two sources:

  1. the client specific global-ignores list;
  2. any svn:ignore property associated with the directory into which the file is being added.

The global-ignores configuration parameter is set in the client’s config file and applies to all additions executed from that client (excluding those that specify the command line option –no-ignore). To this global-ignores list is added, on a directory by directory basis, any patterns associated with svn:ignore properties associated with directories.

The ignore list becomes a little more complicated when combined with wildcards in file names provided to the add commands. Rather than write a long post on the subject I have provided an extract from my Subversion training course (divided into two parts here because of YouTube length restrictions, it’s all one video in the course :) Oh, and WordPress, who host this blog, do not support embedded playlists. If you prefer to watch this as a playlist, go here).

If you watch these videos here I suggest you expand then to full screen otherwise they’re too small to read (again, this is a limitation of wordpress.com hosting).

Part 1

Part 2

h1

Labels versus Baselines

January 7, 2010

A common question from new Configuration Managers is, “What is the difference between a label and a baseline”.

A label is exactly what it sounds like, a name assigned to one or more things (whatever they are). Generally no further qualification is needed.

A baseline, in configuration management circles, has much more significance.

A baseline is an immutable set of items, including all the information that specifies those items and records of how those items where derived. Baselines are subject to audit to confirm their integrity. The precise nature of these audits varies according to lifecycle phase and the environment (safety critical, commercial, defence and so on). Apart from situation specific requirements baseline audits typically aim to establish, as a minimum:

  • The baseline contains all relevant items.
  • The baseline is correctly identified.
  • The records (change records and so on) associated with the baseline are secure, accurate and reflect the true nature of the baseline.
  • The baseline is secure; that is, the baseline items are held in a fashion that ensures their availability.
  • The baseline has been established through the approved process. For example, that all appropriate development, change and testing has be performed according to the established development methodology.
h1

Virtual PCs and Carbon Footprints

January 6, 2010

With all the handwringing done by companies about their green credentials and the need to reduce costs it continues to stagger me that more of them are not turning to remote working, especially for their IT department.

The availability of high performance PCs in almost every IT workers home makes this a real no-brainer. Companies could provide virtual devices to remote workers. The workers install and run these remote devices (virtual machines running approved versions of company software with appropriate VPN connections) on their own hardware and connect in to work.

Iomega recently announces a new package (pdf) that combines VMWare with one of their USB drive products that can be used to clone a PC to virtual form. This USB drive, loaded with the virtual PC, can then be plugged into another machine and the virtual PC run on the host.

Alternatively companies could provide download images for remote workers. Log in to the central server, download your virtual PC image, and run it on your home computer. Login to access the corporate network.

Both of these solutions allow for securing the virtual PC. Disabling USB ports, CD/DVD recorders and so on.

There will be some additional cost to the company in term of telecommunications, they will need improved network connectivity, but this is more than offset by the savings of not needing to provide desk space, lighting, heating, etc. for those working remotely. Additionally, all problems relating to transport problems are removed. There’s no need to worry about driving through snow to get to work when you can log in remotely (a particularly apposite example given the current weather and the estimated £1.3Bn losses caused by people being unable to reach work). And companies can claim a significantly reduced carbon footprint by virtue of not demanding their employees drive fifty miles each way to work every day and not having to provide the aforementioned heating, lighting, etc.

Although remote working cannot be practically extended to all IT workers (and some may not want to work remotely), it is certainly appropriate for many. In particular development and maintenance teams, most operations workers and support desk operators, and most, if not all, management functions. A core staff in server rooms will be required for installation and maintenance of physical assets, including the all important network connectivity.

Most IT workers’ activities do not require them to be in the same room. In fact, even workers in the same room will often e-mail or instant message one another in preference to walking to another’s desk.

With realistic broadband speeds now being available to most households in the UK it is perfectly possible for meetings to be held using conferencing software. I would hazard that these meetings would be more productive too since attendees could be working on other matters while waiting to make their contribution. Meetings could also be automatically recorded, making misunderstandings less likely.

So what is stopping companies adopting remote working?

Principally, trust. Companies simply do not trust their employees. If you are compelled to attend a site then the company feels they are better able to monitor you. This is of course hogwash. Being forced to sit at a desk in an office is no guarantee that you are being productive and any manager who thinks they are ‘keeping an eye’ on employees like this really needs to assess their priorities. A worker is assess by what they do, not where they do it. If the work can be done sitting on the beach, why not let it be done sitting on the beach? The company loses nothing, the work still gets done.

It is ironic that companies are more than willing to reduce costs by outsourcing IT functions to foreign climes but are reluctant to allow their own employees and contractors to provide the same services remotely.

The case for companies to not only allow but to actively encourage remote working is overwhelming and it will happen eventually. The question is not if, but when your company will adopt more remote working.

h1

Stubbing in build processes

January 5, 2010

When developing systems of any size the development team inevitably encounters the following problem. The developers of one sub-system need access to functionality to be provided by another, but the second sub-system is not in a position to provide the functionality and probably will not be for some time. When this happens it is common for the sub-system teams to provide stubs.

Stubs come in several flavours:

  • non-functional stubs provide no functionality, they simply provide a template interface to allow client sub-systems to compile;
  • static-function stubs allow client sub-systems to compile and also provide static responses when the interface is called;
  • dynamic stubs provide more complex responses when called. Methods used to provide these dynamic stubs vary but some common approaches are:
    • record and playback,
    • static tables,
    • randomised response, and
    • minimal function.

A stub is a proxy for the final functionality to be offered by the sub-system. As such we want the stub references in the client code to use the same calling convention as the final product. In the build process we need some means to nominate the stub or the final product into our build.

There are two principal approaches to solving this latter problem. The first relies on the build itself to select the appropriate module (stub or final). In our build script we may have, for example, a simple path modifier that builds either the stub or final code into a library which is then used by clients. The second method relies upon the source control system to deliver either the stub or final source into the source configuration, this source is then built into the target library. In this post I begin looking at some of the issues involved in both build and configuration managing stubs. Read the rest of this entry ?

h1

Stabilizing builds

January 4, 2010

One challenge facing build managers is how to control the environment in which builds are performed. How to ensure that each repeated build uses the same sources, the same libraries, the same compilers, and so on. Only by ensuring all these elements can we truly claim to be able to reproduce a build reliably and only by controlling all these elements can we be certain that derived objects built previously are compatible with incremental builds. Read the rest of this entry ?