When selecting tools, for any purpose, beware the ideal case.
When a product is demonstrated, online or directly, the presentation has, generally, been carefully worked out in advance to present to tool in the most favourable light possible. The tool will perform optimally and the use case will appear flawless. Sadly, real life is never an idealised case like this.
It is very easy to be beguiled by these idealised demonstrations, they look so smooth, so simple, and they seem to address some serious development issues. It is difficult to remain objective, especially when the tool seems to offer an easy solution.
Real life is not a demonstration though. It takes time to migrate to a new tool. Tools all have limitations, and you need to know what these are and how likely they are to cause you difficulties.
You need to know how difficult it will be to migrate your existing system over to the new system. Many tools will demand that you work in a particular way in order to get the best from them, so remember to account for the cost of changing your process too. Are you going to need to sacrifice integration with other tools because the new tool does not integrate properly? Then there are training costs in order to get the whole project trained on the new tool.
Another irritant with many modern tools is the lack of a command line, or an impoverished command line that does not provide access to all of the tools functionality. The move to more and more GUI based interaction presents difficulties for real life scenarios where it is often necessary to integrate tools using scripts. Be careful of the lure of a powerful API. APIs are wonderful to have, but they are seldom as accessible as a command line interface, making quick integrations more difficult to implement.
Do not accept a demo as a reason to change tool. Demand that the tool vendor show your system using their tool. Demand that the vendor show how your system would migrate to their tool. Demand that they show the migration process, how the tool will integrate into your existing process or what you will need to change, evaluate the total cost of migrating to the tool, and so on. Any vendor that refuses, or cannot show the cost benefit of this migration should be dropped immediately from you evaluation. No matter how good the tool looks, no matter how well it performs with an idealised demonstration system, if it cannot perform for your system it is useless.
All of this may seem like I am arguing against looking at new tools, or that I have some downer on new tools in general. I do not. I love new toystools and I think tools should be constantly reviewed in order to make sure we have the best tool for the job. Just remember, the best tool for the job does not necessarily mean the newest tool and the newest tool may come at too high a cost in terms of impact on your environment and process.
