Agile Outsourcing: Managing Technology Metrics in Software Development


A 2005 issue of the Journal of Knowledge and Process Management carries an interesting survey of companies that were engaged in distributed software development across many continents. One of the major problems identified was communication, specifically, automated tools for communication. EquaTerra, an outsourcing consultancy company, in a whitepaper on outsourcing management governance tools identifies issue management, change management, relationship and communication management as some of the factors to be monitored carefully in any outsourced software development project.

In my February 2007 column, “Metrics for Outsourced Software Development,” I outlined three main areas of attention: people, technology and process metrics. This month, I elaborate on the different technology solutions that may be involved in a typical outsourced software development project. In a future column, I’ll tackle how these solutions can address communication effectiveness. This month, I focus on technology metrics (as in the figure below) that are related to these communication mechanisms.

Technology Metrics

The three groups of technology metrics that demand attention are systems, networks and support application metrics. In the context of business process outsourcing, the first two metrics take on a lot more importance since many business processes may be executed across a network and many computer systems may be involved in executing voice, data or knowledge processes. In the case of outsourced software development, the system metrics and network metrics may take a backseat to the support applications metrics.

System Metrics

System metrics deal with primarily system uptime but can also encompass the installation and rollout of operating systems and integrated development environments. Delays in installation of service packs and other patch software have been known to cause more setbacks in software development than the lack of availability of computer systems. System update policies and how well they’re followed could be important metrics to monitor in order to reduce time wasted in tracking down bugs and defects caused by unsynchronized software environments, particularly among geographically distributed development teams. Operating system currency is also a key metric to keep an eye on. With companies upgrading to the next version of the operating system on different timetables, particularly in different countries, keeping the systems of the service buyer and the service provider synchronized can go a long way in eliminating delays caused by chasing down bugs or defects resulting from differences in operating system upgrade timetables.

 Technology metrics.
Technology metrics.

Network Metrics

Increasingly, outsourced software development efforts are managed using web- or network-accessible support applications. Network availability, whether it’s a T1 line the service provider is using or the connection to a dedicated private network, is an important metric to watch. Network latency is also a valuable metric, particularly when distributed code management systems are used.

Support Application Metrics

Support applications are usually hosted by the service buyers on their servers. Larger service providers may have portals that provide these services. In the case of distributed software development involving development teams where daily, or weekly software builds are involved, technology metrics take on increased importance. Support system availability, responsiveness and ease of use are relevant metrics in the context of support applications.

The following are the aspects of outsourced software development that are affected most by technology metrics:

Issue Management

Project management issues may be handled by a web- or network-accessible issue management application. Given time zone differences, especially in offshore software development, availability of these applications is important.

Defect Management

Browser- or network-accessible defect management systems take on great importance, especially when software development and quality assurance/testing are outsourced. Users may be testing new releases or builds of software and filing their observed defects with the defect management system. If this happens in time zones that “follow the sun” or in complementary time zones, defect management systems may require 24/7 availability.

Change Management

Portals and web-based software are available for communication and discussion of change management during the course of an outsourced software development project.

Communication Management

Collaboration portals and web-based software are used as central repositories for all documents used in outsourced software development.

Distributed Build Management

Centralized code management systems may be hosted on a web- or network-accessible server. These days, technically advanced distributed code management systems are available and could be hosted on multiple servers around the world. Geographically distributed software development teams could be checking in their own changes locally on their servers and the software would synchronize it with code bases on servers around the world. That makes availability of systems and networks in these cases vital.

People experienced in successful management of outsourced software development efforts will attest to the importance of support applications. They can also tell you stories about how seemingly smaller aspects such as system or network downtime have caused problems. Mismatched software service packs or patch releases on the systems on which software is developed have caused more wild goose chases than genuine bugs! Directing the roles technology metrics play and keeping a close eye on them is one part of the foundation on which successful outsourced software development efforts are built. Ignoring them will cause otherwise easily avoidable problems.

Useful Links

“Lessons Learned by Participants of Distributed Software Development,” by Seija Komi-Servio and Maarit Tihinen, published in the Volume 12, Number 2 issue of Journal of Knowledge and Process Management:

“Drive Value with Outsourcing Management Tools,” from EquaTerra: