In December 2012, research firm Coleman Parkes released a landmark study about the business benefits of service virtualization. The findings were significant: 60% of the respondents indicated that unavailability of systems for testing routinely caused software projects to be delivered late. Furthermore, 70% of those surveyed said their firms were releasing applications with reduced functionality as a result. Fast forward to July 2013, and Forrester Research releases a “Total Economic Impact Study on Service Virtualization and Testing Automation Solutions.” The study, which also looked at ROI, found that companies that adopt combined service virtualization and test automation solutions would achieve positive ROI very quickly. The test case for the study, a large European bank, achieved payback in less than a month and enjoyed a positive ROI of 1775% over three years.
Yet, despite these overwhelmingly positive metrics, many companies still experience testing downtime due to service unavailability. Or, they have coders write stubs to simulate services. 10 years ago, there wasn’t an alternative to simulation, but today, service virtualization (SV) tools can perform identically for most services―and many can even “learn” and adapt to changes in the service, over time.
To the uninitiated, SV tools may not sound that different from stubs. After all, they are simulating the service. The difference is that these tools connect directly with the service and “learn” from it until they can faithfully replicate the service’s response to the widest possible variety of scenarios. Most can also connect to the service automatically, checking for and tweaking their instruction sets to adapt to changes in the service.
SV tools can handle complex, user-defined scenarios, too, such as validating whether an application can successfully go through a series of steps that confirm a customer’s shipping preference. Some service virtualization platforms can adapt automatically to the new conditions.
At Orasi, we have helped customers needing to test performance and functionality for literally hundreds (and sometimes thousands) of services. We’ve heard many companies describe the pain of dealing with stubs, which take on their own, parallel development lifecycle as the application evolves. The benefits of SV are legion:
- Make sandboxed, scheduled or firewalled services (primary or dependent) available when they wouldn’t otherwise be accessible, enabling functional and performance testing earlier in the application lifecycle.
- Eliminate the fees associated with connecting to certain services (with the exception of initial connection and routine process updates).
- Help isolate problems based on dependencies between services (in composite applications).
- Curtail the expense and manpower of extensive code writing to create stubs.
- Ensure tests aren’t skipped due to unavailability, thereby fostering better quality.
- Reduce the cost and complexity of service testing, thereby helping to identify defects earlier in the application lifecycle.
- Run test what-if scenarios with upcoming features or other variables not currently present in the service.
Of course, service virtualization isn’t perfect. Testers that use this technology must perform spot checks against the real service when they introduce code into production. They also should be alert for data validation errors that may indicate the service and the SV tool are out of synch.
Scott Jefferies, Sr. Solutions Architect at Orasi Software
