APPLICATION NOTE: Why Product Evaluations Are Essential

TestWorks is a rich and diverse product set, but in spite of its advantages it cannot do everything in every situation; it cannot be everything to every user. Hence, a detailed "try it before you buy it" evaluation is going to be a crucial process.

Software testing is always a difficult process -- one for which the positive payoffs are very great, but for which the negative payoffs are very painful. Some software sellers would have you believe that their product, based on a demo that they deliver to you, is the ``end all, be all, do all, final solution'' to all of your needs. Very often, this is what is pitched at an on-site demo, normally attended by some very high-powered, well practiced, sales people.

The reality of software test technology is that perhaps 1% of the applications fit the norm of process success that can be illustrated with a prepared sales demo. The other 99% require some kind of local `testware' programming in order to succeed.

One can understand why companies would avoid evaluations: they know that their products are too limited in scope, and/or they don't have the technical staff needed to support you anyway.

This may be due to the fact that software testing is an immature market, and thus is subject to a lot of sales hype.

An example of how confusing things are is how some vendors toss terms around - terms that every vendor uses differently -- to their own advantage.

The bottom line is this: try it before you buy it. The only software testing solution that works for you is the one that you really see working for you. No single-function test tool solution is going to completely solve anyone's testing problem, even as slick sales and marketing will have you believe. Having actually seen some of these pitches, I have to admit that they really are clever at sweeping the hard parts of the software test problem under the proverbial rug! Caution is advised.

Software Research has been working on software testing problems and their solutions for over fifteen years. I, myself, have been involved in the software testing field since the early 1970s! Through my experiences, one thing about the software testing field is unquestionably true: the problem of achieving good quality software, using test tools and every other imaginable possible solution, was, is, and will remain both a technically subtle and deep problem for which there is no quick fix.

The CASE "silver bullet" that never materialized is not here for CAST either. Moreover, software testing is a multi-faceted problem and requires multi-faceted solutions.

I believe SR's TestWorks tool suite, made up of three bundles of tools (STW/Regression, STW/Coverage, and STW/Advisor), is a strong base for general software testing solutions. On the other hand, I also know no single product or even TestWorks's entire collection of 12 separate tools, is the total test solution. For this reason, SR will continue to expand its product line and to work with others in partnership, to add to its solution. In a recent partnership with Interactive Development Environments and subsequently with its successor AONIX, for example, TestWorks was integrated with the StP products to create an end-to-end automated specification-based and software test solution.

Over time, the products that provide real solutions and deliver genuine value will prevail. You have my assurance: SR will be here to help you when other short-term single point solutions fail to measure up when the going gets tough.

- Dr. Edward Miller
President, Software Research, Inc.
Reprinted from Testing Techniques Newsletter, Spring 1994