DSL for workflow testing

Constant changes, reduced cycle times, lean structure and of course limited resources are norm of today's software development life cycle. Testing of software applications has taken a hit in modern age that customers no longer insist on separate testing teams; rather rely on unit testing and regression testing by developers.

Automation testing came to rescue but met with limited success. One of the primary reasons was automated testing focused on application testing rather than enterprise wide testing. Also, automation testing required a separate dedicated team to develop scripts. For the most part, testers have been testers, not programmers, quoted by Carl J. Nagle comes to my mind, which literally means a dedicated team of programmers are required to develop automation scripts. Needless to mention about increasing investment on resources and the upfront cost involved in procuring the automation tool itself.

BPM (workflow) testing is a nightmare. Especially, if the process holds lot of human tasks, the effort required for regression testing manifolds with time and human errors from testers cannot be ruled out completely. I've thought about an inexpensive alternative with the help of Domain Specific Languages for workflow testing. Let me illustrate with an example,

Typically a QA tests the following in a BPM Application Portal.

* Logins into Portal
* Searches for tasks based on some artifact keys.
* Saves/Completes some assigned tasks.
* Creates process instances.
* Waits for some tasks.

These are the few of the many items that a tester or a Business Analyst does, not necessarily in the same order except for the first item.

Now the Domain Specific Language (DSL) which is to be built abstracts only high level tasks the user performs and gives the ability to the user to write his own scenario for automated testing.

For example, if the user wants to search a task, create an artifact like Payroll and then complete the task, he would simply write as,

login_into_portal_as_user(scott)
tasks[] = search_for_tasks(‘Assigned’, ‘Available’)
iterate all tasks
if task is ‘Payroll’
create_paystub (current_date, pay_amount, employee_details)
complete.task
end

The above DSL code is very granular, but with further analysis and level of testing required we can build the DSL functions at coarse grained level, which would limit and simplify the number of lines a tester/user have to use to build scenarios.

One of the objectives for building custom DSL is that anybody from non developer community can write test case scenarios making use of DSL. The above mentioned syntaxes are just examples and can be made much simpler or close to readable English.

DSL's can definitely complement performance testing tools as well. The DSL testing code need not be in the above format; even as XML would do and having DSL on top of any other workflow testing tool means that the services alone will be tested for functionality.

DSL for testing at a BPM Portal level would better comprehend what an actual tester does. Web flow / UI Level automated testing comes with some disadvantages too, due to the fact that DSL objects are highly dependent on HTML source.

DSL’s are evolutionary and the functionality that have to be tested using DSL should be built brick by brick. Dynamic scripting languages are good candidates for building DSL, reason being faster development time opposed to using a high level language. Ruby + ‘Watir’ (pronounced as water and is a Ruby gem) are a good choice for building a DSL with automated UI level test scripts.

DSL for workflow testing requires constant maintenance with BPM application itself getting modified. DSL also requires a skilled programmer to develop and maintain the DSL itself. Still, with the above disadvantages, developing DSL would be much cheaper and viable option for BPM initiatives.