Tuesday, August 05, 2008

Random Test Automation Framework Attributes

Usually, at least three levels of abstraction.
L1 - protocol, suite
L2 - case, processing, program, scenario
L3 - assertion, item, step, verification point

NOTE: some sites use a L0 concept - a "suite of suites".

Usually, setup (pre-conditions or constraints) and cleanup (post-conditions or return-to-base-state) are able to be specified for L1 and L2 things.

Timeouts should usually be enforced by an automation framework rather than hard-coded throughout the test code. There should be a clear separation between a test framework and the test suites which plug into the framework.

Consider using something like JSON for data-driven things as opposed to XML (to heavy-weight), YAML (just a little over-engineered), INI (too weak), and the sourcing in of language specific syntax. As of this writing, JSON seems like the best overall technology for test input(s).