Scope for usability tests in IS development

Despite being a common, established concept in wide usage, usability tests can vary greatly in goals, techniques and results. A usability test purchased and performed for a specific software product, may result in either minor user interface improvements or radical U-turns in the development. Such variation has been discussed as a problem of the scientific reliability and validity of the testing method. In practice it is more important what 'kind of data' one can expect of the selected method than whether it is reliably always the same data. This expectation of information content or 'scope' is of importance for evaluators, who select and conduct usability tests for a specific purpose. However, the scope is not explicitly stated or even discussed: Too often the premise is that, because a usability test involves users, it brings the (necessary) user-centeredness to the design i.e. takes socio-technical fundamentals as inherently given. Through a literature review of testing practices and analytical considerations, we search for the scope of a usability test, which could deliberately approach the socio-technical tradition and equally develop both the system and the user organization. A case example represents a possible realization of the extended scope of usability test.