Health Research Board

The Proof of Concept phase of WISDOM will be comprehensively evaluated before the system is more widely implemented throughout the country. The evaluation of the Proof of Concept phase began in January 2008 with a review of evaluation literature, with a specific focus on the evaluation of information systems, user-focused evaluation and evaluation of training.

One of the key messages from the research was that, for new information systems to be successful, a broad view of the system needs to be considered. This means that in addition to assessing the technological aspect of the system, it is essential also to look at the processes supporting the system (i.e. operating procedures, governing policies, etc.) and the context within which the new system is being implemented (i.e. the practicalities of the organisation and the work environment). In light of this, the evaluation of WISDOM will adopt the holistic approach of the HOT-fit model (Yusof et al, 2006[i]) by examining the Human, Organisational and Technological aspects of the system.

The review of research further identified a framework developed in Canada that has been adopted as an appropriate approach for the WISDOM Proof of Concept evaluation. The evaluation framework by the Newfoundland and Labrador Center for Health Informatics (NLCHI) was the outcome of a review of information systems in Canada, and provides a comprehensive approach to undertaking a holistic evaluation of the technology within the context that it is designed to work. This involves evaluating for the purposes of accountability; to enhance the performance of WISDOM; and to develop knowledge for future developments. The NLCHI also provides a number of sample evaluation questions that are asked throughout the life cycle of the Proof of Concept. These sample questions are being adapted and developed to suit the specific needs and context of WISDOM, yet will retain the systematic structure and approach designed by the NLCHI.

The outline evaluation plan for the Proof of Concept has been presented to both the Interim National Project Board, the Donegal Project Board and the Donegal Implementation Group, and agreement has been reached on the approach and broad parameters. Pre-implementation data collection and preliminary work for the evaluation will begin in June 2008, and will continue throughout the duration of the Proof of Concept, with a final evaluation report delivered to the National Project Board in June 2010.
----------------------------------------------------------------------------

[i] Yusof, M.M.; Paul, R.J.; and Stergioulas, L.K. (2006) Towards a Framework for Health Information Systems Evaluation. Proceedings of the 39th Annual Hawaii International Conference on System Sciences, 2006.