Developing a framework for evaluating benefits of Information Technology Projects in local communities

Abstract

Governments of the developing world - including South Africa - have a strong commitment and resolve to accelerate the rollout of Information and Communication Technology (ICT) to the benefit of communities. Both government and the private sector are delivering a number of ICT-based interventions in South Africa. These are in an effort to meet the country’s commitments to targets agreed to at the World Summit on Information Society (WSIS) in 2003. It has been found that ICT for Development (ICT4D) initiatives have not been as sustainable as expected. Hard evidence about the economic and social impact from the development of ICT4D is also lacking. Similarly, evidence of evaluation of benefits of these ICT4D projects worldwide is scarce and anecdotal and the knowledge of benefits or impacts from ICT initiatives is still incomplete. One possible reason for this is that there are no established evaluation frameworks. Nor are there clear guidelines as to how to identify suitable indicators or how to develop metrics to evaluate these indicators. Thus the question of how to conduct appropriate evaluation is an important one in the ICT4D landscape. This paper suggests a research design for the development of an evaluation framework and of guidelines for the implementation of the framework. Evaluation theories and frameworks currently used in the ICT4D discipline, which are rooted in multiple disciplines, are presented and their strengths and weaknesses in the application of developmental projects are discussed. A mixed method design within case studies of two telecentres in the Western Cape is proposed. The research will include a future longitudinal design within the two case studies, a survey conducted amongst stakeholders at the end of the data collection period, the implementation of the framework at a third telecentre and the development of guidelines for the implementation of the evaluation framework.