In my previous post ICT Integration: Part One – Defining, I defined what ICT integration looks like in my context. In this post I will suggest several different assessment and monitoring approaches that can be used to measure and assess the impact of this integration.

How do we assess the degree to which we are integrating ICT into the classrooms?

 Current monitoring, and evaluation processes within the project do, to some extent assess the degree to which technology is being integrated into the classroom. This occurs through the analysis of quantitative data which illustrates the number of hours students use the devices. While this provides a clear assessment of the time spent using the integrated technology, it provides little indication of the impact of the integration itself.

 

So how can we measure the impact?

Collaboratively as a group, the Education Specialists have tried to tackle this issue previously. Through meetings and copious amounts of WhatsApp messaging we have identified some strategies that could work within our context to measure the impact of the technology integration.

Strategy One – Triangulate data to measure learning growth

Strategy One – Triangulating data to measure learning growth

  • What? Triangulating data on the time spent using the device with previous and current literacy and numeracy learning achievement data for each child.
  • Why? If the technology integration is proving effective than the students who spend more time on the devices should be making greater gains in numeracy and literacy, compared to those students who use the devices for a lesser amount of time.

Strategy Two – Leverage and analyze the data collected from the app to measure task fluency and understanding

  • What? Analyze the time spent on a task, with the learner’s number of attempts to complete the task before answering correctly.
  • Why? The tasks, which are structured into units are designed using rote-learning principles, meaning that as the tasks progress in the unit they are simply being repeated with no increase in difficulty. As a result, as students progress through the unit they should be completing tasks in shorter amounts of time with higher levels of accuracy. Thus demonstrating learning growth through increased understanding and fluency.

Strategy Three – Leverage and analyze the data collected from the app to measure basic digital literacy skills of learners

  • What? Compare the number of times a learners take to log-on to their profile and begin working.
  • Why? This comparison would reveal any increases (or decreases) in digital literacy skills, being the ability to locate and use the app. Ideally the logon time should decrease the more students use the devices.

Strategy Four – Teacher Questionnaire to measure technical and pedagogical ICT knowledge.

  • What? Conduct qualitative and quantitative teacher questionnaires.
  • Why? As part of the ongoing training facilitated by Education Specialist, each session should allocate time for a questionnaire that aims to measure the technical and pedagogical ICT knowledge of teachers and school leaders. The questionnaire would be modelled off the UNESCO Teacher ICT Competency Framework (UNESCO, 2011) with a particular focus on the following two areas
    • Pedagogy – ICT enhanced learning and
    • Teacher Professional Learning – Digital Literacy.

Strategy Five – Community consultations to measure awareness and values of ICT integration in education

  • What? Hold bi-annual community consultations (mothers’ groups, parent associations ect) to discuss benefits of ICTs, address concerns and collect qualitative data on their awareness and values of ICT integration in schools.
  • Why? In Malawi the prominence of ICT in everyday life is minimal, as such it is believed that the impacts of ICT integration is not just felt within the school but also in communities, particularly on community perception towards the efficacy of ICT.

References:

United Nations Educational, Scientific and Cultural Organization (UNESCO). (2011). UNESCO ICT competency framework for teachers. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000213475