Tag Archives: Learning analytics

Learning Analytics – who is watching the watchers?

We leave footprints where ever we go online.
We leave footprints where ever we go online.

The digital footprint of our students/users gives a better picture of how people are using the systems and the content held in those systems.
Internally for the government department that I work for this means that we are able to see how well content presented in mandatory training is put into practice. Managers can access how long team members have been working through content prior to attempting the tests, which could have impacted on the scores that team members received.
My concern about this is how soon will the length of time someone has spent learning within mandatory course work be used as a measure in performance development meetings by a crackpot manager who does not have the capacity or capability to understand learning styles and the simple point that people learn at many different rates.
Upon saying this I do not feel that learning analytics are bad, but they do need to be used with caution. It would also help if organisations developed strategies around learning analytics to be able to use them in the best to support achieving improved outcomes for students and clients. The Charles Sturt University Learning Analytics Code of Practice  is a good example of documents that bound an organisation to how this valuable data set will be used.
Another exceptional use is an example from the Western Australian TAFE sector. Recently I was discussing learning analytics internally with our ICT department, especially the LMS that we supply the WA TAFEs and how users are enrolled into online courses. From this discussion a new building block was created by out ICT team which I was discussing with a client from a TAFE. He had used this new building block (as well as other reporting functionality) to view how staff were using the various tools within the LMS. The client discovered that staff seemed to be enrolling students individually more than by class rolls. There could be a wide number of factors including rolling enrolments where a cohort could have new people added adhoc over the course of the study period. But what this has highlighted for the TAFE team is that they can tailor training for staff better as they are able to watch the watches and support them to become better online trainers.

Digital users be aware.
Digital users be aware.

This does beg the question, who exactly are watching the watchers?
Recently in the agency that I work one of our mandatory courses grade books had been tampered with by a member of HR staff. Corporate Leadership team requested an independent review by a team external to HR who knew how to interrogate the system logs to determine who had access and tampered with the course grade book (as it is a mandatory regulatory course that all staff must complete and pass to maintain employment). I was able to track back through the logs that the HR team were unaware of, locate how the issue came about and reported back to Corporate Executive with recommendations regarding restrictions to the higher level access functions to ensure that this issue did not happen again as well as rolling the course back to the last backup date as no members of staff had been employed in the period that was impacted, which removed the problem. For the future I noted in the system the issue and why the reset had happened so there was a reason to my wiping a month of course logs. As part of my final report I also suggested further training of the HR team was required which has occurred.
In this instance the watchers were completely unaware that they were being watched and monitored until after the fact. I personally feel that this is not the way morally that we should be using this technology. It should be above board and everyone aware that they can be tracked, no matter what.

Simply a Code of Conduct policy around the use learner analytics is so very important for any organisation.

References

Charles Sturt University (2015). Code of Practice. Retrieved from https://www.csu.edu.au/__data/assets/pdf_file/0007/2160484/2016_CSU_LearningAnalyticsCodePractice.pdf

Welsh S. (2016). INF537, Colloquium 1, Learning Analytics [PowerPoint Slides and Connect recording]. Retrieved from https://connect.csu.edu.au/p65jlka06d6/