Ella Chan
It’s around 8 p.m. and you decide to start your readings for tomorrow. You log into Canvas and scan tomorrow’s text: something about Descartes, again. You give up after about five minutes and shut down the browser. Who cares? Your prof won’t know.
But Canvas will.
UBC’s new learning management system is tracking your data. Everything from how much time you spend on your readings to your quiz answers to how your mouse moves is being recorded.
It’s called learning analytics, a data-based breakdown of how you study and learn.
Proponents think it could revolutionize teaching. But some worry it’s a slippery slope towards an invasion of privacy.
‘Double-edged sword’
UBC has been gathering learning data from students for years. But it’s only recently been able to use it.
Dr. Simon Bates, UBC’s associate provost for teaching and learning, sees learning analytics as part of a new generation of tools to help professors support more students with less individual effort.
“[If] you think about the way that data is used to support positions or experiences or interactions that people have, there’s a huge amount of data that can be used,” he said.
“The flipside is [that] it can be very scary.”
Since 2017, Bates has helped direct the Learning Analytics Project Steering Committee, a group of UBC professors, administrators and student representatives trying to figure out how exactly learning analytics will work at the university. So far, they’ve looked at over 30 prospective learning analytics projects and arranging a network of “data stewards” to make sure that student data is used ethically.
The goal, eventually, is that professors will be able to use data insights from Canvas and other programs to help students succeed in their courses — and make sure it’s done without giving professors access to data students don’t want them to see.
“It’s a double-edged sword,” Bates said. “There’s tremendous potential, but there’s also a need ... to tread carefully and appropriately.”
UBC has been collecting student data for some time. In 2013, the school paid $151,870 to Blackboard Connect — the learning management system that preceded Canvas — for analytical and scientific software. It also signed a contract agreeing to pay Blackboard over $100,000 for each year following that to use outcomes analysis and learning analytics software.
But as many students know too well, Connect was not an ideal learning platform. As for its learning analytics services, Bates says the school “never really got it to work.”
Now on Canvas, Bates says he’s already been able to use some of the system’s “baked-in” features to assist students in his own classroom.
A physics professor, Bates teaches a 100-year level class with a range of skill levels. He says he can use test scores from Canvas to automate emails to students, with pointers on the questions they got wrong.
It’s not a replacement for a good teacher — but Bates says it frees up a lot of time for him to directly engage with his students.
“It’s not a magic bullet,” he said. “You can’t say, ‘I have a 94 per cent … pass rate for my course, and it’s all down to the use of this tool.’ It’s not, but it’s one of the sort of combination of things that I think support students learning in the course.”
['auto'] Ella Chan
A short case
While UBC has been building its learning analytics strategy, students have been left out of the loop — and some have actively tried to resist it.
Bryan Short, for example, tried to opt out.
The second-year master’s of journalism student was curious about what UBC knew about him. So he filed freedom of information requests asking to see all the records the school had collected.
Originally, UBC refused to give them to him. Short had to appeal to the Office of the Information and Privacy Commissioner (OIPC), an independent arbitrator, to get his own data.
When the file did arrive, it contained more information than he had ever imagined.
“It was something like 400,301,000 individual data points about me,” said Short.
The information that was gathered on Short went well beyond test scores and basic analytics. It included details about how he used his mouse, how he interacted with the learning system and when he did it.
“There’s really all these ways to get metadata out of Connect. But it’s not just metadata — professors could pull a lot of personal information from students off of Connect,” said fifth-year engineering student and Board of Governors member Jakob Gattinger, who sat briefly on the learning analytics steering committee in 2017 when he was interim AMS VP Academic and University Affairs.
“They were able to see who had logged on, who had looked at what — and this was all on the system.”
Bates says that even though data has been collected on students, instructors don’t currently have access to anything beyond what they would normally have in the course — meaning those data points remain private.
“There are very strict controls over who has access to that data,” he said.
Short was more concerned that the data had been collected in the first place.
“They weren’t informing students about any of this — anything being collected,” he said.
UBC’s legal basis for collecting the data is a waiver students sign when they create a campus-wide login, which happens before they’re even accepted.
“If you didn’t look for it, you wouldn’t know that you’d actually signed it,” said Gattinger. “And when you go and apply to the university, the last thing on your mind is, ‘Wow, the [learning management system] might take my personal information.’”
Brad Weldon, the OIPC’s director of policy, said ‘secondary uses’ of personal information — like learning analytics — should always be the subject of scrutiny.
“We’re concerned always that that threshold is being respected, that they’re not collecting more information than is necessary,” said Weldon.
Short still decided he would opt out of Canvas. But he quickly found that there was no alternative for students who didn’t want to use the service.
“It was really difficult for me to complete certain courses, and instructors had to make work-arounds to let me succeed in those courses,” said Short.
Weldon says because students who attend post-secondary do so voluntarily, the school can decide which programs are necessary for them to use.
But it didn’t matter. Even though Short tried, learning without Canvas at UBC was basically impossible. He had no choice but to use the service and give up his data.
“It put the onus on the instructor, it put the onus on me and there was really no alternative there,” he said.
Changing learning
Data from learning analytics has a lot of potential to help students. But if used incorrectly, it could just as easily hurt them.
Say you’re a student in a study group: you share PDFs of readings with your friends and meet to discuss them each week before class.
You might be doing extremely well, but Canvas would measure you as having spent no time on readings. Contrarily, you could leave Canvas open in a browser page all week and never look at it, but it would note you had spent hours and hours online.
It’s a limitation that Bates and the committee are actively weighing.
“Even if you can get solid inferences from data, data never tells the whole story,” said Bates.
“You have to be very careful how you interpret the digital breadcrumb trail some of these observations leave behind.”
Professors share some of those worries. Dr. Juliet O’Brien is a lecturer of French at UBC who’s enthused by the potential learning analytics has to help students. But she points out most faculty aren’t trained data stewards and don’t know how to handle personal information.
“None of us have enough pre-training beforehand on how to read and use this data responsibly, and how this data should be available to our concerns,” said O’Brien.
Above all Bates stressed that the field is constantly evolving, which means administrators have to constantly adjust course to make sure they’re balancing privacy and access.
“The technology tools that we have available now — it’s going to be another set of tools in 18 months’ time, and another set of tools in three years’ time,” he said. “Every time you think you have one platform, someone comes along with another platform.”
The question of data
The biggest concern with learning analytics isn’t just that data is being collected — it’s who gets to see it.
Under the Freedom of Information and Protection of Privacy Act (FIPPA) section 30(1), public organizations can’t store personal data anywhere outside Canada, which has prevented UBC from using a huge range of tools open to other universities. A professor using Google Drive or Facebook in a class, for example, is technically illegal even though most students use those services privately.
In 2016, UBC’s Privacy Counsel Paul Hanco*ck lobbied the provincial legislature on behalf of five BC post-secondaries to revise FIPPA.
“... no section of the FIPPA has caused greater challenges for the post-secondary sector in B.C. than section 30(1),” said Hanco*ck. “It erodes our competitiveness. It’s preventing us from using world-class tools that other universities use in other jurisdictions. And it’s adding costs and administrative complexity.”
BC is essentially unique in the developed world for how strict its cross-border information laws are — which means that many professors routinely ignore the restrictions.
“If you look at some of these subscriptions that profs have their students get, that sort of stuff is definitely not okay,” said Gattinger.
In the end, UBC went with Canvas, a cloud-based service operated by Instructure, an American tech company based in Salt Lake City. Instructure stores its data at Amazon servers in Quebec, which means it is compliant with BC law.
But there’s a catch: Canvas has third-party agreements with various service providers, like Amazon, that allows it to access the data for the purposes of repairing the system if it needs repair.
“If you have a warning light on the dashboard, that tells you something’s wrong, but you don’t always know what,” said Bates, who adds that this practice is compliant with FIPPA because it doesn’t retain any information taken during repairs.
“Sometimes, you’ve just gotta let people under the hood, as it were.”
But there’s technically little stopping Instructure or Amazon from using the de-aggregated data to improve their own services — and that’s what has Short worried.
“This is stuff that’s private personal information that you’re sharing in the context of your education of your university, but this information is now accessible to a company in the United States,” argued Short.
And as Weldon notes, it’s virtually impossible to make data completely anonymous.
In a statement to The Ubyssey, Instructure's VP Security Matt Hillary said the company does use analytics internally to improve the services it offers, but stressed the data isn’t identifiable.
“When Instructure does analytics internally for the purpose of improving Canvas, personally identifiable information is removed from datasets prior to performing analytics,” Hillary wrote.
“It is increasingly more difficult to consider information to be de-identified such that it cannot be re-identified. We’re almost at the point where we agree that information cannot be de-identified to the point that FIPPA would not apply to it,” he added.
“If you were able to aggregate the information to the point where we would agree it is de-identified, it probably wouldn’t be useful for analytics.”
Bates is quick to point out that learning analytics is a new technology and that UBC has been extremely careful to protect student privacy — and like O’Brien, he’s optimistic about the insights data could give him as a teacher.
“You hear these stories about learning analytics that say, ‘Well, we can correlate what you had for dinner last night with your success on a midterm,’” he said.
“Maybe you can and maybe you can’t, but all the data I have is the same data I have access to already as an instructor. I’m just able to interrogate it in a different way.”
—With files from Alex Nguyen
Update, March 27 9:08 p.m.: This article previously stated that Dr. O'Brien is an associate professor. She is in fact a lecturer.
