How a university saved $250K and weeks of time by automating feedback

Five years ago, Alfred Kobayashi walked into a bureaucratic armageddon when he started his job as Head of Open Innovation at a Peruvian university.

One big task: improving the process for professor evaluation at Universidad Peruana de Ciencias Aplicadas (UPC).

Alfred was met with a dizzying paper chase that involved thousands of paper questionnaires being handed out in classrooms and getting shoved into envelopes. These were then stuffed into mailboxes around campus before being collected and sent to third-party providers for analysis.

And that was just the half of it. Check out this breakdown of this toe-curling process, which was costing the university weeks of time and hundreds of thousands of dollars:

How to get in-person response rates with a digital survey?

As cumbersome and costly as this process was, student evaluation an essential part of UPC's rigorous quality control system. This also involves self assessment review, peer review, and 360 feedback.

The quality of education in places like Peru is highly questioned," said Alfred. "So we can't afford to lose an ounce of the quality we offer, especially as a university that's renowned for its innovation."

When Alfred was brought on board, UPC was in the process of getting accredited by the U.S. schools accreditation commission, WASC. This would give it an edge over other universities in the country and open up a world of new possibilities.

So, how could Alfred help maintain the same high standard of professor feedback while innovating on the old process to save time and money?

Of course, they could have just emailed all students with a link to a survey, but UPC had tried that years before and only got a completion rate of 30%.

As Alfred put it: "Normally the student will forget, or ignore the email, or even delete it—because they don't care.

"So the challenge was to create something digital that could reach the same level of completion as the manual process, which was 95%. Nobody wants to fill in anything online. People really hate filling in surveys," he said.

One of the main reasons why the completion rate was so high with the manual process was that students took it in the classroom during regular class time. So Alfred reasoned that if it was going to be digital, it would still have to be done within classroom hours.

"We knew it wouldn't work if they got the survey after. It would have to be a programmed approach to sending students the surveys at the moment when they're in class."

How did they solve it?

Alfred's team developed a platform in Python that could use the Typeform Responses API and Webhooks to replace the old manual process. By modifying the existing web app portal where students see their timetables, they were able to push a typeform survey at the class time. This happens with all classes during one week near the end of the semester. Here's how it looks to the student when they open the web app:

"So let's say I'm taking my history lesson in the morning. When I log on, a button appears saying ‘professor evaluation survey.’ When I click it a typeform appears, which I complete and submit. And that's it," said Alfred.

Here's an example of how one of UPC's typeform surveys looks (we've translated from the original Spanish version):

Green means go

The system, in theory, makes form-filling child's play. But, if you've ever met a student—or been one—you'll know they have a tendency to get distracted. So how do the back office team make sure all the typeforms are getting filled in? The secret is a "traffic light" system in the back-end that ensures that the completion rate remains as high as it was in the old process.

"We developed a dashboard in-house that controls all of the classrooms. It works like a switchboard with different cells representing different classrooms."

🟡 After the first 10 minutes of a class, the cell of that classroom appears in yellow on the dashboard. If you double click it you can see all the details of the teacher, the course, and the students.

🔴 If no typeforms are submitted within 15 minutes, the classroom cell turns red. If this happens, someone from the back office team is sent to the classroom to remind the teacher to give students time to complete their surveys.

🟢 When a student completes a typeform survey on their phone or laptop, the cell changes to green in the dashboard. If the class finishes and the minimum survey completion rate (92%) hasn't been reached, the system reprograms the survey for the next class time. This normally happens overnight. "Using this system we've been able to achieve the same completion rate as the manual system, but quicker," said Alfred.

Anonymity breeds honesty

Besides the impressive completion rates and processing time benefits, the quality of the survey responses has been maintained. A big part of this is by preserving anonymity—or at least the feeling of anonymity—of the forms.

UPC uses Hidden Fields in their typeforms to recognize who they’re sending surveys to. Not only does this ensure that a student can't take the same survey more than once, it also helps to foster the all important feeling of anonymity.

"No other university has a system like ours. Other institutions send a survey in a mass email where the student has to enter their name, the name of their course, the professor's name, etc. The student then worries about what they write because it's obvious they aren't anonymous. In our system, thanks to Hidden Fields, the teacher doesn't see who has completed the surveys".

Reporting, evaluating, rewarding

So what happens to all the feedback that UPC collects from the students, as well as from the typeforms they use for peer feedback, 360 feedback, and self evaluation?

"The results come to the teacher via the intranet," said Alfred. "They receive a report where they can see the questions and all of the statistics as averages. They can view by the courses they teach, as well as see an overall score."

The evaluation process is so important for UPC that it can make a difference between a professor retaining their job or being let go.

"If a teacher receives an average score of less than 7 out of 10 for two cycles then it's 'adios,'" explained Alfred.

The average score for professors at UPC is around 8, but there are some exceptional teachers who regularly score above 9.5.

"We reward these teachers," said Alfred. "We categorize professors based on their scores—we have gold professors, silver professors, then the rest. The 'golds' get access to free materials, bigger lockers, discounts, and things like that."

Feeling the benefits

Although they still have a team behind the scenes who tag all the open text field questions from the typeforms, the process is much leaner overall than before.

It's just one of several huge benefits UPC has seen since creating the Typeform-powered feedback system—the most notable being their WASC accreditation:

"When we achieved WASC accreditation they recognized the pillar of academic evaluation, said Alfred." They appreciated the effort that went into this system of maintaining the level of completion and validation of the surveys."

Besides enabling them to get WASC accredited, here's a summary of the key benefits for UPC that Alfred said they've seen since switching to the Typeform solution:

• Saved at least $250,000 per year in costs.

• Retained 95% survey completion rate of manual process.

• Data processed quicker—it used to take 2 months for a teacher to receive the feedback, now just 1 week.

• Mitigated the risk involved in switching from paper to digital survey, by maintaining the same high-quality teacher feedback.

• Students and teachers are very satisfied with completing typeforms.

• The evaluation process still works flawlessly, even when students are not present physically.

What next?

Inspired by what Alfred and his team at UPC has achieved? Here's some more stuff that can help make your life easier, whether you run programs in academia or online:

• Get started with our education and online learning templates, which cover everything from enrolling students to getting valuable information to improve the quality of your courses.

• Read more about our APIs to discover how you can embed typeform into your institution's existing systems.

• Find out how coding bootcamp is using Typeform and other tools to handle student intake.

• Got a story to tell? We'd love to hear about how you're using Typeform to get your jobs done. Tell us about it here and it could be you we write about next!