Mystery shopping in IT support: how to audit internal service quality

Natalia Totska, founder of the CustomerVista Advising agency, and Olha Oleksiuk, customer experience manager at Modus X, describe how they ran a mystery shopping project in IT support. Modus X grew out of the IT department of the DTEK group and supports the whole group: about 20,000 requests a month from 12,000 employees. Support has a first line that receives and classifies requests and solves simple cases, and a second line made up of six departments, including cyber security and infrastructure. The company already tracked CSI, CSAT and NPS, but the team wanted to see how a customer moves through specific touchpoints.

The main obstacle: support works with internal systems, access portals and two-factor authentication. Someone outside the corporate domain cannot open a real request, so external mystery shoppers were ruled out. Employees of the group became the mystery shoppers instead. The team analysed last year's requests, took the three most frequent topics in each department, split the scenarios between first and second line, included the chatbot, and set channel quotas in proportion to past statistics. The total sample was 100 questionnaires across 15 scenarios. Because requests had to be genuine, nobody could be assigned a scenario, so the quotas were monitored by hand every day and full scenarios were closed.

Participants were invited through a group-wide mailing and through the communications staff of each company. About 200 people signed up: the speakers explain that 100 questionnaires need twice as many participants, since not everyone will have a real need during a two-week fieldwork window. The instructions were simple: do not invent cases, contact support only with a real problem, and never tell anyone in support that you are taking part. Before the main stage they ran a three-day pilot with Modus X employees only. The questionnaires looked fine, but Olha called every participant and collected remarks that led to changes in question wording, technical settings and instructions.

The speakers also cover communication with the support team. Engineers were told about the launch and its purpose, and that this was not an individual assessment or a control exercise; no names were recorded in the questionnaires. Scenarios, timing and participants were kept confidential. Participants were thanked with a business book of their choice and a thank-you card, shipped across Ukraine and to Europe. Engagement was high from the first hour, and service levels rose further during the project, because any request could be a mystery one.

The results did not contradict the regular satisfaction survey; they confirmed and deepened it. One example: the access management portal did not stand out as a problem in the general survey, but in the mystery shopping breakdown by department customers clearly commented on it, and an improvement plan is now being prepared. Most findings concerned processes and digital channels rather than staff behaviour, so solutions were worked out together with the teams at a workshop that produced an improvement plan for 2026. The speakers also concluded that a separate unified book of standards was not needed.

In the Q&A Natalia explains why a general survey is not enough: even an average score of 8.7 out of 10 can hide 10% of complaints, and broad criteria do not cover specific scenarios. Olha adds that auditing recorded conversations misses personal chats and in-person help at someone's desk. The fieldwork itself took two to three weeks, the next wave is planned for the fourth quarter, and volunteers are already on a waiting list.