A new performance paradigm: watermelon effect in HR services

Abstract

In the past 10 years at least, not only that human resource specialists have been challenged to upskill and re-skill themselves to stay relevant in a world dominated by technology, but they are in the driver's seat for selecting the right technology for employees and leadership, for managing large scale implementations and integrations, managing vendors, with the objective to keep the workforce engaged. Technology development disrupted traditional thinking, changed consumer behavior, and led to what we could call systemic malfunctions - the rapid development of technology gave no time to human resource specialists to acquire expertise in data analytics, information technology, user behavior, etc. - which translated into watermelon effect: metrics look good, but reality feels different. This study aims to add one more perspective in evaluating human resource technology performance, showing that we are measuring mostly service efficiency instead of measuring employee and business impact and outcomes. The methodology used in this study is Action-Research, mainly because is mostly aligned with Agile philosophy and business operating models. The existent literature is rich in proposing new ways to keep technology users engaged, and delighted, however, is not provide practical and relevant guidelines on how to measure their perception over the holistic experience mediated by technology. Outdated key performance indicators are no longer capable of driving further innovation, setting the stage for failure in keeping employees engaged. Furthermore, we believe that now more than ever before it 's time for consultants to play the role of service aggregators, to bring together technology, processes, and service providers, to unify the experience of employees.

Headnote

Abstract

In the past 10 years at least, not only that human resource specialists have been challenged to upskill and re-skill themselves to stay relevant in a world dominated by technology, but they are in the driver's seat for selecting the right technology for employees and leadership, for managing large scale implementations and integrations, managing vendors, with the objective to keep the workforce engaged. Technology development disrupted traditional thinking, changed consumer behavior, and led to what we could call systemic malfunctions - the rapid development of technology gave no time to human resource specialists to acquire expertise in data analytics, information technology, user behavior, etc. - which translated into watermelon effect: metrics look good, but reality feels different. This study aims to add one more perspective in evaluating human resource technology performance, showing that we are measuring mostly service efficiency instead of measuring employee and business impact and outcomes. The methodology used in this study is Action-Research, mainly because is mostly aligned with Agile philosophy and business operating models. The existent literature is rich in proposing new ways to keep technology users engaged, and delighted, however, is not provide practical and relevant guidelines on how to measure their perception over the holistic experience mediated by technology. Outdated key performance indicators are no longer capable of driving further innovation, setting the stage for failure in keeping employees engaged. Furthermore, we believe that now more than ever before it 's time for consultants to play the role ofservice aggregators, to bring together technology, processes, and service providers, to unify the experience of employees.

Keywords', employee experience, technology, performance, KPI, innovation

Introduction

As the Human Resource (HR) function of organizations started to play a crucial role in the era of digitization, the technology used to mediate the interactions between HR Specialists and Employees became the focus of the leadership. Creating a virtual space to foster collaboration and engagement, organizations started to investigate plugging more employee tasks into the digital space, life at work being now handled by the systems - like annual leave requests, references, time and attendance, bonuses, all one can think of. Given the fast-paced development of technology, and the discovery of its use, companies implement systems in a more intuitive way rather than adopting a more academic approach, based on research and proven results; part of it because of the time pressure and part of it because of the lack of specific bibliography. Conducting studies to analyze the effects of technology adoption on employee perception is imperative if companies want to meet their objectives. Moreover, the pandemic brought irreversible changes, as digital transformation became the priority of all companies, which means the perception of technology (and its perverse effects) needs to be looked at too. This study is specifically addressing the need to help understand how to measure the efficiency of the technology considering new expectations influenced by the use of technology itself (watermelon).

Companies developed metrics to measure the quality of their services, and the efficiency of the technology, but most often these are not applied to their internal functions, such as human resources. Moreover, the companies adopting a modern model, look at metrics that picture the employee engagement level which is not a result of HR operations and its processes. Only two decades ago large companies started to outsource their internal functions to specialized service providers and this has been the beginning of performance measurement. It is this context that creates the need to understand the metrics associated with the service and the effects of technology use.

In this fast-paced changing landscape, no wonder that HR faces multiple times the watermelon effect - a concept associated with key performance indicators showing positive quality of service (green color), although the perception of quality of service is poor (red color). The concept has been borrowed from safety experts, who have been highlighting that in their experience, they encountered situations when surface through parameters were reflecting a good quality of processes, whereas reality on the field was exposing professionals to incidents that could have been avoided (ABB, 2017).

In HR services the concept is known especially when associated with outsourcing. Large companies engage multiple partners in an ecosystem, without necessarily looking at the end-to-end process, generating a watermelon effect only by contracting different partners to manage different areas of HR - payroll with one provider (mostly because there are few payroll service providers who have global coverage), employee benefits with another one, service center with a third one and so on. The reason behind this is cost efficiency and proof of expertise. Consultants, often engaged to look at the employee experience holistically, focus on the end-to-end experience too high level, making room for gaps that later on will lead inevitably to the watermelon effect.

Irina Butnaru, one of the two authors of this article, has spent have spent 11 years working as a leader in HR Services, implementing and improving constantly service models for organizations, based on technology and human interaction. She helped organizations shape service and technology adoption strategies, end to end processes, and workflows, constantly focusing on how to achieve a more rapid adoption, because, in the end, employees will have to utilize the technology to fulfill their responsibilities in relation to their employment.

Leading a global organization (line of business) providing HR services to international companies, she constantly looked for ways to improve the quality of services, and level of performance, to meet clients' expectations that have been constantly changing (due to technology context - changing user expectations).

There is no unique recipe for success, and it has been fascinating to witness how fast large organizations must adapt and adopt new technologies. In this fast-paced changing landscape, no wonder that she faced multiple times the watermelon effect. This study focuses on bringing a new dimension to the watermelon effect in business - technology use is triggering new needs that lead ultimately to the watermelon effect or on contrary, show the opposite (indicators are red but employee perception is good).

It has been often, especially between 2010 and 2020 that we have heard - you are not being proactive, you are not understanding our employees, you are not understanding the real impact, how can we improve the service beyond the contractual agreement, etc. HR service providers equip themselves to deliver the best services to their clients, meeting contractual expectations, especially knowing payroll has a significant impact on employees; however, it is often that they hear it still needs improvement.

On the one hand, service providers looked at the perception of their stakeholders - is it that they feel the service is not meeting expectations because they are dealing mostly with the issues and escalations? - but on the other hand, they looked at the performance of their service centers - Are they listening? Are they empathetic? Are they really solving the issue?

Usually, service providers make use of data analysis to get to the bottom of the issues, sometimes combined with qualitative interviews (mostly with detractors) - in business services root cause analysis is the usual methodology, to understand the problem and to find solutions that would translate into an action plan. Most of the issues tackled through root cause analysis cover timeliness and quality gaps, translated into adding resources into the service center and re-train the existing staff to remind them of procedures to be followed.

Still, the watermelon effect comes back regularly, hence, it's time to look at how performance is measured, and how the performance is tied to what end-user (employee) experiences, and it's time to start wondering if technology design doesn't have a more important role than anticipated when using HR systems. Especially when the business world faces a silent crisis of HR technology, HR specialists have to deal with data, systems, and integrations, in other areas of expertise than theirs.

It's time to question the need for a holistic set of key performance indicators, to measure both technology performance and service center performance, indicating the experience quality, and the outcomes generated by the vendor ecosystems.

There are numerous studies and a rich bibliography in the field of human resource management but there is a gap in addressing specific human resource service performance indicators, part of it because the function is measured holistically by employee engagement scores, part of it because service providers are reactive instead of being proactive by adopting a more academic approach.

1. Literature Review

As mentioned earlier, HR's key performance indicators address the business impact and/or outcomes like attrition rate, absenteeism, revenue per employee, engagement score, inclusion rating, diversity index, etc. One of Irina's business objectives over the years has been to standardize and leverage best practices across client services and it is a prerequisite to analyze the status-quo before standardization. Her empirical studies over the years, conducted on a set of 20 companies revealed a set of seven performance indicators that are being used by more than 10 companies, indicators that are largely used by most of the companies in measuring their success in providing services, as shown in the below (Figure 1) will detail each KPI for alignment, given the numerous studies with slightly different views on the definitions and methodology.

One of the most common metrics used by the business world is the Customer Satisfaction Score - CSS - which is defined as "the overall happiness a customer feels when interacting with a company's products and services" (American Society for Quality, 2024). On a scale from 1 to 10, the user rates the interaction by responding to a question: How satisfied are you today with the service provided? Measurement scale: 1-6 Detractor; 7-8 Passive; 9-10 Promoter.

One observation would be that CSS it is measuring the perception of the overall experience or use of a product, without necessarily understanding the standards against which the service or product is measured. If we limit the analysis to services, the interaction starts with technology and a service representative might come in the process to pick up a query that could not be resolved with the help of technology only. Moreover, if the technology is provided by a partner of the ecosystem and the service center is operated by another partner, does it mean the end user will receive two surveys? One for the experience associated with the use of the technology, and one associated with the use of the service center services?

Net Promoter Score - NPS - "measures the loyalty of customers to a company" (Qualtrix, 2020). Mostly applicable to senior leaders, from a scale from 1 to 5 where 1 is very poor and 5 is very good, they are being asked to evaluate 3 areas: innovation, technology, and added value. NPS can also be measured with a single question survey (how likely would you recommend us to a friend?) and reported with a number from -100 to +100. A higher score is desirable.

This section will not be expanded in our study, since our objective is to identify the indicators related to the experience of HR Technology, and employees are not being asked if they would refer the HR service to a peer, but rather their loyalty is measured through the engagement surveys where they need to evaluate if they would recommend the company as employer. However, the quantitative score without qualitative data may be misleading, hence, it is highly recommended to use both in establishing the action plans.

First Response Time - FRT - is defined as the average time to provide a first response to the requestor, except for the acknowledgement of receiving the request. FRT is to be calculated as the average (sometimes the median) of the resolved tickets/cases within the established timeframe versus the total number of tickets/cases received.

More than twenty years ago, when automation was not so present, at least not widely, this indicator was extremely important to all service providers, indicating the time to provide a first response to a customer request. As the technology evolved, the systems have been used to provide an automated first response, setting expectations with requestor/end-user/customer and comforting them. In essence, this need may have been created by the unknown at that time - the uncertainty that the email or form filled in had been received by the service provider.

Average Handling Time - AHT - is defined as the average time spent by the service provider to handle a request - be it through a call, be it through a chat conversation, be it through an email - and was used to measure efficiency in providing the solution. This indicator has been crucial in forecasting resources. The indicator has been instrumental in leading the way to reduce or eliminate handoffs between service representatives, which was causing the long average time in some cases (Dixon, Toman, and Delisi, The Effortless Experience - Conquering the New Battleground for Customer Loyalty, 2013).

One observation would be that a good AHT doesn't necessarily mean a great experience for the end-user. The watermelon effect appears when service center representatives are measured by AHT and they rush the conversation or they rush the resolution, creating the space for making mistakes, and being under pressure. One way to constantly check the validity of AHT is in conjunction with other metrics - CSS, UR. AHT in itself is nothing more than the basis for forecasting resource needs. Also, the effect can be the way around too - too long AHT but great customer satisfaction.

Average Resolution Time - ART - is defined as the average time spent by a provider to work on a ticket/case including time in pending (with other providers/or clients). As organizations expand and new technologies and services arise, we cannot talk about single service providers in HR, in Finance, or Procurement. Most of organizations have a vendor ecosystem and ensure smooth delivery of services to their employees by coordinating the network.

Internet access, highly automated systems in our day-to-day personal life, and AI everywhere created the same expectancy from employer technology - real time action, real time resolution. However, when dealing with multiple stakeholders one cannot expect resolution in real time, this metric being the one with most instances of service complaints seen as watermelon effects - metrics showing expectations have been met by each service provider, but employees complaining about time to resolve their query.

Starting in 2017 the business world began to talk about experience indicators, a leverage from technology user experience indicator. We needed to shift to indicators that would reflect more what customers expect, not what service providers can offer. The same was applicable to employees, they were expecting the same experience from technology, be it at home, be it at work. Still, only 4 companies out of the 20, measure performance of the service using this KPI too.

Quality Assurance - QA - is measured through all or selected indicators: accuracy of data provided, correctness of the solution provided, professionalism of the representative, and ability to meet the customer needs.

The quality assurance scores reflect the competence of the service representatives, encompassing the training quality, compliance controls, personal skills, and management strategy, predicting the ability to upskill and provide future services, more complex, and more sophisticated. It is well known that one negative interaction has a 4 times higher impact on quality perception, hence the importance of this indicator (Dixon, Toman, and Delisi, The Effortless Experience - Conquering the New Battleground for Customer Loyalty, 2013). The service providers not only started in 2015/2016 to take the proactive approach, to align to the new attention span of 8 seconds but understood the weight of disappointment over delight and worked on equipping their staff with technology that would enable them for success (Swinscoe, 2016). Service providers who want to be successful and known as trusted partners invest in quality assurance functions, train resources, implement controls, identify points of failure, and close the loop with more training, up-skilling, reskilling, and understanding what causes the failures.

The more we automate, the more service representatives will handle more complex and new queries arising from consumer behavioral changes.

Customer Effort Score - CES - is one of the recently added metrics that evolved from the need to drive customer loyalty (Dixon, Freeman, and Toman, Stop Trying to Delight Your Customers, 2010).

On a scale from 1 to 3, the user rates the easiness of the interaction/use of technology by responding to a question: How easy was it for you to get the service today? where 1 is very difficult, 2 not difficult but not easy, and 3 is very easy. Another way of measuring customer effort score is by the number of touchpoints within a process.

Large organizations, which are the subject of the research, evolved to a high degree of automation, that would imply a high level of efficiency, implying among others, a low level of user effort. Technology, through the bots, should be sufficient to complete any employee related tasks, such as annual leave requests, balances, payroll inquiries, policy clarifications, references, etc. because the data is kept in the data lake, the bot is integrated with all employee data systems and should easily identify the solution. However, the employees' queries are becoming more and more complex, identifying new needs as society evolves. It takes time to understand new trends, find complex solutions to match most of the expectations, and feed the bots to be able to respond.

Why change systems as a result of measuring perception when technology cognition varies from user to user and efficiency standards vary too? This has been the trigger to leverage the agile methodology of implementing platforms and systems to the HR Technology space and apply the Action-Research method to enable analysis of behavior and change in the perception of technology users. Especially because in 2020, 100% of the service providers in all locations have been operating remotely, due to the Pandemic restrictions, making the virtual employee space more important than ever before.

Along with the development of technology, we witnessed changes in consumer behavior, and with this, we want to emphasize the importance of history and social context - it is specifically the development of technology that changed user behavior, expectations, and evaluation criteria. We adopt a technology, and we become loyal to it, but we also develop new expectations, and needs. We did not imagine that once we have access to integrated data systems, we will develop the expectancy for predictive data. A few years ago, we had no other expectation than to have access to aggregate data. No wonder we are facing the watermelon effect - the question is how much of this is being influenced by continuously changing expectations of technology users, and how much of this has to do with the fact that service providers operate in an ecosystem that needs alignment. Independent of the cause, both need key performance indicators reviewed.

When analyzing the key performance indicators, we cannot study the technology or the people only (service providers), but we need to identify a way to study both and ensure some level of alignment.

Technology is continuously evolving, and developers work with Agile methodology. In technology, well-defined plans are only around the platforms, tools, and systems to be used, the technology landscape and the integrations to be built, but then the functionality of the tools and systems is not documented a priori, part of the plan, but it is documented in real time, as the technology implementation and configuration happens. It is precisely to allow developers to explore the technology capabilities and allow integration of improvements that come from testing and experiencing the tools. Agile is not just about being flexible but also about a mindset that fosters collaboration, efficiency, and continuous improvement (Brook Appelbaum, 2023).

In social sciences, a similar approach in terms of managing change from within and following a more adaptable path to implementing a change is represented by Action-Research. Moreover, this methodology applied by "internal resources of an organization seeking to inquire into the working of their own organizational system, in order to change something in it, can be considered or understood as undertaking insider action-research." (Shani and Coghlan, 2021).

Action-Research as a method applied in the workspace, was inspired by Kurt Lewin, a renowned German American psychologist based in the USA, a pioneer in the field of psychology applied in society and organization. The method can be applied in any situation of change in a phased manner, based on 3 steps - planning, action, and evaluation/research and in a democratic organizational environment will promote critical thinking and collaboration. Today the methodology most often applied in technological projects is Agile, a methodology based on the same principles. But technology alone can't bring big benefits. It has also been mentioned in the previous sections that the world of training has most easily adopted these innovative methods and multiple controversial theories around which the most advanced organizational structures in the world have been built. Collaboration and immediate action are the key components to adapt solutions to the current dynamics.

NTL - The National Training Laboratories in the USA have experienced behaviorist theories in conjunction with the research-action method and helped create a practice and methodology to help organizations develop - team building, process consultation, conflict management democracy and group autonomy have become recurring themes in the literature. The development of the organization directly influences the quality of social life so that the task and responsibility of organizations to define and solve problems, to introduce new forms of leadership, to change the organizational culture cannot be fulfilled at an intuitive level without considering social behaviors and emotional balance / emotional well-being.

Organizations now have an impressive set of data, but they lack a solid foundation to capitalize on it and the intuitive way to draw conclusions does not necessarily lead to essential changes. A rigorous collection of data and the choice of appropriate methods of analysis, together with the immediate implementation/correction of the results, can have a considerably greater and at the same time positive impact on the organization and extrapolate, on the society (Chevalier and Buckles, 2013).

A study conducted by the Swedish Healthcare Organization with the aim to improve an integrated care system for elderly persons (Lifvergren, Huzzard, and Hellstrom, 2015) is very eloquent for the business world and how we constantly try to improve services and reduce cost, but with a focus on innovation and increased satisfaction at the same time

Starting in 2002, elderly integrated care has been prioritized in Sweden, being a strategic objective for the country. A report from 2006-2007 (SALAR, 2008) was suggesting unclear outcomes from a patient perspective after numerous improvements were applied in 16 out of 20 counties in Sweden. The challenges that prevented the outcomes from appearing, were around the complexity of the stakeholder ecosystem, being formed out of specialists (doctors, nurses) politicians too, and administrators, but at the same time around the linear approach that has been adopted. According to (Mintzberg and Glouberman, 2001) it is vital to ensure collaboration of all stakeholders in the system, including the patient.

The research has been conducted with patient representatives, co-workers, managers from various healthcare providers, politicians, and union representatives, to ensure a 360 degrees approach. The project team developed a routine inspired by the action-reflection group: mapping the patient pathway, identifying, analyzing, and measuring weak points, designing and introducing improvements, evaluating implemented solutions, and reflecting. The problems identified were always confirmed through quantitative data using structured data collection, stratification of data, control charts, Pareto charts, fishbone diagrams, cause-effect diagrams, and histograms. Measurements were also repeated after improvements to ensure the practicality of actions taken.

At the end of 2015 the Orjan network presented some key outcomes of its work:

* elimination of waiting time at the reception, except for non-acute heart ailments

* reduction in the number of hospital admissions to the medical clinic (18%)

* initiation of process work in many other clinics

* increase in staff awareness and learning along with patient pathways.

The reason we provided more details from the study is to make a parallel and highlight the similarities between the study approach and the corporate business model. Large companies developed business models that encompass all of the above. The perception is that all the business routines emerged intuitively and are not grounded hence, case studies and white papers are valid sources of information, and they contain valuable empirical data. From defining the problem to be solved or improvement need, data collection and analysis, regular meetings to track progress, address risks and issues, and change the course of action, all exist in the business practice and are followed at all levels. Hence, it has been easy to conclude that action-research is the most appropriate way to study watermelon effect and other pressing business matters.

2. Methodology

To elaborate more on the objective of this study, the human resource service providers are usually dealing with service issues around timeliness and quality of service - two areas where there is a constant focus to improve, the theory which has been validated by the metrics identified as being most commonly used, because 3 are used by 20 companies out of 20 (as shown earlier in Figure 1): Customer Satisfaction, Average Handling Time and First Response Time.

We selected 2 different companies to conduct case studies since in both watermelon effect seemed to be present and related to the same metrics - quality and timeliness of the service. Companies hold important data that is and should be used to help improve the services, products, and overall business performance. Technology is supporting data analysis more than ever, having access to integrated systems and making possible the use of tools without necessarily having deep expertise in statistics and data interpretation. But, having access to data is not everything, companies should challenge the status-quo including the metrics, leading to new correlations and identifying new ways to improve. An interdisciplinary approach is adding more and more value, by creating diversity and the possibility to look at the issues from all angles. Focus groups are quite common in the corporate environment, hence bringing together specialists from quality assurance, business analysis, data intelligence, business operations management, client management, clients and representatives has been achieved, ensuring diversity not only from a business experience perspective but from an educational background too. Starting with a set of data from systems, collecting input from each participant helped define the problem to be solved and set objectives along with directions for action.

During both case studies process blueprints and diagrams have been used to map the employee journey and each process, step by step, to understand the experience end to end. Processes that were designed in early 2010 were no longer relevant for employees in early 2020 (Kalbach, Mapping Experiences: A Complete Guide to Customer Alignment Through Journeys, Blueprints and Diagrams, 2021). According to Jim Kalbach, there are two types of alignment: the alignment to the experience from the outside in and internal alignment between the teams that create the experience (Figure 2).

The first case study, conducted between 2019 and 2021 has been focused on the timeliness perceived issues (average resolution time) - watermelon effect was present in employee perception. Although the service center was providing the resolution in 1-3-5 days depending on criticality as agreed with the stakeholders, the employees were unsatisfied. The agreed timeliness was met -95% and showing green, but the employees were complaining and escalating.

The study has been conducted on a global company, having 18.000 employees worldwide. The focus has been on Europe, where the complaints were mostly coming from. Europe - 6.521 employees, LATAM - 1.823 employees, US - 7.310 employees, APAC - 2.346 employees. We did not factor in any demographic data other than nationality/region, this being a subsequent step if the study would not reveal useful information without these attributes.

We conducted a focus group oriented on the improvement of timeliness along with customer satisfaction KPIs formed of 3 quality assurance specialists, 2 data intelligence specialists, 3 business analysts, 2 operations managers, 2 team leaders, 1 project manager, and 1 Black Belt operations specialist to define the problem and identify a plan to address the issue.

The second case study from 2019 - 2021 on Average Handling Time (AHT), has been conducted with an international company subsidiary in the United Kingdom, having cca. 21.500 employees, delivering professional services.

The objective was to improve metrics, since a data analysis on a different matter (chatbot enhancement) revealed that in 2019 compared to 2013 (data has been stored from 2013 and this was the basis we were starting from), the time used to resolve phone queries increased from an average of 8 minutes to 23 minutes and still, Customer Satisfaction Scores were not reflecting any severe issues - by contrary, it was quite linear, the average being 86,95% (green, according to dashboards, against a target of 85%), as shown in the graph below (Figure 3).

The dilemma we had to deal with was that the customer satisfaction survey was showing linear positive results (CSS), while AHT was disappointing; so, it was very unusual because until that time, efficiency was the most important attribute of the service provided. So, this could be seen as the reverse of the watermelon effect: although AHT was red, the perception of the service was green, validated by CSS.

The need was identified because of not being able to meet the expected performance levels across several years - the target was set for 7 minutes per call in 2013, to handhold the user to navigate the portal and/or resolve any technical issues that may arise (password reset, wrong browser, etc.). The customer was somehow satisfied because there were almost no complaints associated with timeliness coming from the business but on the other hand, the service provider was paying considerable penalties for not meeting the service levels. The service provider appointed a team of professionals - operations managers, quality analysts, data analysts, and business analysts, led by the service center responsible - to identify the root cause and remediate the problem for the customer and the company too.

3. Results

Case Study 1 : Although the timeliness metrics were showing as yellow to green - 92% - 95,89% - from March to October 2019, the Customer Satisfaction Survey results were quite linear and still on a negative trend, as shown in the chart below (Figure 4) throughout the same period of time - 84% to 82%, which was showing a contradiction - if timeliness would improve and end-user would get matters sorted, how come satisfaction does not improve?

During the discussions of the focus group, we concluded there are 2 hypotheses that need to be tested:

* The prioritization was not aligned with employee expectations (queries and inquiries around payroll are a priority that allows 1 day resolution - time and attendance, promotions, demotions, bonuses, etc. - followed by employee referrals, mortgage, medical, etc., within 3 days, and rest within 5 days).

* Overall processing time (vendor + other vendors or customer organization like managers, workers council, HR, etc.) was not aligned to employee expectations - in cases where one or more vendors were involved, the natural time was not equal to processing time (vendor has a performance indicator of 1/3/5 days but excluding pending time with customer organization or other vendors).

300 interviews were conducted during March 2020 and August 2020 with the help of 25 service center representatives. The sample has been selected from the reports showing very low and high customer satisfaction scores from January 2020 to March 2020, taking into account a 90% confidence level and 5% margin of error. (The sample was statistically calculated at 258 rounded to 300 to allow 12 interviews for each representative for training purposes too).

The interview duration has been 7 minutes on average and consisted of 2 major opening questions: What made your experience with HR Service Center disappointing? and What can we do better next time?

The representative assigned the feedback to a sub-process (onboarding - pre-employment check, onboarding - induction day, etc.) and reported using a mapping sheet (interview number, positive/negative, process, sub-process, timeliness, accuracy, representative issue). In addition, the interviews have been recorded and a tool embedded into the voice solution has been used to automatically analyze the frequency of words, and tone of voice, and extract the clusters.

The preliminary conclusions of the discussions (starting point for further data analysis and validation) were that we are not measuring the natural or perceived time of resolving tasks but the effort time on the service center side. The employees were complaining about the end-to-end time - comprising the time of all vendors and internal stakeholders - which is changing the paradigm. We now need to look at the aggregated service, at the experience overall, thinking or working in silos is not an option any longer.

The onboarding moment/journey is one of the most important for companies and employees and from the data it was obvious that it was the focus (most of the negative feedback was in this area). Some studies concluded that employees are more likely inclined to stay longer with the company if the experience associated with onboarding is positive. Also, some studies show that employees who did not have a pleasant onboarding experience, are likely more inclined to leave the company in the first 9 months of their employment.

The end-to-end process is relevant for the study also because it involves more vendors specialized in different areas - we might take as examples specialized recruiting services, professional background check services, pre-employment screenings (for visa, residency, work permit, etc.), HR outsourcing partners (transactional - offer letter, contracts, employee file, etc.), training services - allowing analysis on internal alignment, between all entities (vendors) behind the holistic experience.

The analysis started with feedback provided by the interviewed employees for different areas of the journey - the below chart (Figure 5) shows the type of feedback (positive or negative depending on the position in the grid - 30% to 130%) correlated with the area of the journey.

The perception of the overall onboarding experience was quite positive, but the negative feedback mainly came from the background check step and the induction day which were two areas out of the service center control. Still, there were some improvements that the service center could apply - setting expectations with candidates (if the candidate handling service is outsourced) on the duration, and effort expected, and setting some notifications on the way to alleviate the pressure of the waiting time (while background checks are performed, candidates might be under unnecessary stress). With regards to induction day - the first day of an employee with the company - an agenda along with contact details of their manager and their trainer for the day, could help set expectations and eliminate the stress of the unknown.

The plan was to prepare draft communications to candidates, establish timing for sending them out, align vendors through client communication to avoid doubling the effort and overcommunicating to candidates, set deployment date and launch. The effort associated with the mentioned deliverables is time consuming, given that my client was quite large - so we decided to start with one country (UK, approximately 200 new employees every month) to act as a pilot, and if results show improvement, deployment for rest of the countries to follow. This was in line with the study since the analysis was conducted on data and subjects from this country only. Usually, we do not recommend deploying different features in each country, for efficiency purposes (imagine an upgrade change a feature that is different from country to country; testing and implementation would either need to be done by dozens of professionals or by the same professionals but with a longer delivery timeline).

The improvements have been deployed in December 2020, the month chosen because of the lower number of new joiners, to help measure the results through qualitative interviews (lower effort).

Although the improvements were not impacting the real time process (the professional background check was still long), the perception was different, because the candidate did not have to wait for results without knowing that it would take up to 3 weeks and on the way, being reminded that it is in progress and it is usual to take long, they won't stress about what if something goes wrong.

The watermelon effect was coming from the fact that employees were evaluating the overall experience but providing feedback when they reached out to the service center, hence, the metrics were associated with the service center performance.

The time from the moment the candidate has accepted the offer to their first day with the company is what employees will evaluate when reaching out to the service center to ask for an update on their background check status, to get support on navigation when uploading necessary documents to their employee file, etc.

Case Study 2: After focus group discussions and preliminary conclusions, the managers looked at the individual performance (2,5 years experience on the job), language skills (understanding the request), process knowledge (service representative skills on processes, portal navigation, etc.) to understand where the issue comes from. One might rightly ask why the company did not change the service provider - although AHT was not met, this was not seen as a metric that adds value necessarily, but rather the company agreed to add resources several times. The performance was improving for a couple of months and then mysteriously was again back to square 0 and even worse. The phone queries have always been analyzed and used as a feed for technology (from 2017 for chatbot feed especially), in the sense of upgrading technology to resolve issues employees were dealing with during navigation - like areas of improvement (annual performance not being straight forward, pay-slip information not being detailed enough on the portal, etc.) - because navigation was the main area employees would call the service center (supposing). We decided to look at the data extracted from the calls and analyze the words and themes; call recordings were only available for the past 6 months due to regulations hence we could use data collected for training purposes.

The technology adopted in 2012 - when the solution was implemented (both technology and service center) - was on-premise, and since then, we adopted cloud technology and hence, access to more integrated data. Also, the behavior of the consumer of technology was changing, not only at work but in day-to-day life. Hence, it was necessary to conduct a comprehensive study on consumer behavior, based on grasping the new needs generated by access to technology, which was not a priority for the business to invest in, at that time.

Still, to get an idea of potential behavior changes, the study over average handling time had to be conducted on the type of queries associated with process and technology - a mapping between the type of query and process along with an existent feature on the portal (e.g. is portal able to answer through an article question around pay-slip?). This was the result of a focus group organized to discuss the matter. The focus group brought together representatives from operations, quality and training, data analytics, management, and clients.

The relevant data (type of queries) for the purpose of this study was available only from 2017 on, when the service center got access to a more sophisticated ticketing/engagement system, integrated with the telephony, capable of providing a more complex set of data, to allow comprehensive analysis. Until 2017, only the data associated with the identity of the caller, the identity of the service representative, and the duration of the call along with the outcome type (successful/not successful) have been stored, due to the limitations of technology in use.

The data provided some relevant insights, on type of queries mostly addressed through service center, shown in Table 1, and has led to a series of improvements that could change employee perception. Having access to the resolution provided through service center, we could identify some areas where behavior was changing:

* employees were no longer conducting their own research on internet to understand pay-slip information but they wanted the information to be personalized and available to them on the portal;

* employees expected to be advised on type of benefits to choose in order to maximize income.

* employees wanted to have access to their retirement plan information through the portal, to simulate their income if retiring in the next few years.

The surprise was around employees developing new expectations, maybe emerging from use of technology - they were expecting support for some features that were not yet available, not planned and not communicated.

The other navigation queries were quite usual, employees looking for support in different areas, at that time but there are no guarantee employees will not develop new expectations in relation to the features of technology.

The plan to address the findings has been discussed and agreed with all stakeholders:

* introduce a simple table explaining the pay-slip information in the payroll section of the portal.

* introduce a field in the benefits catalogue to distinguish between taxable and non-taxable benefits (some benefits may carry additional tax, while others will not incur additional tax)

* introduce a pension calculator, in line with local regulations, to allow employees to simulate their income at retiring age but in advance too.

The enhancements have been deployed throughout September 2021 and March 2021, and at the end of September 2021 the analysis on type of navigation queries along 12 months, was showing quite different in some areas like shown in Table 2;

* Pay-slip information navigation queries decreased significantly from 14% to 7,80%, showing improvement.

* In the benefits area the feature introduced might have caused new expectations and needed further analysis. However, it was obvious the benefits area was not in accordance with employee expectations.

* Retirement queries dropped significantly from 15,3% to an average of 3% which validated that the measure taken was successful.

Reflecting on the results of the study, the management decided to closely monitor the behavior of the employees in relation to HR technology, to constantly adapt to their expectations and improve the portal to maximize self service

The conclusion was that we need to consider AHT as an indicator of employee / consumer needs and not necessarily as an indicator of service center performance. Especially during Pandemic, we could notice some new needs emerging from the isolation - the need to communicate.

Later in 2023 the customer decided to change the flexible benefits platform, to provide employees with access to a more dynamic environment, providing more insights and relevant simulation to help them make their own decisions.

The opposite watermelon effect has been confirmed - taking longer time to resolve employee queries was not necessarily poor performance but proactive attitude of service center representatives who identified ways to help employees find available online tools to perform simulations, helped them understand how policies apply. Or seen as a watermelon effect, customer satisfaction was good/green but did not feel different because of long time on the phone, by contrary, employees were appreciative for the support received by service center to help them navigate the websites in providing them with solutions.

4. Discussion

We are facing today a silent crisis of HR technology along with the silent quitting of employees, and while technology can be incredibly transformative, can also be contributing to employee disappointment if not chosen carefully.

Companies have to make choices over platforms and vendors to use, and HR specialists don't necessarily have the skills to make the right choices - HR professionals are expected to know employee behavior and how integrations work, and they are expected to set up key performance indicators that would help them measure the quality of the tools, the quality of the processes, the quality of the experiences, and still, there is no unique recipe to guide them to success.

Our study comes to support the hypothesis around changing employee expectations especially because of the use of technology, and access to aggregated data, hence generating a watermelon effect that companies need to consider as a source of inspiration for their next enhancements rather than as a measurement of the service itself only.

Service providers should look at measuring the end-to-end experiences, not only their individual performance. It is imperative to build technology around humans, especially now that the business and technology world become human-centric. Probably now it is more than ever before, the time for consultants to become service aggregators, and play the role of coordinating the vendor ecosystem for their clients, offering them holistic experiences.

Analyzing the metrics studied one by one, the first conclusion is that the average handling time for queries of employees increased because of new needs of the employees - they needed predictive analytics regarding their pension, for example (they wanted to know how much they contributed to the pension fund and what their contribution will look like if they decide to retire next year).

During the analysis, we could notice the proactiveness of the service center representatives - the first requests on the pension simulation were closed without a real time resolution, by the book, as there was no answer in the policies, in the portal articles, in frequently asked questions - who raised the concern and identified existent solutions on the internet, going the extra mile, because all employees want to succeed in the end. The process of adding a potential response to the frequently asked questions book of a service center is not an easy task, because it needs validation in several forums. Hence, comprehensive analysis, company case studies need to be conducted to become relevant and especially in large international companies this may be a process that takes years of continuous efforts with a financial impact that not all are willing to invest.

After careful consideration, companies should use AHT as an indicator of employee expectations changes and avoid setting a target - employees might feel quickly dismissed if service center representatives are rushing the conversation.

This conclusion helped in upgrading the portal with an automated simulator for their pension contribution. The monitoring time - set for 6 months shown in Figure 6 revealed a decrease to 12,11 minutes/call, but in the last 2 months, the monitoring time increased to 12,56 which needed further analysis on new needs of employees. Still, the technology enhancement reduced the handling time by almost 50% and significantly reduced the number of queries in this area - by 12,3% to be more specific - which is a clear indicator that the new need for data incorporated within technology has been the right resolution.

The main conclusion of the Average Handling Time study was not only that employees need improvements in technology (portal) but they develop new needs once data is available to them. The expectations to get support on navigating the portal in 2013 changed to expecting predictive analytics from the portal - hence requesting the information from the service center in the absence of the information being available on the portal - in 2019.

It is more important now to stay connected to employees' perception of technology and identify ways to keep them engaged, especially with the shift to a hybrid way of work. If companies want to obtain high employee engagement, they need to fully satisfy the basic needs of their employees, and make sure they don't fail them, before getting to a more strategic objective of the use of technology - a platform to keep employees engaged through communication and appreciation, a workspace that is bringing together the employees working from home, employees working from different geographies, employees whose expectations might be different from culture to culture.

On the other hand, keeping employees connected through employee portals / platforms and systems does not guarantee a seamless experience across areas of their life at work - looking at the moments that matter for employees, seeing them as full or end-to-end experiences instead of measuring fragmented service performance indicators, will help improving engagement.

Measuring quality of service through customer satisfaction surveys is no longer a matter of measuring accuracy, relevance of resolution provided, and language capabilities, but needs to be seen as the perception of the overall experience in that moment of work life. And the results should be the starting point for finding the issues, across the vendor ecosystem.

After the study was concluded a new employee satisfaction survey was to be launched after each interaction with the service center - meant to measure only the perception of the service provided, to make sure a distinction is made between the end-to-end experience of the moment and the interaction with the service center.

The new Customer Satisfaction Survey has been implemented in May 2021 (Figure 7) and it is still being monitored, the aim being to switch the focus from traditional metrics to perception over the service, seen as the starting point for identifying new ways to measure performance.

Conclusions and recommendations:

This study aims to provide a new performance paradigm in HR service delivery - focusing on traditional key performance indicators may lead to a watermelon effect (in time, and not always linear, to exclude potential fraud) and this effect does not necessarily depict the performance of the service provider (be it the company itself, be it an outsourcing company, or an ecosystem of service providers), but might be a valuable resource for improving the end-user experience.

The watermelon effect - seen as the contradiction between satisfactory service key performance indicators and disappointment of end-users, or the contradiction between process and experience - can unlock the opportunity to explore new technologies and services, based on new needs generated by exactly the development of technologies and services.

The study focused essentially on 3 metrics - Customer Satisfaction, Average Handling Time, and Average Resolution Time (Timeliness) as the most used by my customers throughout my career, but the effect can be observed in all other areas of the service:

What if we study the Utilization Rate across HR processes to determine if mandatory tasks once completed result in positive perception over experience because of the actual completion and not because of the technology friendliness?

What if we study the First Response Time to understand if this is linked to resolution or to acknowledgment of the query? Do we understand end-users expect a real-time resolution? Are we equipped to solve it real-time? Are we measuring what employees expect or the availability of automation in technology (automated acknowledgement)?

What if we study Quality Assurance by re-thinking the expectations? In early 2010 when services were being moved offshore, language skills were weighing much more than they do today when technology has the capability to translate in real time. Do all companies consider calibrating with customers? Do all companies consider the accuracy or fairness of the solution provided? Applying Root Cause Analysis principles along with or as part of Participatory Action Research methodology, companies will embark in an adaptative journey to secure the foundation for better employee engagement.

HR function might need service aggregators, with the mission to unify the vendor ecosystem, measure their unitary performance, and start looking into providing seamless experiences, now that integrations are mostly native, AI can ensure the same look and feel across systems. And once unitary experience is enabled, the watermelon effect should be studied continuously to understand the new needs of the employees generated by the use of technology (expanding possibilities).

Our future research will focus on identifying a set of new indicators as a result of studying the watermelon effect over the years, set of indicators that will reflect the changing needs and expectations of technology users. The new set of indicators will be the result of aggregating or combining traditional key performance indicators, to measure the overall experience of the users - from a process perspective, but also from a technology perspective, engaging the user in the process.

The bottom line, HR needs to focus on measuring the experience of the employees when assessing the quality of the services provided by their vendors. Monitoring the traditional metrics is not sufficient and will lead to a watermelon effect more often in the future. Having access to data does not mean data is the only source of truth, especially in the fast-paced development of technology. As the new priorities in the world change and the focus is more human centric, it is time for qualitative research around experiences. A holistic approach to look at every step of an employee instead of only tracking disparate metrics depicting service performance and efficiency will add value and will improve the business performance overall.

References

References

ABB (2017), Avoiding the 'Watermelon'Effect (retrieved from ABB: https://library.abb.com/).

American Society for Quality (2024), American Society for Quality (retrieved from https://asq.org/quality-resources/customer-satisfaction#:~:text=Customer%20satisfaction% 20is%20defined%20as,changes%20its%20products%20and%20services).

Brook Appelbaum (2023), The Agile Manager's Handbook, E-Book: Planview Ltd (retrieved from https://www.planview.com/resources/guide/agile-methodologies-a-beginners-guide/).

Chevalier, J. M., and Buckles, J. D. (2013), Participatory Action Research, London: Routledge.

Dixon, M., Freeman, K., and Toman, N. (2010), Stop Trying to Delight Your Customers, Harvard Business Review, July-August.

Dixon, M., Toman, N., and Delisi, R. (2013), The Effortless Experience - Conquering the New Battleground for Customer Loyalty, New York: Penguin Group.

Kalbach, J. (2021), Mapping Experiences: A Complete Guide to Customer Alignment Through Journeys, Blueprints & Diagrams, Sebastopol: O'Reilly.

Lifvergren, S., Huzzard, T., and Hellstrom, A. (2015), After a Decade of Action Research: Impactful Systems Improvement in Sweedish Healthcare, in: Bradbury, H. (Ed.), The Sage Handbook of Action-Research, SAGE Publications Ltd., pp. 336-350. https://doi.org/10.4135/9781473921290

Glouberman, S., and Mintzberg, H. (2001), Managing the care of health and the cure of disease-Part I: Differentiation, Health care management review, 26(1), pp. 56-89. https://doi.org/10.1097/00004010-200101000-00006

Qualtrix (2020), Qualities, retrieved from: https://www.qualtrics.com/uk/experiencemanagement/customer/net-promoter-score/

SALAR. (2008), The Swedish Healthcare System: How Does It Compare with Other EU Countries, the United States and Norway? Swedish Association of Local Authorities and Regions.

Shani, A. B. (Rami), and Coghlan, D. (2021). Action research in business and management: A reflective review, Action Research, 19(3), pp. 518-541. https://doi.org/10.1177/147675031 9852147

Swinscoe, A. (2016), How to Wow, 68 Effortless Ways to Make Every Customer Experience Amazing, Edinburgh Gate: Pearson Education Limited.

WHO (2000), The world health report 2000, World Health Organisation.

WHO (2012), World Health Statistics, World Health Statistics.