Performance Analysis Report

Introduction and Background

This report is submitted to Mike Merrill, the Vice-President of Sales, as well as Jane MacKenzie, the Sales Director of Exclusive Clothes and Gifts (ECG), as a report of findings and recommendations on the performance and training issues analyzed by EduCadre.

Members of EduCadre met ECG’s management team, who believe that increased knowledge and skill of telephone operators and supervisors in describing the features and benefits of the products is key to increasing catalog sales and reducing customer complaints.

Mike Merrill, VP of Sales, agreed to allow EduCadre to conduct a thorough gap analysis, that included both performance and training needs assessment. This was done to determine whether, in fact, a training solution would solve the performance issue or whether an alternative solution could be found.


Purpose of the Report

The purpose of the Performance Analysis at Exclusive Clothes and Gifts (ECG) is to provide the management team with the results of EduCadre’s analysis, including causes of and possible solutions to increasing catalog sales and reducing customer complaints.


Analysis Methods

To determine the causes and recommend relevant solutions, EduCadre performed the following tasks:

    • Interviewed executive and managerial staff about their area of focus within the ECG organization.
    • Observed telephone operators and supervisors’ meetings and daily operations to get an accurate view of the ECG day-to-day processes.
    • Surveyed operators and supervisors about customer service training, product information training, and training opportunities, as well as their thoughts on management practices.
    • Held focus groups with telephone operators to determine the processes for ordering, complaints, training, and recommendations.
    • Interviewed managerial staff to determine and document existing processes to create a Customer Journey Map of the step-by-step experience of the customers engaging with ECG.
    • Conducted focus groups with customers to identify strengths, opportunities to improve and primary concerns in regard to the speed of service and operator knowledge of products.
    • Conducted “Mystery Shopper” observations to engage with customer service and document the experience through a variety of scenarios.
    • Reviewed organizational documents, such as employee handbooks and company memos to gain insights into management styles, legal policies and safety procedures.


Data Summary: Customer Complaints

The following chart outlines the distribution of customer complaints across Wait Times, Operator Etiquette, and Operator Product Knowledge.

This bar graph shows the wait times, operator etiquette, and operator product knowledge

Source: Supervisor Survey

As shown, customers complained more often about Operator Etiquette, followed by Operator Product Knowledge, and then Wait Times.


Data Summary: Call Length

The following chart illustrates the average length of calls. Data was collected from a review of 15 recorded calls and listening to five Telephone Operators for three calls each (15), for a total of 30 observed calls.

This bar graph shows the average call length

Source: Call Reports

Of the 30 calls observed, 10, or 33%, were completed in under 10 minutes, which would satisfy the six calls per hour target.  However, double this amount of calls, 20, or 67%, were completed in between 20 – 25 minutes.


Data Summary: Outcome of Longer Calls

The following chart outlines the outcomes of the observed calls that were over twenty minutes (aka longer calls).

This pie chart shows the outcome of longer calls.

Source: Call Reports

This chart is based on the observations of longer calls which included 8 transfers to a supervisor (either initiated by the customer (5) or by the Telephone Operator herself (3)); 7 unsuccessful calls (defined as the Telephone Operator losing patience and getting frustrated with the customer (5) or the customer cancelling the order (2)); and 5 successful calls (where the Telephone Operator was able to find the information; note that even on these calls, the customer was observed to be curt).


Data Summary: Telephone Operator Preferences

The following chart highlights TO responses to the question: “I would feel more successful in my job if….”

This bar graph shows what Telephone Operators would feel more successful if...

Source: Telephone Operator Survey

Per their responses, over 80% (20 or more) of TOs indicated that they feel they would be more successful in their jobs if they were provided with more effective computers and software and if they received training in shorter increments.


Telephone Operators also indicated their learning preferences:

This bar graph shows Telephone Operators learning preferences.

Source: Telephone Operator Survey

Per their responses, over 70% (18 out of 25) indicated that they prefer visual and hands-on learning.

Current State, Desired State, and Recommended State

The table below gives the condition of the current state, the desired state, and EduCadre’s recommendations

Current StateDesired StateRecommendation to Achieve Desired State
1) Telephone Operators (TOs) are not sure what to say when dealing with frustrated customers, they don’t have consistent scripts to start and end calls, and they miss opportunities to recommend related products.
TOs are not provided with training on customer service skills.
Customers often get put on hold and there are many customer complaints regarding operator demeanor.
Operators who have customer service skills and an ability to de-escalate frustrated customers as well as serve customer needs by politely suggesting products that may be better suited or are related to the customer’s requirements.
Customers receive a high level of service and there is a reduction in customer complaints.
Reduced call wait times.

Increased customer satisfaction.
Provide customer service training that encompasses complaint handling skills, phone etiquette, conflict resolution and de-escalation skills as well as sales techniques like upselling and product recommendation.
2) All returns are automatically transferred to Supervisors, and Telephone Operators frequently transfer challenging calls to Supervisors. Supervisors spend most of their time on phone calls and have little time to help TOs or coach performance skills.Telephone Operators are able to handle a wider range of calls and are able to pre-empt potential customer complaints.
Supervisors are able to spend more time coaching, providing feedback. and encouraging operators.
Update procedures to clarify conditions to transfer to supervisor (e.g., as a last resort) and train TOs in customer service skills, and handling returns and exchanges.
3) Operators can't find information on the various products quickly and there isn’t a consistent level of information provided for each product. There is little to no capacity to immediately find related products, and current training doesn’t sufficiently prepare operators to address detailed customer product questions. Average call lengths are about double the target amount (6 calls per hour).There is intuitive organization and navigation of product information, and consistent level of information provided for each product along with recommendations for related products. TOs are able to handle 6 calls per hour.Develop electronic product guide organized by category and searchable by product name and number with more consistent level of information for each product. The guide can be updated when new products are added. Work with IT to implement similar organization of product information in the new system based on the updated product guide.
4) There will be a roll-out of a new order entry system (SimplyOrder) in Q3/19.All Telephone Operators are well-versed and comfortable functioning in the new order system, SimplyOrder prior to the system going “live”. SimplyOrder Training including hands-on instruction, knowledge checks and practice, and job aids.
5) Phone operators use older headsets which have poor audio quality which adds to poor understanding of what the customers are saying. Poor audio quality can also lead to operator fatigue.Phone operators use new, ergonomic headsets provided by the company and have an opportunity to choose from a selection of quality headsets.Invest in high audio quality, ergonomic headsets for operators to use.
6) New Telephone Operators are expected to remember how to use the telephone and order entry system after their first day and are primarily given demonstrations of procedures, using an LCD projector.Operators are given adequate time to learn and hands-on practice using the ordering and telephone the system before taking orders.Update onboarding procedures to include hands-on participation and knowledge checks.
7) High rate of dissatisfaction and turnover among Telephone Operators.High satisfaction and retention of Telephone Operators.Address employee biggest challenges (easier access to product information) and invest in training and development of TOs and Supervisors to support TO performance. Award retention via incentives.
8) Operators use Internet Explorer 11Operators can access the internet using a modern browser.Give operators and supervisors the latest version of Firefox, Chrome and Edge.
9) Operators are expected to handle increased call volumes during prime seasons.Operators are able to pay sufficient attention to customer needs during all seasons of the year.Hire additional operators for seasonal work. Seasonal operator onboarding will take less time due to improved training.
10) Operator attendance at monthly meetings is low. Often the subject of the meetings is the operators’ poor performance. The operators are not compensated for meeting attendance.All of the operators attend the meetings. The operators are compensated for the meetings. Hold meetings daily or at least weekly. Compensate operators for the meetings.
11) Supervisors don’t provide consistent performance feedback (lack of consistent use of call behavior checklist and other on the job feedback and during the yearly performance evaluations); Supervisors are often promoted to management-level positions without being provided any management-focused training.Supervisors with high capacity to provide constructive feedback and ability to manage operator performance.Clarify expectations for supervisors (e.g., identify frequency of meetings with their Supervisees)
Provide management-focused training for Supervisors on how to provide constructive feedback (ongoing and how to conduct the annual review); update the call behavior checklist with time for Supervisors to practice administering it.
12) Incentive program is based only on call volume and is a time-limited percent off discount for any order for one week. An incentive program that inspires employee performance. Change incentive policy from percentage off merchandise to voucher for merchandise, monetary bonus or time off, and expand when provided (for example, to reward retention, or sales).
13) Customer service survey is administered by Telephone Operator staffCustomer service survey is administered electronically or via a third-partyIncorporate use of the SimplyOrder to administer the survey.



Proposed Solutions for Consulting Firm to Carry Out

1Product Reference GuideEduCadre will work with ECG staff to develop an updated electronic product reference guide (searchable by product name and number and organized by product category) with consistent level of information for each product. EduCadre will work with IT to implement this organization and level of detailed information into the new ordering system. The guide can be updated as new products are added to highlight new offerings.
2Customer Service TrainingEduCadre will develop a series of training sessions that focus on complaint handling skills, phone etiquette (including developing updated scripts), conflict resolution and de-escalation skills as well as sales techniques like upselling and product recommendation. These will be instructor-led trainings, including focus on use of Updated Product Reference Guide (to provide hands-on practice with this tool) as well as other job aids.
3SimplyOrder TrainingEduCadre will develop a series trainings and job aids to train on the features and use of the SimplyOrder software.
4Supervisor TrainingEduCadre will work with ECG Supervisors to provide instructor-led training with use of role plays and scenarios to train Supervisors on how to provide constructive feedback (ongoing and as part of the annual performance review). EduCadre will also work with Supervisors to update the call behavior checklist and will train them on how and when to administer it.



Evaluation Plan

Evaluation in this project will take two forms, formative evaluation utilized by EduCadre during the creation of training materials and summative evaluation utilized by EduCadre after training has been administered. These evaluation methods will serve two purposes. First, they allow EduCadre to make preliminary determinations of the effectiveness of training before fully implementing training solutions. Second, they gauge the effectiveness and impact of the training from ECG’s perspective.

First, a formative evaluation will utilize pilot testing to gauge the effectiveness of the three trainings EduCadre will create: the customer service training, the SimplyOrder training, and Supervisor training. EduCadre will assess trainee knowledge and skills using objective numerical metrics (bolded for easy identification) in unique ways for each of these trainings:

    • For customer service training, EduCadre will use a comparison of pre-test and post-test performance data. Trainees will complete a short multiple-choice quiz before the training that covers specific terminology, concepts, and techniques that will be covered in the training. A post-test that covers the same techniques, concepts, and terminology, though using different questions different modalities (matching questions, short answer questions, for example) will be used and scores will be compared. If sufficient retention is attained as measured by 10 percent or better improvement in scores, the development will continue. If not, the training will be adjusted, and the pilot study repeated with different trainees.
    • For the SimplyOrder training, pilot testing will consist of scenario-based post-testing. A series of scenarios will be presented to the learner who will respond using a sandbox version of SimplyOrder to complete orders. Orders will be timed and graded for accuracy and completion. Trainees will be expected to complete orders in under 10 minutes, fulfilling ECG’s expectation of 6-calls-per-hour average order time.
    • For the supervisor training, a series of knowledge checks involving multiple choice, short answer, and peer-completed rubric will be administered. Supervisors will be expected to achieve a grade of 75 percent.

Summative evaluation will take place after trainings have been fully implemented and operators have successfully demonstrated adequate knowledge and/or skill retention through in-training assessments. This evaluation will provide ECG with data related to TO performance before and after training. This evaluation will measure the following metrics:

    • Average operator call volume(total number of calls per shift/number of operators compared before and after training on a shift-to-shift (e.g., day-shift to day shift or night-shift to night-shift) basis,
    • Average call time(total call time minutes/total number of calls) compared before and after training on a shift-to-shift basis, and
    • Number of calls referred to supervisorsfor complaints before and after training on a shift-to-shift basis
    • Reduction of customer complaints before and after training on a shift-to-shift basis

EduCadre estimates that each of these metrics can be improved by 10 percent. This significant improvement in average call volume per shift alone should translate to a significant improvement in sales, and the simultaneous reduction in average call time should contribute an additional sales level improvement. Further, it is reasonable to predict that a reduction in complaints should result in an increase in repeat customers, which also should have a positive impact going forward.

As EduCadre strives to be a partner in its clients’ growth, we always welcome input from management on metrics for summative evaluation as well as observations, impressions, and general considerations for improving our training methods and delivery. We value your input, because your learning is our job.



Request for Proposal

EduCadre has proven its expertise in helping ECG through each phase of this project. EduCadre is specifically skilled in developing training, needs assessments, usability of both print and eLearning solutions. At the request of ECG, EduCadre would be happy to submit a proposal to design and develop the solutions outlined in the section, “Proposed Solutions for Consulting Firm to Carry Out.”


WP Twitter Auto Publish Powered By :