Multi-Rater Feedback

Unlike, the traditional top-down appraisal where a supervisor appraises the performance of their subordinate, Multi-Rater Feedback incorporates multiple perspectives by using feedback from a variety of sources. These sources include peers, subordinates, customers, self, and supervisor.

May be called multi-source feedback, mult-rater feedback, multi-level feedback, upward appraisal, peer review.

The results of this type of feedback process provide an understanding how how the employee is perceived from different perspectives. This process helps an individual understand how others perceive them.

Feedback is essential to facilitating performance improvements. Feedback allows people to utilize their strengths to their advantage. Feedback informs employees which actions create problems for others and to know what changes may be needed.

Chapter Highlights

1. Uses
2. Benefits
3. How it is conducted
4. Requirements
5. Benefits of using the Internet
6. Demos
7. Item Bank

Principle of Multi-Rater Feedback Provides an understanding how how the employee is perceived from different perspectives.

Multi-Rater Feedback: Uses

Uses for Multi-Rater Feedback include:

• Employee Development
• Performance Appraisal
• Performance Management
• Training Needs Assessment
• Evaluation of Training
• Attitude Survey
• Organizational Climate Survey
• Customer Satisfaction Survey

This process can also be a motivator of performance since it shows the employee that their opinions and views are considered important.

Benefits

May improve service to customers if they are able to offer feedback to the employee. Multi-Rater Feedback offers a more complete picture of the employee's performance. This feedback can provide guidance on skills that an employee may need to develop.

How it is conducted.

1. Develop questionnaire A questionnaire used for Multi-Rater Feedback typically contains items that are rated on a 5 point scale. These items may be developed to measure different dimensions of job performance (e.g., communication, teamwork, leadership, initiative, judgment, ...). Questionnaires also typically include one or more open-ended questions to solicit written feedback.

   Questionnaires typically include from 50 to 100 items. When estimating the amount of time to complete the questionnaire you should estimate about 1 minute per questionnaire item.

   If using a printed questionnaire form, you should consider using forms that can be scanned into a computer.

2. Ensure confidentiality of participants Steps must be taken to ensure the confidentiality of the feedback results. For example, feedback ratings from several subordinates may be combined (averaged) to mask the identity of an individual subordinate. Comments or written answers to questions may be summarized in the results to mask the identity of the author. The confidentiality helps ensure that the results are genuine.

3. Provide training/orientation Often the feedback process involves use of one or more questionnaires, confidential information, and involvement from many different areas of an organization. Therefore, training and orientation to the feedback process is needed to facilitate a smooth feedback process. During this training/orientation, employees should be informed of what Multi-Rater Feedback is and why it is being implemented at your organization. You may want to provide samples of the questionnaire items and/or feedback results.

4. Administer the feedback questionnaire Distribute questionnaire forms (if using printed copies) with instructions. May want to prepare answers to common questions if other employees will be assisting in the administration. If possible, post the questions and answers to your web site for easy access.

   It is important to monitor the progress through the system in order to contact employees who need to complete forms.

5. Analyze the data Basic data analysis would include averages of ratings. More complicated analyses may include item-analysis and/or factor-analysis. Types of analyses include: Performance Dimension Summary; Summary-Performance vs. Expected; Individual Item Ratings; Item Ratings-Performance vs. Expected (normed); Highest- or Lowest-Rated Items (shows individual's strengths and weaknesses); Group & Organizational Ranking, and Recommendations for Development.

   You may want to analyze the data by organizational division or department to assess group and organizational strengths and weaknesses. This can be used to support or promote training and organizational development.

6. Develop and Distribute Results Feedback results should be shared with the employee. It should not be mandatory that the employee share the results with their supervisor. However, you may want to make this an optional part of the performance review of the employee.

   Most results for an employee will include a comparison of their ratings to the ratings of their supervisor and and average of the ratings from others (peers, customers...). The comparisons may be in the form of numbers or simple bar charts.

   You may want to provide individual review sessions or group workshops conducted by a facilitator to help individuals review and understand the results and develop appropriate goals and objectives.

Dimensions

Responses are collected for items that fall under a specific dimension of job performance. A single questionnaire may contain dozens of questions that measure responses on one or more dimensions. Example dimensions are shown below.

• Problem Solving Items under this dimension measure how well a person can understand information and options, give appropriate considerations to information, make correct decisions, analyze and interpret information, and react to changing situations.
• Planning and Organizing Items under this dimension measure a person's ability to develop plans and objectives, develop long-term solutions, set business objectives adhere to schedules.
• Communication Items under this dimension measure the ability to present information formally and informally in both written and orally. Also measures the ability to communicate with customers, staff, peers and supervisors.
• Supervisory Skills Measures the individual's skill level in planning, organizing and overseeing the work of subordinates. Also measures a person's ability to manage work flow efficiently.
• Administrative Skills Measures an individual's ability to implement and monitor actions to ensure compliance with policies and regulations. Also helps identify the ability to distribute information, allocate staff and maintain records or documents.
• Business Control Measure the skill in, and concern for, controlling expenses, reducing costs, setting performance standards and reviewing budgets.
• Manager Potential Index A measure of the composite score of the scales reflecting an individual's overall ability to successfully perform management tasks.

Requirements?

A Multi-Rater Feedback process requires a coordinated effort to collect hundreds, or even thousands, of pieces of data. In addition, the validity of the results is enhanced by ensuring confidentiality of the participants.

The survey administration may require time (to complete the questionnaires) and computer resources to analyze the data.

Benefits of using Internet

Fortunately, there are many software companies that offer tools to assist you in conducting this type of feedback process. Some links to these companies can be found at the HR-Software network.

• Web Access Are your employees scattered throughout the globe? Or, across different states? No problem, just use the Internet as the medium through which your feedback process is conducted.

• Confidentiality Although the data is collected by a third-party, the confidentiality is enhanced in that fewer internal employees have access to view the data.

• Simplified Data Analysis Spending time entering data into a spreadsheet or database for analysis? Use web based tools to collect, store, and analyze data.

Demos

Try this online demo from HR-Survey.com

Item Selector

Item Bank

Item Bank

CSS