Multi-Rater Feedback

Unlike, the traditional top-down appraisal where a supervisor appraises the performance of their subordinate, Multi-Rater Feedback incorporates multiple perspectives by using feedback from a variety of sources. These sources include peers, subordinates, customers, self, and supervisor.

May be called multi-source feedback, mult-rater feedback, multi-level feedback, upward appraisal, peer review.

The results of this type of feedback process provide an understanding how how the employee is perceived from different perspectives. This process helps an individual understand how others perceive them.

Feedback is essential to facilitating performance improvements. Feedback allows people to utilize their strengths to their advantage. Feedback informs employees which actions create problems for others and to know what changes may be needed.

Chapter Highlights

1. Uses
2. Benefits
3. How it is conducted
4. Requirements
5. Benefits of using the Internet
6. Demos
7. Item Bank

Principle of Multi-Rater Feedback
Provides an understanding how how the employee is perceived from different perspectives.

Multi-Rater Feedback: Uses

Uses for Multi-Rater Feedback include:

This process can also be a motivator of performance since it shows the employee that their opinions and views are considered important.

Benefits

May improve service to customers if they are able to offer feedback to the employee. Multi-Rater Feedback offers a more complete picture of the employee's performance. This feedback can provide guidance on skills that an employee may need to develop.

How it is conducted.

  1. Develop questionnaire A questionnaire used for Multi-Rater Feedback typically contains items that are rated on a 5 point scale. These items may be developed to measure different dimensions of job performance (e.g., communication, teamwork, leadership, initiative, judgment, ...). Questionnaires also typically include one or more open-ended questions to solicit written feedback.

    Questionnaires typically include from 50 to 100 items. When estimating the amount of time to complete the questionnaire you should estimate about 1 minute per questionnaire item.

    If using a printed questionnaire form, you should consider using forms that can be scanned into a computer.

  2. Ensure confidentiality of participants Steps must be taken to ensure the confidentiality of the feedback results. For example, feedback ratings from several subordinates may be combined (averaged) to mask the identity of an individual subordinate. Comments or written answers to questions may be summarized in the results to mask the identity of the author. The confidentiality helps ensure that the results are genuine.

  3. Provide training/orientation Often the feedback process involves use of one or more questionnaires, confidential information, and involvement from many different areas of an organization. Therefore, training and orientation to the feedback process is needed to facilitate a smooth feedback process. During this training/orientation, employees should be informed of what Multi-Rater Feedback is and why it is being implemented at your organization. You may want to provide samples of the questionnaire items and/or feedback results.

  4. Administer the feedback questionnaire Distribute questionnaire forms (if using printed copies) with instructions. May want to prepare answers to common questions if other employees will be assisting in the administration. If possible, post the questions and answers to your web site for easy access.

    It is important to monitor the progress through the system in order to contact employees who need to complete forms.

  5. Analyze the data Basic data analysis would include averages of ratings. More complicated analyses may include item-analysis and/or factor-analysis. Types of analyses include: Performance Dimension Summary; Summary-Performance vs. Expected; Individual Item Ratings; Item Ratings-Performance vs. Expected (normed); Highest- or Lowest-Rated Items (shows individual's strengths and weaknesses); Group & Organizational Ranking, and Recommendations for Development.

    You may want to analyze the data by organizational division or department to assess group and organizational strengths and weaknesses. This can be used to support or promote training and organizational development.

  6. Develop and Distribute Results Feedback results should be shared with the employee. It should not be mandatory that the employee share the results with their supervisor. However, you may want to make this an optional part of the performance review of the employee.

    Most results for an employee will include a comparison of their ratings to the ratings of their supervisor and and average of the ratings from others (peers, customers...). The comparisons may be in the form of numbers or simple bar charts.

    You may want to provide individual review sessions or group workshops conducted by a facilitator to help individuals review and understand the results and develop appropriate goals and objectives.

Dimensions

Responses are collected for items that fall under a specific dimension of job performance. A single questionnaire may contain dozens of questions that measure responses on one or more dimensions. Example dimensions are shown below.

Requirements?

A Multi-Rater Feedback process requires a coordinated effort to collect hundreds, or even thousands, of pieces of data. In addition, the validity of the results is enhanced by ensuring confidentiality of the participants.

The survey administration may require time (to complete the questionnaires) and computer resources to analyze the data.

Benefits of using Internet

Fortunately, there are many software companies that offer tools to assist you in conducting this type of feedback process. Some links to these companies can be found at the HR-Software network.

Demos

Try this online demo from HR-Survey.com

Item Selector

If you are interested in developing a 360 Feedback Questionnaire, this would be a good place to get started. Click the link below to access our Item Selector program. This will display a form containing a list of items. Just select the items you want included on your questionnaire and follow the instructions on the form.

Item Selector

Item Bank

Access an item bank containing over 1,600 items.

Item Bank

CSS