DEPARTMENT OF COMMERCE

PERSONNEL MANAGEMENT DEMONSTRATION PROJECT

BOOZ ALLEN INTERVIEW GUIDE FOR DEMONSTRATION GROUP INTERVIEWEES

Spring 2007

INTERVIEWEE’S NAME: _______________________

INTERVIEWEE’S TITLE: _______________________

INTERVIEWER’S NAME: _______________________

I. INTRODUCTION/BACKGROUND 

My name is ________ and this is __________. We’re from Booz Allen Hamilton in Virginia. We are here today to talk about the Department of Commerce's Personnel Management Demonstration Project. 

As you probably know, DoC initiated the Demonstration Project in March 1998 – and was extended through 2008 – as a means of testing whether a series of alternative personnel practices could be more successful than traditional personnel practices in helping DoC achieve its goals. A series of HR interventions were implemented in a subset of the organization. DoC contracted with Booz Allen to conduct a comprehensive program evaluation of the Demonstration Project’s performance in meeting its objectives.

(INTERVIEWER: Provide the following information if you think that the interviewee needs it.)  The Demonstration Project includes two groups and our assessment compares human resource practices in these two groups. The first group is made up of those work units within DoC where a series of human resources practices have been implemented under the Demonstration Project. We call this the Demo Group. The second group consists of DoC work units similar to those participating in the Demonstration Project but who have maintained the traditional human resources practices. We call this the Comparison Group. 

The purpose of today's interview is to get your perspectives on how the Demonstration Project has been operating, now that nine [four] years have passed. We are interested in talking to you because of your role as a [NAME THEIR ROLE]. [INTERVIEWER: Determine whether the person is involved with only the Demo Group, or also has an opportunity to view activity in the Comparison Group.]

Any information you share with us today will be held confidential. We will be aggregating results from all interviews and focus groups (held across the nation) and will not be attributing any findings to any individuals or any locations. Booz Allen will then use the interview data, plus information from our focus groups, survey, and objective data analysis, to evaluate the ninth year of the Demonstration Project.

We anticipate that this interview will take about 50-60 minutes. Do you have any questions before we begin?

II. INTERVIEW QUESTIONS 

Interviewers: Select the appropriate set(s) of questions based on your understanding of the interviewee's role.

HR Directors & Staff:

1. Which of these recruitment interventions (INTERVIEWER: SHOW LIST OF RECRUITMENT INTERVENTIONS), if any, have been successful in the past nine [four] years in your organization's efforts to attract and hire high quality candidates?
2. Which intervention has made the biggest impact?
3. Has the employee-position fit of new hires improved since the Demonstration Project started? Is this attributable to any of the Demonstration Project interventions? If so, which ones and why?
4. Has the hiring process—the time from application to offer made—been faster since the Demonstration Project started? If so, which steps have been more efficient?
5. Has the timing of the hiring process had any impact on DoC’s ability to hire top candidates?
6. Have you tried using paid advertising for attracting candidates? If so, has it been successful?
7. Have you used the flexible starting salaries as a tool to recruit desirable candidates? If so, has it been successful?
8. Do you think flexible starting salaries have had any impact on achieving diversity in the workplace? How so?
9. Have you noticed any difference in the quality of employees hired since the Demonstration Project began? How so?
10. To what degree are performance-based pay increases encouraging retention?
11. To what degree, if at all, are bonuses encouraging retention?
12. In what ways, if any, has offering flexible pay increases with promotions encouraged retention among high performers?
13. How have supervisors reacted to the supervisory performance pay intervention?
14. Do you believe that pay levels are competitive with other public sector organizations? Are they competitive with private sector organizations?
15. How have supervisors reacted to their increased involvement in classification since the Demonstration Project began? [INTERVIEWER: Supervisors now use the automated classification system to process their own classifications.]
16. For HR staff, is the delegated authority for classification easier than the previous system? In what ways?
17. For HR staff, does the delegated authority for classification allow classification decisions to be made faster to use then the previous system? In what ways?
18. What methods are in place to ensure the accuracy of the classification decisions? Are they working?
19. What methods are in place to ensure consistency across managers in classification decisions? Any impact on employee perceptions of fairness?
20. Have there been fewer classification decisions under the broad-banding classification system? [INTERVIEWER: The hypothesis is that fewer resources would be used. Broad banding grouped several GS grades together into a pay band; in the traditional GS system, HR would do a classification action to promote the person from one grade to another; in the Demo, movement through the pay band does not require a classification action.]
21. Has anything happened in this organization or in the geographic area that had an effect on your ability to hire, pay, and retain staff? [INTERVIEWER: Examples could be a hiring freeze, a reorganization, a RIF or other downsizing, a major employer in the area laying off lots of employees that then become available as candidates to DoC.]
22. Have there been any challenges with ensuring that the Demonstration Project interventions are consistent with workforce diversity objectives?
23. Have employees (supervisory or non-supervisory) raised any concerns regarding the Demonstration Project?

Pay Pool Managers:

1. Please describe the process that rating officials go through for determining employees' performance scores.
2. Please discuss how they evaluate employees against the performance elements and contribute to decision-making about performance-based increases and bonuses.
3. In what ways, if any, could this performance appraisal process be improved?
4. What challenges have you faced in arraying data on performance scores across rating officials and across the pay pool (rather than ranking) to maintain the linkage between scores and pay actions? (INTERVIEWERS: Be aware that because many employees felt that the assignment of numerical rankings created a competitive environment, the Demonstration Project has since eliminated the individual rankings and now arrays the data in score order to maintain the linkage between scores and pay actions without assigning a numerical rank.)
5. What challenges have you faced in using the performance pay table? Can you recommend any improvements for the table?
6. How do you make determinations regarding the distribution of performance-based bonuses?
7. How frequently do you have to override managers’ compensation requests?
8. Does the process you use help to differentiate between high and lower performers?
9. What is the extent of your communication (frequency, mode) with rating officials regarding performance scores and pay issues? What are some common issues you discuss?
10. Over the past 9 [4] years, have you seen any rating inflation? If yes, how is it controlled?
11. Have you found that rating officials are generally consistent in how they make ratings?
12. What mechanisms are in place to ensure consistency in scores across rating officials?
13. In your role as pay pool manager, what do you do to ensure consistency across the pay pool? What do other pay pool managers do?
14. What is the extent of your communication (frequency, mode) with other pay pool managers regarding performance scores and pay issues?
15. What mechanisms are in place to provide employees with performance feedback or otherwise involve them in the performance management process?
16. Did you feel sufficiently prepared or trained to perform as a pay pool manager when you took on these responsibilities? How do you feel now?
17. What would you recommend to prepare pay pool managers for their role in ensuring consistency across pay pools?
18. What are some common problems or concerns that you have dealt with? How do you deal with them? What’s working well?
19. In what ways, if at all, do the interventions implemented under the Demonstration Project encourage supervisors to perform better as supervisors?
20. In what ways, if at all, do the interventions implemented under the Demonstration Project encourage employees to perform better? Are there any ways in which these interventions are inhibiting employee performance?
21. How are measures used in your organization in making pay for performance decisions?
22. Do higher performing employees receive higher pay raises or bonuses? Do you have any suggestions for ways to improve the link between performance and pay?
23. Has the employee-position fit of new hires improved since the Demonstration Project started? Is this attributable to any of the Demonstration Project interventions? If so, which ones and why?
24. Has the hiring process—the time from application to offer made—been faster since the Demonstration Project started? If so, which steps have been more efficient?
25. Do you think flexible starting salaries have had any impact on achieving diversity in the workplace? How so?
26. Have you noticed any difference in the quality of employees hired since the Demonstration Project began? How so?
27. To what degree are performance-based pay increases encouraging retention?
28. To what degree are bonuses encouraging retention?
29. In what ways, if any, has offering flexible pay increases with promotions encouraged retention among high performers?
30. How have supervisors reacted to the supervisory performance pay intervention?

Rating Officials:

1. Please describe the process you go through for determining employees' performance scores. Please discuss how you evaluate employees against the performance elements and make decisions about performance-based increases and bonuses.
2. What are the main benefits and drawbacks of the 100-point performance appraisal system?
3. What are some common problems or concerns that you have dealt with? How do you deal with them? What’s working well?
4. How frequently do you have to override managers’ rating or compensation requests?
5. What is the extent of your communication (frequency, mode) with other rating officials? What kinds of issues, if any, do you discuss?
6. Do pay pool managers encourage frequent communication between rating officials? If yes, how so? If no, why not?
7. Are there mechanisms in place to ensure consistency in performance scores across rating officials? If yes, how are they working?
8. What is the extent of your communication (frequency, mode) with the pay pool manager regarding performance scores and pay issues? What kinds of issues do you discuss?
9. What mechanisms are in place, if any, to provide employees with performance feedback or otherwise involve them in the performance management process?
10. How effective are the mid-year reviews with employees?
11. In what ways do the interventions implemented under the Demonstration Project encourage supervisors to perform better as supervisors? Is there any intervention that is inhibiting supervisor performance?
12. In what ways do the interventions implemented under the Demonstration Project encourage employees to perform better? Is there any intervention that is inhibiting employee performance?
13. Do higher performing employees actually receive higher raises or bonuses? Do you have any suggestions for ways to improve the link between performance and pay?
14. How do you feel about your increased involvement in the classification process?
15. To what degree is the automated classification system (ACS) easy to use?
16. To what degree is the automated classification system (ACS) time-consuming?
17. How do you feel about your increased involvement in the classification process?
18. Did you feel sufficiently prepared or trained to perform the classification function at the beginning of the Demonstration Project? How do you feel now?
19. Which of these recruitment interventions (INTERVIEWER: SHOW LIST OF RECRUITMENT INTERVENTIONS), if any, have been successful in the past five years in your organization's efforts to attract and hire high quality candidates?
20. Which of these interventions has had the biggest impact?
21. Has the employee-position fit of new hires improved since the Demonstration Project started? Is this attributable to any of the Demonstration Project interventions? If so, which ones and why?
22. Has the timing of the hiring process had any impact on DoC’s ability to hire top candidates?
23. Have you used the flexible starting salaries as a tool to recruit desirable candidates? If so, has it been successful?
24. Have you noticed any difference in the quality of employees hired since the Demonstration Project began? How so?
25. To what degree are performance-based pay increases encouraging retention?
26. To what degree are bonuses encouraging retention?
27. In what ways, if any, has offering flexible pay increases with promotions encouraged retention among high performers?
28. How have supervisors reacted to the supervisory performance pay intervention?
29. Do you believe that pay levels are competitive with other public sector organizations? Are they competitive with private sector organizations?

Directors & Administrative Officers:

1. Which of these recruitment interventions (INTERVIEWER: SHOW LIST OF RECRUITMENT INTERVENTIONS), if any, have been successful in the past nine [four] years in your organization's efforts to attract and hire high quality candidates?
2. Which of these retention interventions (INTERVIEWER: SHOW LIST OF RETENTION INTERVENTIONS), if any, have been successful in the past five years in your organization's efforts to retain high performing employees?
3. Have managers performed effectively as pay pool managers?
4. Have there been any challenges to acclimating managers to this role? Have there been any difficulties keeping managers in this role?
5. Have managers performed effectively as rating officials? Have there been any challenges to acclimating them to this role? Have there been any difficulties keeping managers in this role?
6. Have you noticed improvements in individual performance since the Demonstration Project began? If yes, do you think this is attributable to any of the interventions? If so, which ones and how?
7. Have you noticed improvements in organizational performance since the Demonstration Project began? If yes, do you think this is attributable to any of the interventions? If so, which ones and how?
8. Have there been any challenges with ensuring that the Demonstration Project interventions are consistent with workforce diversity objectives?
9. In hindsight, would you do anything differently in terms of how the objectives and processes of the Demonstration Project were initially communicated to employees and supervisors?
10. Has the Demonstration Project caused any issues for management that were not foreseen?
11. Have there been any unanticipated benefits of the Demonstration Project? Any unanticipated drawbacks?
12. Do you have any suggestions for ways in which the Demonstration Project interventions could be improved?
13. Has anything happened in this organization or in the geographic area that had an effect on your ability to hire, pay, and retain staff? [INTERVIEWER: Examples could be a hiring freeze, a reorganization, a RIF or other downsizing, a major employer in the area laying off lots of employees that then become available as candidates to DoC.]

EEO Staff or Representatives:

1. Do you feel that any of the Demonstration Project interventions have had a positive impact on minorities, women, or veterans in the Demonstration Project? If yes, how? (INTERVIEWER: SHOW LIST OF INTERVENTIONS)
2. Do you feel that any of the Demonstration Project interventions have had a negative impact on minorities, women, or veterans in the Demonstration Project? If yes, how?
3. What are the biggest challenges with ensuring that the Demonstration Project interventions are consistent with workforce diversity objectives?
4. Have employees (supervisory or non-supervisory) raised any concerns regarding the Demonstration Project?
5. Are there any differences in how the nine Merit System Principles are followed under the Demonstration Project versus how they are typically followed elsewhere in DoC? If yes, how?
6. Are there any differences in how the twelve Prohibited Personnel Practices are avoided under the Demonstration Project versus how they are typically avoided elsewhere in DoC? If yes, how?
7. Overall, does the Demonstration Project promote building and maintaining a diverse workforce?
8. Under the Demonstration Project, are the contributions of all employees respected and maximized?