Following are GAO’s written responses to questions after a House hearing earlier this year on the national security personnel system at DoD.
GAO noted that NSPS was implemented too quickly.
What steps could have been taken to roll out NSPS in a more orderly and fair fashion?
As we have previously reported, we support the need to expand broad banding approaches and pay-for-performance-based systems in the federal government.2 However, moving too quickly or prematurely to implement such programs, whether at DOD or elsewhere, can significantly raise the risk of doing it incorrectly. Hasty implementation could also set back the legitimate need to move to a more performance- and results-based system for the federal government as a whole. Thus, while it is imperative that we take steps to better link employee pay to performance across the federal government, how it is done, when it is done, and the basis on which it is done can make all the difference in whether or not such efforts are successful. In our view, one key need is to modernize performance management systems in executive agencies so that they are capable of adequately supporting more performance-based pay and other personnel decisions.
While our previous work does not prescribe a process and time frames for rolling out systems such as NSPS, we have stressed that agencies should have an institutional infrastructure in place that would include, at a minimum, (1) a human capital planning process that integrates the agency’s human capital policies, strategies, and programs with its program goals and mission and desired outcomes; (2) the capabilities to effectively develop and implement a new human capital system; and (3) the existence of a modern, effective, and credible performance management system that includes adequate safeguards, including reasonable transparency and appropriate accountability mechanisms, to ensure the fair, effective, and nondiscriminatory implementation of a new system. Prior to NSPS implementation, we cautioned that, while the DOD leadership had the intent and the ability to implement the needed infrastructure, it did not have the necessary infrastructure in place across the department.
Further, our work has continued to stress the importance of incorporating internal safeguards into the design and implementation of large-scale pay-for-performance programs. In 2008, we evaluated DOD’s efforts to implement nine safeguards and accountability mechanisms.3 We found that, while DOD had taken some steps to implement internal safeguards to ensure that NSPS is fair, effective, and credible, the implementation of some safeguards could be improved. First, DOD does not require a third party to analyze rating results for anomalies prior to finalizing employee ratings, and therefore it is unable to ensure that ratings are fair and nondiscriminatory before they are finalized. Second, the process has lacked transparency until recently because DOD did not require commands to publish final rating distributions, though doing so was recognized as a best practice by NSPS program officials at all four components.
In 2008, the department revised its NSPS regulations and guidance to require commands to publish the final overall rating results. Third, NSPS guidance may discourage rating officials from making meaningful distinctions in employee ratings because it indicated that the majority of employees should be rated at the "3" level, on a scale of 1 to 5, resulting in a hesitancy to award ratings in other categories. We continue to believe that improved implementation of these safeguards will help bolster employee confidence in the system and ensure that the system is fair, effective, and credible.
If NSPS continues, what steps should now be taken to move forward?
We have previously reported that converting to NSPS was a significant transition for the department.4 We have further reported that it will take time for employees to accept the system, based on the studies conducted by the Office of Personnel Management (OPM) on the federal government demonstration projects for performance management. First, and foremost, DOD needs to assess and address employee engagement in the system. DOD has collected survey data and conducted focus groups of employees under NSPS, but it is missing a key piece—an action plan.
Our 2008 report recommended that DOD develop and implement a specific action plan to address employee perceptions of NSPS ascertained from feedback avenues such as, but not limited to, DOD’s survey and DOD’s and GAO’s employee focus groups. At a minimum, this plan should include actions to mitigate employee concerns about the potential influence that employees’ and supervisors’ writing skills have on the panels’ assessment of employee ratings and the lack of transparency and understanding of the pay pool panel process. Such a plan would demonstrate to employees that the department is listening to their concerns and making plans to address, as appropriate, the concerns that are identified. In short, DOD needs to tell the employees that they are going to take action on their concerns. In addition, the recently announced study by DOD and OPM is an opportunity to assess the status of the system. While the review intends to include a thorough examination of all NSPS policies, regulations, and practices, we would like to see DOD leverage this opportunity to assess for itself how the department is implementing internal safeguards. Specifically, we are interested in an update of how the safeguards have been incorporated into their policies and how the safeguards are working.
The Deputy Secretary of Defense, in cooperation with the Office of Personnel Management recently announced that the department would halt conversions of DOD civilian employees to NSPS, pending the outcome of a review by DOD and OPM. The proposed review will assess whether or not NSPS is meeting its objectives of being a fair, transparent, and effective personnel system. Finalizing the details of such a review’s overall framework, scope, timeline, and leadership will take time.
As DOD and OPM leadership hold discussions to determine the overall framework, scope, and timeline of the review, what guidance or suggestions would you give to DOD and OPM to include in the methodology of this study?
As we have previously reported, the extent to which DOD incorporates internal safeguards into the design and implementation of NSPS and how it addresses employee perceptions of NSPS are key to the success of the system.5 Moving forward, as DOD and OPM embark on a study of NSPS and review how NSPS operates and its underlying policies, DOD has a unique opportunity to consider our previous recommendations, as well as all of the other internal safeguards key to ensuring that performance management systems in the government are fair, effective, and credible.
In addition to a review of internal safeguards, this study provides DOD the opportunity to look at employee engagement in the process and develop an action plan to address employee concerns about NSPS. As we approached our work, we used a methodology that systematically took into account employee input from all levels. We used a combination of survey analysis, interviews, and employee discussion groups to obtain information on employee perceptions. In general, the combination of employee surveys with interviews or discussion groups is helpful because it yields useful information at the population level, as well as the individual employee experience level.
One concern expressed by employees who have converted from GS to NSPS is that there is no real career progression. Under the GS system, an employee steadily moves up through the various grades and can actually monitor actual career progression. There appears to be no such similar movement in NSPS; an employee, while receiving pay increases and bonuses, may remain in the same pay band for his/her entire career.If this is a valid concern, how can it be addressed, if NSPS continues?
First, DOD needs to collect more information on what the issues are surrounding this employee perception on career progression, including the underlying causes and the extent of this concern, so that the department can determine if it is indeed a valid concern. For example, is there an issue with lack of career progression or are employees perceiving that there is an issue as a result of lack of communication or education on the new system? In our 2008 report, we recommended that the department develop and implement a specific action plan to address employee perceptions of NSPS ascertained from feedback avenues such as, but not limited to, DOD’s survey and DOD’s and GAO’s employee focus groups. We believe that this is another example of how the department could use such an action plan to guide its approach for addressing employee concerns. Specifically, the plan may incorporate various communication and education strategies to help employees understand how the shift from pay grades to pay bands still affords them opportunities for professional development, as well as movement through the pay band. While we acknowledge that change takes time to gain employee acceptance and that the implementation of NSPS is a large-scale organizational transformation, employee concerns, such as these, must be heard and addressed accordingly in order to ensure greater employee acceptance and, ultimately, successful implementation of the NSPS performance management system.
Based on the GAO testimony, one safeguard GAO believes needs to be implemented to increase employee confidence in the pay for performance system is for DOD to have a third party analyze the pay-pool recommendations for "anomalies" before any final decision is made to determine whether an employee’s rating accurately reflects the employee’s performance and whether any non-merit based factors contributed to the "anomaly." Explain how you see this third party analysis working.
Given that each agency has its own set of unique challenges and its own approach for handling those challenges, we believe that the department is in the best position to determine how to appropriately design and implement a predecisional analysis for NSPS. That said, we believe that the third-party analysis should be conducted by an independent reviewing office, such as a human capital office, that is able to conduct the analysis outside of the chain of command. Taking the analysis outside of the chain of command helps to ensure that the process remains as independent as possible. Seeing that DOD currently has over 200,000 civilian employees under NSPS, the department could consider phasing in the third-party analysis by starting with a representative sample of employees. A phased implementation approach recognizes that different components of agencies will often have different levels of readiness and different capabilities to implement new authorities. Moreover, a phased approach allows for learning so that appropriate adjustments and midcourse corrections can be made before new policies and procedures are fully implemented organizationwide.
In your view, who would the third party be, a DOD entity or a non-DOD entity?
As noted in our response to question 1, we would expect the third party to be a DOD entity that is removed from the chain of command—that is, the human capital office or an office of opportunity and inclusiveness.
What criteria does GAO see as constituting an anomaly?
Generally, an anomaly would be characterized as a set of ratings for which there is a statistically significant difference in comparison to the larger group. For example, if the data indicated that there was a statistically significant difference between the ratings of a particular subset of the larger workforce compared to the larger group at that same level, this could constitute an anomaly. The presence of an anomaly is not alone proof that there is a problem. Rather, identifying an anomaly in the data prior to finalizing the rating decisions would enable management to investigate the situation and determine whether the results are justified and merit-based.
In investigating "blatant discrimination" or "egregious decisions" would the employee be contacted and interviewed?
The predecisional reviews are to help achieve consistency in the performance management process and provide reasonable assurance that the performance decisions are merit-based and fair. Due to the nature of the investigation, the employee would not be contacted during an investigation. However, information provided by employees, such as the self-assessment, can be considered during the review process, as could information provided by responsible managers regarding underlying reasons for any anomalies.
Would a single third party be evaluating all 1,600 pay pools across DOD to get a DOD wide view of anomalies, or would 1,600 third-party reviews be conducted at each pay pool without regard for a comprehensive DOD look?
As noted in our response to question 1, each agency has its own set of unique challenges and its own approach for handling those challenges. As a result, we feel that the department is in the best position to determine how to appropriately design and implement a predecisional analysis for NSPS. One approach, as noted in our response to question 1, would be for DOD to phase in the third party analysis by starting with a representative sample of employees. Such an approach recognizes that different levels of readiness and different capabilities exist among agency components and allows for learning so that appropriate adjustments and midcourse corrections can be made before full implementation.
What effect would the third-party analyses have on the timeliness of the pay-pool process?
It would likely add time to the existing process. However, we believe that it is important that DOD take steps to ensure that its employees’ ratings are perceived as fair reflections of their performance. Taking additional time to complete a predecisional analysis is one safeguard that DOD can implement to raise employee confidence in the fairness and credibility of the system.
Would the pay pool decisions on all the other employees in the pay pool be held up until the "anomaly" was resolved?
Given that the predecisional review is intended to take place prior to the ratings being finalized but before they are certified and released to employees, all other ratings would not be released until the predecisional review was completed and appropriate responses (which could include inaction) were determined for any anomalies identified.
How would GAO see the anomaly being corrected — a directive to the rater to change the rating, or some disciplinary action against the rater, or some other form of corrective action?
Where managers provide information that explains the merit-based factors and reasons for the anomalies, the managers would not change the ratings. On the other hand, managers could determine that some vital information was not considered that would provide a basis for changing the rating. In all cases, it is the unit manager, not the third party conducting the predecisional review, that would determine whether a change would be warranted. Further, the review is not intended to change the results to portray an "ideal" distribution, or to alter the outcome of the performance management process. The purpose of the predecisional review is to identify if anomalies exist and, if found, inform managers of the need for further review to provide reasonable assurance that the basis for each rating is fair, credible, and meritbased.
Does GAO see any appeal rights for the manager or employee involved in the "anomaly"?
Although the third-party reviewer identifies the anomalies, it is the responsible manager that examines the basis underlying the ratings and is held accountable for ensuring the ratings are merit based. While it is unlikely, should a manager be disciplined, he or she might have appeal rights, depending on the nature of the discipline imposed.
Any employee has a right to appeal his or her final rating. However, these appeal rights do not apply to the predecisional review process since it is intended to take place prior to the ratings being finalized but before they are certified and released to employees.