A Government Accountability Office report found that the Department of Veterans Affairs lacks sufficient data on its VET TEC pilot to “assess the effectiveness of the program at getting veterans into jobs.”
The Department of Veterans Affairs needs to do a better job of analyzing the effectiveness of its “promising” technology education pilot program to determine how well it is helping veterans find employment, according to a report released by the Government Accountability Office on Thursday.
GAO’s report analyzed the VA’s Veterans Employment Through Technology Education Courses—or VET TEC—pilot program, which is open to veterans who qualify for education assistance under the GI Bill. The report said the pilot is designed “to help veterans quickly acquire high-technology skills in the fields of computer programming, computer software, media applications, data processing or information sciences.”
VET TEC was created after passage of the Harry W. Colmery Veterans Educational Assistance Act—also known as the Forever GI Bill—in 2017, which directed the department to establish a pilot program to “provide eligible veterans with the opportunity to enroll in high technology programs of education.” The five-year pilot, which officially launched in 2019 with $15 million in annual funding, has since received additional financial support from Congress, with the Consolidated Appropriations Act—which was signed into law in March 2022—authorizing an additional $80 million for the program.
GAO found that over 6,700 veterans enrolled in VET TEC from May 2019 through May 2022, with approximately 66% of enrollees completing the program. But the report noted that VA was unable to effectively determine the employment rate of the program’s graduates, meaning that “VA may be unintentionally sharing inaccurate information about the success of VET TEC at employing veterans who complete the program.”
“VA calculates an employment measure for certain VET TEC participants for whom VA has made a final milestone payment decision,” the report said. “However, VA does not calculate an employment rate for all VET TEC participants who completed the program, in accordance with other government and industry approaches. As a result, VA lacks sufficient information to compare VET TEC to other programs or to assess the effectiveness of the program at getting veterans into jobs.”
Despite the VA saying that 66% of all VET TEC participants were employed 180 days or more after completing the program, GAO’s analysis—which included all VET TEC participants, regardless of when they completed their training—found that only 46% of program graduates were employed.
The report also noted that VA was also unable to account for the 13% of veterans who dropped out of the program, since VA officials “do not systematically collect and analyze data to understand why some enrollees drop out of training.”
A previous GAO report from February 2022 that provided “preliminary observations” of the pilot noted similar concerns about the program, finding at the time that “VA has not yet implemented most leading practices for effective pilot program design with the exception of communicating with stakeholders.”
“Specifically, VA does not have documented, measurable objectives for the program,” the previous report said. “As such, VA cannot yet determine an assessment methodology or evaluation plan to measure performance of the pilot.”
GAO’s latest report identified a number of positive benefits of the pilot, including its high completion rates and high levels of diverse enrollees. The report found that veterans enrolled in VET TEC were “generally more racially and ethnically diverse and more likely to have a service-connected condition,” such as an illness or injury incurred or worsened during their military service, than “working-age veterans.”
But GAO once again found that the lack of “consistent, clear, and measurable program objectives” made it difficult to determine the program’s effectiveness.
“As a result, assessing and evaluating VET TEC by the end of the pilot will likely be difficult for VA,” the report said.
GAO made six recommendations for the VA, including for the department to “develop an employment rate calculation consistent with standard approaches” and “determine data needed to fully inform employment outcomes.”
VA concurred with five of GAO’s recommendations but did not agree or disagree with the recommendation to create a standardized employment rate calculation, saying that it was “conducting an environmental scan of employment rate calculations used in similar programs, to assess the feasibility, applicability and whether it is advisable for implementation with the VET TEC program.”
Source: https://www.nextgov.com/emerging-tech/2022/10/va-needs-better-data-employment-outcomes-promising-tech-training-pilot-watchdog-says/379120/
