An Integrated Framework to Quantify the Impact of Competency factors on Project Performance

Abstract

Construction companies measure their performance by how well a construction project delivers relative to their goals. The performance of construction projects is governed by merging many events and interactions with different participants and processes in a constantly changing environment. Many such occurrences and interactions are defined as "competencies”. These competencies are the inputs that can be quantified and used to distinguish between the poorest-performing and best-performing projects. Evaluation of project performance has gained significant attention in the construction domain. Within this conversation, one difficulty is that project management literature review does not provide a coherent nor commonly accepted definition of project success. A project's success could be defined differently depending on the stakeholders, since every individual has a subjective set of success requirements. Thus, determining whether a project is a success is a complex problem and a project cannot always be seen as a total successful or complete failure. In the current construction industry, much effort has been spent on precisely measuring and controlling individual performance metrics (i.e. cost) and determining performance indices for them. However, there have been far fewer comprehensive indices that evaluate project performance across numerous performance metrics such as cost, schedule, safety, quality, and communication. Many previous studies have used these performance metrics as ex-post lagging measures to assess the performance of completed projects. Yet, these metrics do not leverage any opportunity to improve the success probability early in the project and serve only as a historic review. What is crucial, but often missed, is that the performance of a construction project is driven by diverse and numerous leading factors that occur among and across project phases. These factors are defined in this research as “competency factors.” Project performance and competency factors are two distinct terms, thus not interchangeable. Project competencies are leading factors to project success, hence forecasting tools enabling project managers to predict successful projects. Identifying competency factors is equivalent to asking, “what does it take to be successful in the project?” while investigating performance measures asks, “how will we know whether the project has been successful or not?” In other words, competency factors are leading indicators that focus on achieving success, while project performance focuses on identifying an actual project’s success status. Identifying project competencies early in the project can help contractors to better manage construction projects by understanding what causes projects to achieve its goals and outcomes Previous research also focused on either a limited set of project competencies, a single phase of construction, or both without combining the project competencies into an integrated framework to measure project success. Most of these studies undermined how the competency factors selection process attempts to capture the main events and interactions that occur throughout the main project phases. Additionally, most of the studies either lacked a quantitative assessment of project competencies or overlooked the unequal contributions of different competencies to project performance. Previous research has considered, in many situations, how project competencies are a measure of project performance. But. they did not investigate project competencies as a prerequisite for project performance, or the fact that project competencies are leading indicators of project performance improvement. In addition, none of the previous studies evaluated the relative impact of the different project competencies on project performance. Therefore, to overcome these limitations of previous research, this thesis develops a comprehensive framework and methodology for evaluating project competencies and identifying their relationship to project performance. The scope of this research is to comprehensively investigate project competencies as leading indicators of project performance by using two robust, data-driven weighting mathematical frameworks. The objectives are to 1) create a comprehensive performance score named the Project Performance Assessment (PPA) score by combining eight performance metrics spanning over six performance areas: cost, schedule, productivity, safety, quality, and communication; 2) benchmark successful projects from less-than-successful projects by performing K-means clustering; 3) compute a customized competency category-specific score from various competency factors spanning nine categories: alignment & team integration, planning & design, procurement management, risk management, health, safety & environmental, quality management, labor productivity, execution & commissioning, and change management & project control; 4) assess the overall effect of each increased competency category-specific score on the likelihood of a project being a successful or less-than successful project; 5) quantitatively assess the relative contribution of each competency category to the PPA score and provide one unique comprehensive score named the Competency Category Assessment (CCA) Score; 6) utilize a developed regression model to examine the degree of association between the CCA score and the PPA score, which construction practitioners can use to predict the overall performance score of a project. The final stage is to assess the models’ robustness and validation to determine their reliability and reasonability, as well as testing the models’ future predictability. This study will allow construction practitioners to comprehensively assess both their project’s competencies and performance using precise mathematics. Thus, the results of this study can assist the construction industry in accurately identifying leverage training and development areas for improvement.

The results of this analysis showed that the top three metrics are: construction schedule growth, construction cost growth, and % delta having 18.43%, 17.38%, and 15.22%, respectively. On the other hand, the percentage of rework and the number of lost-time injuries per 1 million dollars have the lowest contribution in calculating the PPA score. The analysis also concluded that successful projects outperform less-than-successful projects, as they have 16% average cost savings, 42% average schedule reduction, 10% average productivity improvement, 7% average less rework and 66% average increase in communication efficiency. The top-weighted competency factors in each category were found to be: 1) project team has the expertise to take on the project; 2) IFC drawings are issued to support the construction activity; 3) all bulky materials needed for the start of construction are purchased & onsite; 4) risks impacting the construction phase of the project are well assessed and the risk monitoring and response framework are well defined in terms of cost; 5) quality control plan is communicated among the project team; 6) a plan to secure and retain skilled labor and a craft evaluation/qualification program are in place; 7) a coordination plan is well defined between all project parties; 8) change management roles and responsibilities are well defined to project parties. After computing the competency-category specific scores for the eight categories, it was found that successful projects outperform less-than-successful projects by an average of 1.3×-1.9× in all eight categories. This shows that being competent in all categories will increase the likelihood of a project being successful. The study also highlighted the importance of pre-construction planning & design and change management & project control towards project success, as they are the highest weighted categories, 18.61%, and 16.37%, respectively. Therefore, the construction industry should focus more on preconstruction activities because they are a critical link to all future operations of a project. Moreover, the median/mean for CCA score for successful projects (83.2/80.4%) is found to be higher than less-than-successful projects (48.1/44.4%). Last, but not least, regression analysis is performed to examine the degree of association between the CCA score and PPA score; given that the assumptions of regression analysis are met, the derived model has a coefficient of multiple determination (𝑅𝑅2) of 51.1% and correlation accuracy of 71.5%. Model validation is conducted using five-fold cross validation and the testing of 20 new projects introduced to the model to check the reliability and reasonability of the developed model.