Towards the improvement of project team performance based on large language models

Mykyta Rohovyi, Marina Grinchenko

Abstract


The subject of the study is a method for identifying poor quality project sprint task descriptions to improve team performance and reduce project risks. The purpose of the study is to improve the quality of textual descriptions of sprint tasks in tracking systems by implementing models for identifying and improving potentially poor task descriptions. Research Questions: 1. Can poor quality project sprint task descriptions be identified using clustering? 2. How to utilize the power of large language models (LLMs) to identify and improve textual descriptions of tasks? Objectives: to analyze research on approaches to improving descriptions using clustering and visualization techniques for project tasks, to collect and prepare textual descriptions of sprint tasks, to identify potentially poor task descriptions based on clustering their vector representations, to study the effect of prompts on obtaining vector representations of tasks, to improve task descriptions using LLMs, and to develop a technique for improving project team effectiveness based on LLMs. Methods of vector representation of texts, methods of dimensionality reduction of PCA and t-SNE data space, methods of agglomerative clustering, methods of prompting were used. The following results were obtained. An approach to improving the performance of the project team based on the use of LLM was proposed. Answering the first research question, it was found that there are no linguistic features affecting the perception of textual descriptions of project sprint tasks. In response to the second research question, a model for identifying potentially poor task descriptions is proposed to reduce project risks associated with misunderstanding of task context. Conclusions. The results suggest that project sprint task descriptions can be improved by using large-scale language models for project team understanding. Future research recommends using project source documentation and project context as a vector repository and source of context for LLM. The next step is to integrate the LLM into the project task tracking system.

Keywords


project; project team; task description; project task management system; model; neural network; large language model

Full Text:

PDF

References


The Scrum Guide. The Definitive Guide to Scrum: The Rules of the Game. Available at: https://scrumguides.org/docs/scrumguide/v2020/2020-Scrum-Guide-US.pdf#zoom=100 (аccessed 10.05.2024).

Cleland, D. L., & Gareis, R. Learn and apply successful international project management techniques. McGraw Hill Professional, 2010. 575 p.

Strielkina, A., Tetskyi, A., & Krasilshchykova, V. Risk and uncertainty assessment in software project management: integrating decision trees and monte carlo modeling. Radioelectronic and Computer Systems, 2023, no. 3(107). pp. 217-225. DOI: 10.32620/reks.2023.3.17.

Jira Software. [The IT industry]. Available at: https://www.atlassian.com/software/jira. (аccessed 10.05.2024).

Covey, S. R. The 7 Habits of Highly Effective People: Powerful Lessons in Personal Change. Simon and Schuster, 2004. 372 p.

Kalyan, K. S. A survey of GPT-3 family large language models including ChatGPT and GPT-4. Natural Language Processing Journal, 2024, vol. 6, pp. 100048. DOI: 10.1016/j.nlp.2023.100048.

Agathokleous, E., Rillig, M., Peñuelas, J., & Yu, Z. One hundred important questions facing plant science derived using a large language model. Trends in Plant Science, 2023, vol. 29, no. 2, pp. 210-218. DOI: 10.1016/j.tplants.2023.06.008.

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., & et al. Language Models are Few-Shot Learners. arxiv, Computation and Language, 2020, pp. 1-75. DOI: 10.48550/arXiv.2005.14165.

Cherednichenko, O., Ivashchenko, O., Lincényi, M., & Kováč, M. Information technology for intellectual analysis of item descriptions in e-commerce. Entrepreneurship and Sustainability Issues, 2023, vol. 11, no. 1, pp. 178-190. DOI: 10.9770/jesi.2023.11.1(10).

Peggy, G., Strode, D., Sharp, H., & Barroca, L. An onboarding model for integrating newcomers into agile project teams. Information and Software Technology. 2022, vol. 143, article no. 106792. DOI: 10.1016/j.infsof.2021.106792.

Femmer, H., Fernández, D., Wagner, S., & Eder, S. Rapid quality assurance with Requirements Smells. Journal of Systems and Software, 2017, vol. 123, pp. 190-213. DOI: 10.1016/j.jss.2016.02.047.

Dalpiaz, F., Gieske, P., & Sturm, A. On deriving conceptual models from user requirements: An empirical study. Information and Software Technology, 2021, vol. 131, article no. 106484. DOI: 10.1016/j.infsof.2020.106484.

Li, Z., Wang, X., Jing, W., Wu, J., Zhang, Z., Liu, Z., Sun, M., Hui, Z., & Liu, S. A Unified Understanding of Deep NLP Models for Text Classification, Available at: https://arxiv.org/abs/2206.09355 (accessed 08.05.2024).

Ishizuka, R., Washizaki, H., Tsuda, N., Fukazawa, Y., Ouji, S., Saito, S., & Iimura, Y. Categorization and Visualization of Issue Tickets to Support Understanding of Implemented Features in Software Development Projects, Applied Sciences, 2022, no. 12, iss. 7, article no. 3222. DOI: 10.3390/app12073222.

Briciu, A., Czibula, G., & Lupea. M. A study on the relevance of semantic features extracted using BERT-based language models for enhancing the performance of software defect classifiers. Procedia Computer Science, 2023, vol. 225, pp. 1601-1610. DOI: 10.1016/j.procs.2023.10.149.

Abdu, A., Zhai, Z., Algabri, R., Abdo, H., Hamad, K., & Al-antari, M. Deep learning-based software defect prediction via semantic key features of source code systematic survey. Mathematics, 2022, vol. 10, no. 17, article no. 3120. DOI: 10.3390/math10173120.

Chawla, P., Hazarika, S., & Shen, H.-W. Token-wise sentiment decomposition for convnet: Visualizing a sentiment classifier. Visual Informatics, 2020, vol. 4, iss. 2, pp. 132-141. DOI: 10.1016/j.visinf.2020.04.006.

Frattini, J. Identifying Relevant Factors of Requirements Quality: An Industrial Case Study. Requirements Engineering: Foundation for Software Quality, 2024. vol. 14588, pp. 20-36. DOI: 10.1007/978-3-031-57327-9_2.

Moharil, A., & Sharma, A. TABASCO: A transformer based contextualization toolkit. Science of Computer Programming, 2023, vol. 230, article no. 102994. DOI: 10.1016/j.scico.2023.102994.

Sonbol, R., Rebdawi, G., & Ghneim, N. Learning software requirements syntax: An unsupervised approach to recognize templates. Knowledge-Based Systems, 2022, vol. 248, article no. 108933. DOI: 10.1016/j.knosys.2022.108933.

Berhanu, F., & Alemneh, E. Classification and Prioritization of Requirements Smells Using Machine Learning Techniques. Conference Proceedings: 2023 International Conference on Information and Communication Technology for Development for Africa (ICT4DA). IEEE, 2023, pp. 49-54. DOI: 10.1109/ICT4DA59526.2023.10302263.

The Public Jira Dataset. Available at: https://zenodo.org/records/5901804. (accessed 15.05.2024).

Devlin, J., Chang, M., Lee, K., & Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2019, vol. 1, pp. 4171-4186. DOI: 10.18653/v1/N19-1423.

Grinchenko, M., & Rohovyi, M. A model for identifying project sprint tasks based on their description. Innovative Technologies and Scientific Solutions for Industries, 2023, no. 4 (26), pp. 33-44. DOI: 10.30837/ITSSI.2023.26.033.

Yuanzhi Li, Bubeck, S., Eldan, R., Del Giorno, A., Gunasekar, S., & Lee, Y. Textbooks Are All You Need II: phi-1.5 technical report. Microsoft Research, 2023, pp. 1-16. DOI: 10.48550/arXiv.2309.05463.

Nielsen, F. Hierarchical Clustering. Introduction to HPC with MPI for Data Science, Springer, Chapter, 2016, ISSN 1863-7310, pp. 221-239. DOI: 10.1007/978-3-319-21903-5_8.

Jolliffe, I. T., & Cadima, J. Principal component analysis: a review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 2016. vol. 374, iss. 2065, pp. 1-16. DOI: 10.1098/rsta.2015.0202.

Linderman, G., & Steinerberger, S. Clustering with t-SNE, Provably, SIAM Journal on Mathematics of Data Science, 2019, vol. 1, iss. 2, pp. 313-332. DOI: 10.1137/18M1216134.

Rohovyi, M., & Grinchenko, M. Project team management model under risk conditions. Bulletin of the National Technical University "KhPI". Series: Strategic Management, Portfolio, Program and Project Management, 2023, no. 1(7), pp. 3-11. DOI: 10.20998/2413-3000.2023.7.1.




DOI: https://doi.org/10.32620/reks.2024.4.19

Refbacks

  • There are currently no refbacks.