Науковий вісник НГУ

Software detection of Ukrainian-language texts generated by AI: methods, estimations, challenges

• 
• 

User Rating:  / 0

PoorBest 

Details

Category: Content №4 2025

Last Updated on 26 August 2025

Published on 30 November -0001

Hits: 3656

Authors:

S. D. Prykhodchenko*, orcid.org/0000-0002-6562-0601, Dnipro University of Technology, Dnipro, Ukraine, This email address is being protected from spambots. You need JavaScript enabled to view it.

O. Yu. Prykhodchenko, orcid.org/0000-0001-5080-737X, Dnipro University of Technology, Dnipro, Ukraine, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

O. S. Shevtsova, orcid.org/0000-0002-0148-5877, Dnipro University of Technology, Dnipro, Ukraine, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

I. H. Olishevskiy, orcid.org/0000-0001-8573-3366, Dnipro University of Technology, Dnipro, Ukraine, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

* Corresponding author e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

повний текст / full article

Naukovyi Visnyk Natsionalnoho Hirnychoho Universytetu. 2025, (4): 150 - 159

https://doi.org/10.33271/nvngu/2025-4/150

Abstract:

Purpose. To evaluate the effectiveness of existing AI-generated text detection tools in identifying machine-generated content within students’ qualifying papers. The research aims to analyze the accuracy, reliability, and limitations of these detection methods, particularly for texts written in Ukrainian, and to explore potential improvements in AI content recognition.

Methodology. The study employs a combination of statistical analysis and experimental evaluation of AI-detection tools. A dataset of AI-generated and human-written academic texts is used to assess detection accuracy, with additional experiments conducted to analyze the influence of linguistic structures, translation effects, and text formalization. The research also incorporates computational techniques to examine error rates and determine patterns in AI-generated content recognition.

Findings. The study results provide insights into the performance of current AI-detection systems, highlighting their strengths and weaknesses. The analysis reveals significant challenges in detecting AI-generated text, particularly in non-English languages such as Ukrainian. Additionally, the study identifies the impact of translation on detection accuracy and discusses the effectiveness of different linguistic and statistical approaches.

Originality. This research contributes to the ongoing discourse on academic integrity by addressing the limitations of AI-detection tools for non-English academic texts. Unlike previous studies focusing primarily on English-language AI-generated content, this study provides a unique perspective on the challenges of detecting AI-generated text in Ukrainian, offering novel insights into the adaptation of detection models for diverse linguistic contexts.

Practical value. The findings of this study are valuable for educators, researchers, and policymakers concerned with maintaining academic integrity in the era of generative AI. By identifying weaknesses in current detection systems and suggesting possible improvements, the research provides a foundation for developing more robust AI-detection methodologies that can effectively apply to academic texts in multiple languages.

Keywords: AI-generated text, academic integrity, detection tools, higher education, natural language processing, machine learning, software modelling, algorithm, multilingual text analysis

References.

1. Shah, A., Ranka, P., Dedhia, U., Prasad, S., Muni, S., & Bhowmick, K. (2023). Detecting and Unmasking AI-Generated Texts through Explainable Artificial Intelligence using Stylistic Features. International Journal of Advanced Computer Science and Applications, 14, 110. https://doi.org/10.14569/IJACSA.2023.01410110

2. Majhi, D., & Santra, P. (2023). Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection? SRELS Journal of Information Management, 60, 173-181. https://doi.org/10.17821/srels/2023/v60i3/171028

3. Kumarage, T., Agrawal, G., Sheth, P., Moraffah, R., Chadha, A., Garland, J., & Liu, H. (2024). A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization. https://doi.org/10.13140/RG.2.2.23893.60645

4. Rashidi, H., Fennell, B., Albahra, S., Hu, B., & Gorbett, T. (2023). The ChatGPT conundrum: Human-generated scientific manuscripts misidentified as AI creations by AI text detection tool. Journal of Pathology Informatics, 14, 100342. https://doi.org/10.1016/j.jpi.2023.100342

5. Mindner, L., Schlippe, T., & Schaaff, K. (2023). Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT. https://doi.org/10.1007/978-981-99-7947-9_12

6. Amirjalili, F., Neysani, M., & Nikbakht, A. (2024). Exploring the boundaries of authorship: a comparative analysis of AI-generated text and human academic writing in English literature. Frontiers in Education. https://doi.org/10.3389/feduc.2024.1347421

7. Ma, Y., Jiawei, L., & Yi, F. (2023). Is This Abstract Generated by AI? A Research for the Gap between AI-generated Scientific Text and Human-written Scientific Text. https://doi.org/10.48550/arXiv.2301.10416

8. Chaka, C. (2024). Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review. Journal of Applied Learning & Teaching, 7, 1-12. https://doi.org/10.37074/jalt.2024.7.1.14

9. Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., Šigut, P., & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19. https://doi.org/10.1007/s40979-023-00146-z

10.      Gotoman, J., Luna, H., Sangria, J., Cereneo, C., & Barbuco, D. (2025). Accuracy and reliability of AI-generated text detection tools: A literature review. American Journal of IR 4.0 and Beyond, 4(1), 1-9. https://doi.org/10.54536/ajirb.v4i1.3795

11.      Chaka, C. (2024). Accuracy pecking order – How 30 AI detectors stack up in detecting generative artificial intelligence content in university English L1 and English L2 student essays. Journal of Applied Learning & Teaching, 7(1), 1-13. https://doi.org/10.37074/jalt.2024.7.1.33

12.      Ladha, N., Yadav, K., & Rathore, P. (2023). AI-generated content detectors: Boon or bane for scientific writing. Indian Journal of Science and Technology, 16(39), 3435-3439. https://doi.org/10.17485/IJST/v16i39.1632

13.      Price, G., & Sakellarios, M. (2023). The effectiveness of free software for detecting AI-generated writing. International Journal of Teaching, Learning and Education, 2(6), 31-38. https://doi.org/10.22161/ijtle.2.6.4

14.      Walters, W. (2023). The effectiveness of software designed to detect AI-generated writing: A comparison of 16 AI text detectors. Open Information Science, 7. https://doi.org/10.1515/opis-2022-0158

15.      László, B. (2024). Artificial Intelligence-Generated Text in Higher Education ‒ Usage and Detection in the Literature. Interdisciplinary Description of Complex Systems, 22, 238-245. https://doi.org/10.7906/indecs.22.3.1

16.      Jung, M., Panizo, C., Dugan, L., Fung, M., Chen, P.-Y., & ­Liang, P. (2025). Group-adaptive threshold optimization for robust AI-generated text detection. arXiv Preprint. https://doi.org/10.48550/arXiv.2502.04528

17.      Ji, J., Guo, J., Qiu, W., Huang, Z., Xu, Y., Lu, X., Jiang, X., Li, R., & Li, S. (2025). I know myself better, but not really greatly: Using LLMs to detect and explain LLM-generated texts. arXiv Preprint. https://doi.org/10.48550/arXiv.2502.12743

18.      Shivarkar, P. (2025). LLM AI-generated text detection. International Journal of Scientific Research in Engineering and Management, 9(1), 1-9. https://doi.org/10.55041/IJSREM41593

19.      Walker, E., Evans, L., Mitchell, A., Zhang, Z., Patel, R., & Chen, I. (2024). Text classification in detection of AI-generated content using BERT. ResearchGate Preprint. https://doi.org/10.13140/RG.2.2.30193.90722

20.      Krois-Lindner, A. (2011). International legal English: a course for classroom or self-study use. Cambridge University Press. Cambridge professional English. ISBN 9783125351905.

• < Prev
• Next >