Deep Learning-Based UI Design Analysis: Object Detection and Image Retrieval Using YOLOv8

Abstract

Data-driven design models support various types of mobile application design, such as design search, promoting a better understanding of best practices and trends. Designing the well User Interface (UI) makes the application practical and easy to use and contributes significantly to the application’s success. Therefore, searching for UI design examples helps gain inspiration and compare design alternatives. However, searching for relevant design examples from large-scale UI datasets is challenging and not easily stricken. The current search approaches rely on various input types, and most of them have limitations that affect their accuracy and performance. This research proposed a model that provides a fine-grained search for relevant UI design examples based on UI screen input. The proposed model will contain two phases. Object detection was implemented using the deep learning model ‘YOLOv8’, achieving 95% precision and 97% average precision. Image retrieval, leveraging the cosine similarity technique to retrieve the top 3 images similar to the input. These results highlight the system’s effectiveness in accurately detecting and retrieving relevant UI elements, providing a valuable tool for UI designers.