Content area
More than 500 statutes and rules are interlinked with building codes and regulations, with 24 amendments introduced in 2023. This complex network of references, combined with frequent revisions, poses significant challenges for both legal experts and laypeople in interpreting building codes. To address this, prior research has explored the application of retrieval-augmented generation (RAG) techniques to large language models (LLMs). However, these efforts have struggled to retain contextual information due to data segmentation. This study proposes and evaluates six variants of zero-shot Chain-of-Thought (CoT) prompts, leveraging longcontext window (LCW) LLMs for retrieving relevant legal texts. Among these, the prompt requiring minimal human guidance achieved 63.16% accuracy on a dataset of 171 legal interpretative question-answer (LIQA) pairs, outperforming other variants by up to 5.7%p. Furthermore, incorporating a high-level community summary improved the model's robustness to prompt variations. This approach reduced the need for extensive document hops whenlocating relevant provisions, narrowing the performance gap to 1.17%p. Lastly, the LCW LLM consistently referenced definition provisions, ensuring greater consistency in legal interpretation. The less the guidance, the better the performance. Less is Better!
Details
1 Department of Architecture and Architectural Engineering, Yonsei University, Seoul, Republic of Korea