Query-driven Relevant Paragraph Extraction from Legal Judgments

Santosh T.y.s.s., Elvin A. Quero Hernandez, Matthias Grabmair


Abstract
Legal professionals often grapple with navigating lengthy legal judgements to pinpoint information that directly address their queries. This paper focus on this task of extracting relevant paragraphs from legal judgements based on the query. We construct a specialized dataset for this task from the European Court of Human Rights (ECtHR) using the case law guides. We assess the performance of current retrieval models in a zero-shot way and also establish fine-tuning benchmarks using various models. The results highlight the significant gap between fine-tuned and zero-shot performance, emphasizing the challenge of handling distribution shift in the legal domain. We notice that the legal pre-training handles distribution shift on the corpus side but still struggles on query side distribution shift, with unseen legal queries. We also explore various Parameter Efficient Fine-Tuning (PEFT) methods to evaluate their practicality within the context of information retrieval, shedding light on the effectiveness of different PEFT methods across diverse configurations with pre-training and model architectures influencing the choice of PEFT method.
Anthology ID:
2024.lrec-main.1177
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
13442–13454
Language:
URL:
https://aclanthology.org/2024.lrec-main.1177
DOI:
Bibkey:
Cite (ACL):
Santosh T.y.s.s., Elvin A. Quero Hernandez, and Matthias Grabmair. 2024. Query-driven Relevant Paragraph Extraction from Legal Judgments. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 13442–13454, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Query-driven Relevant Paragraph Extraction from Legal Judgments (T.y.s.s. et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.lrec-main.1177.pdf