- Published
- 2024-01-01
- Venue
- Transactions of the Association for Computational Linguistics, vol. 12, pp. 157-173
- Cited by
- 1,550
- Open access
- Yes, diamond
- DOI
- 10.1162/tacl_a_00638
Abstract
While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models.
Cite this paper
Formatted from the source metadata, not typed by hand. For a whole reference list, use the citation generator or ask your assistant through Find Research.
- APA 7
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
- MLA 9
Liu, Nelson F., et al. "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics, vol. 12, 2024, pp. 157–173, https://doi.org/10.1162/tacl_a_00638.
- Chicago
Liu, Nelson F., Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics 12 (2024): 157–173. https://doi.org/10.1162/tacl_a_00638.
- GOST
Liu N. F., Lin K., Hewitt J. [и др.] Lost in the Middle: How Language Models Use Long Contexts // Transactions of the Association for Computational Linguistics. — 2024. — Vol. 12. — P. 157–173. — URL: https://doi.org/10.1162/tacl_a_00638
- BibTeX
@article{liu2024lost, title = {{Lost in the Middle: How Language Models Use Long Contexts}}, author = {Nelson F. Liu and Kevin Lin and John Hewitt and Ashwin Paranjape and Michele Bevilacqua and Fabio Petroni and Percy Liang}, journal = {Transactions of the Association for Computational Linguistics}, year = {2024}, volume = {12}, pages = {157--173}, doi = {10.1162/tacl_a_00638}, url = {https://doi.org/10.1162/tacl_a_00638} }- RIS
TY - JOUR AU - Liu, Nelson F. AU - Lin, Kevin AU - Hewitt, John AU - Paranjape, Ashwin AU - Bevilacqua, Michele AU - Petroni, Fabio AU - Liang, Percy TI - Lost in the Middle: How Language Models Use Long Contexts JO - Transactions of the Association for Computational Linguistics VL - 12 SP - 157 EP - 173 PY - 2024 DA - 2024/01/01 DO - 10.1162/tacl_a_00638 UR - https://doi.org/10.1162/tacl_a_00638 ER -
Ask Claude about this paper
Add Find Research to Claude or ChatGPT and ask in plain language. The assistant reads this record, finds the papers around it and cites only what it actually found:
- “Summarize doi:10.1162/tacl_a_00638 and list its main limitations.”
- “Find newer studies that test the same question, with free PDFs.”
- “Cite this and five related papers in APA for my literature review.”
Setup takes about a minute: add the connector, then start the free trial.
Record from OpenAlex; open-access location from Unpaywall and the indexes. Find Research links only to legal copies.