One connector, four scholarly indexes, real DOIs and legal free PDFs. Works in Claude and ChatGPT.
Paste it into Claude or ChatGPT as a custom connector. 3-day free trial.
Ahmed Elnaggar, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, T. Fehér, Christoph M. Angerer, Martin Steinegger, Debsindhu Bhowmik, Burkhard Rost
Computational biology and bioinformatics provide vast data gold-mines from protein sequences, ideal for Language Models (LMs) taken from Natural Language Processing (NLP). These LMs reach for new prediction frontiers at low inference costs. Here, we trained two auto-regressive models (Transformer-XL, XLNet) and four auto-encoder models (BERT, Albert, Electra, T5) on data from UniRef and BFD containing up to 393 billion amino acids. The protein LMs (pLMs) were trained on the Summit supercomputer using 5616 GPUs and TPU Pod up-to 1024 cores. Dimensionality reduction revealed that the raw pLM-embeddings from unlabeled data captured some biophysical features of protein sequences. We validated the advantage of using the embeddings as exclusive input for several subsequent tasks: (1) a per-residue (per-token) prediction of protein secondary structure (3-state accuracy Q3=81%-87%); (2) per-protein (pooling) predictions of protein sub-cellular location (ten-state accuracy: Q10=81%) and membrane versus water-soluble (2-state accuracy Q2=91%). For secondary structure, the most informative embeddings (ProtT5) for the first time outperformed the state-of-the-art without multiple sequence alignments (MSAs) or evolutionary information thereby bypassing expensive database searches. Taken together, the results implied that pLMs learned some of the grammar of the language of life. All our models are available through https://github.com/agemagician/ProtTrans.
Formatted from the source metadata, not typed by hand. For a whole reference list, use the citation generator or ask your assistant through Find Research.
Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., Bhowmik, D., & Rost, B. (2021). ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 7112–7127. https://doi.org/10.1109/tpami.2021.3095381
Elnaggar, Ahmed, et al. "ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning." IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, 2021, pp. 7112–7127, https://doi.org/10.1109/tpami.2021.3095381.
Elnaggar, Ahmed, Michael Heinzinger, Christian Dallago, Ghalia Rehawi, Yu Wang, Llion Jones, Tom Gibbs, et al. "ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning." IEEE Transactions on Pattern Analysis and Machine Intelligence 44, no. 10 (2021): 7112–7127. https://doi.org/10.1109/tpami.2021.3095381.
Elnaggar A., Heinzinger M., Dallago C. [и др.] ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning // IEEE Transactions on Pattern Analysis and Machine Intelligence. — 2021. — Vol. 44, № 10. — P. 7112–7127. — URL: https://doi.org/10.1109/tpami.2021.3095381
@article{elnaggar2021prottrans,
title = {{ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning}},
author = {Ahmed Elnaggar and Michael Heinzinger and Christian Dallago and Ghalia Rehawi and Yu Wang and Llion Jones and Tom Gibbs and T. Fehér and Christoph M. Angerer and Martin Steinegger and Debsindhu Bhowmik and Burkhard Rost},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2021},
volume = {44},
number = {10},
pages = {7112--7127},
doi = {10.1109/tpami.2021.3095381},
url = {https://doi.org/10.1109/tpami.2021.3095381}
}TY - JOUR AU - Elnaggar, Ahmed AU - Heinzinger, Michael AU - Dallago, Christian AU - Rehawi, Ghalia AU - Wang, Yu AU - Jones, Llion AU - Gibbs, Tom AU - Fehér, T. AU - Angerer, Christoph M. AU - Steinegger, Martin AU - Bhowmik, Debsindhu AU - Rost, Burkhard TI - ProtTrans: Toward Understanding the Language of Life Through Self-Supervised Learning JO - IEEE Transactions on Pattern Analysis and Machine Intelligence VL - 44 IS - 10 SP - 7112 EP - 7127 PY - 2021 DA - 2021/07/07 DO - 10.1109/tpami.2021.3095381 UR - https://doi.org/10.1109/tpami.2021.3095381 ER -
Add Find Research to Claude or ChatGPT and ask in plain language. The assistant reads this record, finds the papers around it and cites only what it actually found:
Setup takes about a minute: add the connector, then start the free trial.
Record from OpenAlex; open-access location from Unpaywall and the indexes. Find Research links only to legal copies.