Theofanis Karaletsos

Just catching up on posit::conf 2024 that I missed this year. I know that webR for server less shiny is all the craze right now, and rightfully so, but to me Charlie Gao and Will Landau and their {mirai} + {crew} ecosystem for 1st class async processing in R is at least as important. If webR and shiny are about making interactivity of compute and viz easier to share, {mirai} and {crew} (and {crew.cluster}) are about democratizing large scale scientific computation for the R community. Absolutely love the combo. Many thanks to the authors!

39

3 Comments

There is often an unspoken assumption by ML researchers that all we need to make progress and model every aspect of our world is data. We do not ascribe to this in my lab. In fact, we believe that data will never be enough. Our experiences with the nuances and complexities of scientific problems have informed us to the insufficiency of data to capture continuous physical processes, which afterall govern our biological and physical world. An example of this is this series of two papers led by my wonderful PhD student, Anowarul Kabir and advanced by a precious multi-year collaboration of my lab with Los Alamos National Lab: Anowarul Kabir, Manish Bhattarai, Kim Rasmussen, Amarda Shehu, Anny Usheva, Alan R Bishop, and Boian S Alexandrov. Examining DNA Breathing with pyDNA-EPBD. Bioinformatics 39(11):btad699, 2023. https://lnkd.in/gZaHE4hS Anowarul Kabir, Manish Bhattarai, Selma Peterson, Yonatan Najman-Licht, Kim Ø Rasmussen, Amarda Shehu, Alan R Bishop, Boian Alexandrov, Anny Usheva. DNA Breathing Integration with Deep learning Foundational Model Advances Genome-wide Binding Prediction of Human Transcription Factors. Nucleic Acids Research: gkae783, 2024. https://lnkd.in/gfTTqcPy Our goal: advance an exceptionally challenging problem in molecular biology, prediction of transcription factor binding sites. Our first step: capture the underlying physics that is missing in the data. Our second step: integrate that now with the data we have in a foundation model for predicting transcription factor binding sites. Performance improves. Most importantly, when we look at the sequence motifs that constitute "signatures" of what makes a transcription factor binding site, we obtain answers. All in the open, nothing opaque.

33

1 Comment

Sharing some of our research at M42 Health on evaluating Large Language Models (LLMs) 📊 🤖 Some quick key takeaways: 1️⃣ Evaluating LLMs is far from being trivial, with challenges present even in seemingly simple tasks such as answering multiple-choice questions (MCQs). 2️⃣ We show that different evaluation methodologies can lead to discrepancies between 5 and 26% in reported accuracy for the same model and task/dataset! 3️⃣ This research highlights the need for robust and standardized evaluation frameworks, and greater transparency in the reporting of both performance metrics and evaluation methodologies used to allow better cross-model and cross-study comparisons. 4️⃣ Finally, it shows the important role of evaluation frameworks in the standardization of assessment of LLMs, such as EleutherAI's harness framework (used by some leaderboards Hugging Face), Stanford University's HELM, OpenCompass, among others. The paper also offers a short analysis of popular MCQ benchmarks and some of these methodologies. Read more at https://lnkd.in/dnveBx4c Joint work with Clément Christophe, PhD, Praveenkumar Kanithi, Tathagata Raha, Prateek Munjal and Shadab Khan #AI #MachineLearning #NLP #LLM

79

2 Comments

📢 New Preprint Alert Beyond Precision Medicine: A Style Transfer Framework for Holistic Personalization in Older Adult Healthcare Just released: a perspective piece where we explore how the concept of style transfer (from AI) can help address the unique, often complex needs of older adults in healthcare. Current personalization methods mainly center on clinical data, but this approach integrates medical information with each patient’s lived experience—their lifestyle, social support, and individual priorities. Key points: 1) We present a model that layers clinical insights with real-world factors, helping co-create healthcare plans that are both medically sound and relevant to the individual. 2) The framework allows for adaptable, responsive plans that evolve with each patient’s health and personal circumstances. Many thanks to Nick Hird and Samik Ghosh for collaboration that shaped this work. The preprint is under review; you can read it here: https://lnkd.in/gtKM4qKJ #Healthcare #AI #StyleTransfer #PersonalizedCare #PrecisionMedicine

43

Reading Building LLMs for Production by Louis-François Bouchard and Louie Peters. At the moment I am reading chapter VIII: Advanced RAG. Looking forward to read the chapter on Deployment this weekend. In general, I like the the style of the book. It is easy to read and understand. I think this book is a good starting point for people who are going to put LLMs in production for the first time. A very practical guide. #LLMs #AI #books

27

2 Comments

Interesting that on the MIMIC IV readmission models, the LLMs get closer to the traditional ML models. Perhaps because LLMs are not great at representing numeric values (especially at the model sizes tested) that are informative of the prediction for acute outcomes, whereas readmission relies more on diagnoses and other more linguistically-coded inputs. (Though that doesn't explain the poor performance on MIMIC-III readmission)

3

Interesting article. With the ever increasing size of context windows, massive prompts could potentially outperform fine-tuning for LLMs, based on a study from CMU. That means depending on situations, one can choose to front load their cost to fine-tune LLMs, or to have larger prompts and pay more per transaction to get to similar performance. https://lnkd.in/euFTP8QS

5

Nature article highlighting the collapse of generative models as training data of future generations uses output of previous generations.    https://lnkd.in/eNzsZ6F4

10

Lately I've been curious to learn more about how AI is impacting biology and drug development. With all of the hype around Generative AI for text, there seems to be a growing discussion around Causal AI to identify new proteins and predict treatment outcomes. I thought I'd share a few helpful things I've read in the area: This article from Century of Bio offers a great explanation of how causal AI is used for protein discovery. It also gives a really approachable breakdown of how this kind of AI works. https://lnkd.in/g3U3fA4D This article from Nature Medicine goes into detail around the benefits of causal AI for predicting medical outcomes and how it can lead to better individual patient predictions: https://lnkd.in/ggnsvD4z This paper looks at the early success of AI developed drugs in the place where it matters most: clinical trials. It's interesting to see that the early pipeline of drugs designed using AI are in line with historical averages for classically developed drugs (around 40%) https://lnkd.in/g3Y6Jf5S I'll be keeping a close eye on this space! #ML #AI #CausalAI #Biology

20

🚀 Excited to share the latest insights from Clarifai's research team! In our new blog post, Isaac Chung & Phat V. explore how you can supercharge your LLMs using Retrieval-Augmented Fine-Tuning (RAFT). Developed by researchers at UC Berkeley (arXiv:2403.10131), RAFT enhances RAG by introducing a new fine-tuning recipe that leverages external knowledge. This approach improves your AI’s efficiency and performance in "open-book" settings, making your applications smarter and more precise on your data. If you're interested in pushing the boundaries of what's possible with LLMs, this is a must-read. Check out the full article here: https://lnkd.in/gwSWndnu

25

👉🏼 Expansive data, extensive model: Investigating discussion topics around LLM through unsupervised machine learning in academic papers and news 🤓 Hae Sun Jung 👇🏻 https://lnkd.in/ef_MF54v 🔍 Focus on data insights: - Utilized data from Web of Science and LexisNexis to explore topic modeling methods for Large Language Models (LLMs). - Evaluated various topic modeling approaches such as LDA, NMF, CTM, and BERTopic based on diversity and coherence metrics. - BERTopic demonstrated superior performance in diversity and coherence across both platforms. 💡 Main outcomes and implications: - News articles covered a wide range of topics related to LLM applications, focusing on specialized domains. - Research papers concentrated on technical aspects of LLM technology. - Insights gained can help understand future challenges and opportunities for LLMs in different sectors. 📚 Field significance: - Enhances understanding of how LLMs are utilized in academic and news contexts. - Provides insights for enterprises leveraging LLMs for service delivery. 🗄️: #topicmodeling #unsupervisedlearning #machinelearning #datainsights

1

SPACE2: Improved computational epitope profiling using structural models identifies a broader diversity of antibodies that bind to the same epitope. Web server: https://lnkd.in/gJT3wgh8 Paper: https://lnkd.in/gwEs93Fq Group your antibodies by what epitope they bind to on Tamarind Bio! Abstract: The function of an antibody is intrinsically linked to the epitope it engages. Clonal clustering methods, based on sequence identity, are commonly used to group antibodies that will bind to the same epitope. However, such methods neglect the fact that antibodies with highly diverse sequences can exhibit similar binding site geometries and engage common epitopes. In a previous study, we described SPACE1, a method that structurally clustered antibodies in order to predict their epitopes. This methodology was limited by the inaccuracies and incomplete coverage of template-based modeling. In addition, it was only benchmarked at the level of domain-consistency on one virus class. Here, we present SPACE2, which uses the latest machine learning-based structure prediction technology combined with a novel clustering protocol, and benchmark it on binding data that have epitope-level resolution. On six diverse sets of antigen-specific antibodies, we demonstrate that SPACE2 accurately clusters antibodies that engage common epitopes.

32

#RAG DOES NOT REDUCE HALLUCINATIONS IN #LLMS: Think I've called this out a few times, but the validation is nice - https://lnkd.in/eAi6zxW2. The article makes two well established points: 1) "Hallucination is a problem of reasoning and not relevance! Any amount of relevant text fed through RAG to a machine will retain the original perplexity and entropy in the system to hallucinate independent of the text"; and 2) "The initial challenge arises from the retrieval phase’s reliance on semantic similarity, which is quantified by inner product spaces or cosine similarities. This metric, while capturing semantic closeness, does not differentiate between factually accurate and inaccurate information." This should be clear to everyone, but the industry still shills flimsy RAG scaffolding anyway (shovels to gold miners).

104

34 Comments

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 Comments

Lately, I've been experimenting and building data pipelines for Future Therapeutics and realized how little we talk about data pipelines in cheminformatics. It's a domain where, at any given time, we handle large volumes of data from multiple sources. I'm sharing a special collection of articles on Cheminformatics Data Pipelining, designed to raise awareness and enhance skills in this critical area. Part 1 of 8 is published. Have a look and feel free to share your feedback. Also, let me know if I miss something specific for cheminformatics. I would also love to talk to those actively involved in data pipelining projects to hear your thoughts and approaches. #cheminformatics #drugdiscovery #datapipelines https://lnkd.in/gDa7G2_z

24

Thanks Helen Fu Thomas for giving me the opportunity to talk about our “Enterprise AI Acceleration” construct: AI Aware AI Learning AI Experimenting Ai Accelerating It’s tool we use when working with our Ringer Sciences clients to candidly gauge where an organization is on their journey and help them plot where they want to go and how to get there. We’ve found it to be a helpful conversation guide. #AI #EnterpriseAI #Digitaltransformation

14

4 Comments