Huan Sun

For #InContextLearning, how do we attribute an #LLM prediction to each task demonstration? 🚀 The work of Zijian Zhou, Xiaoqiang Lin, Xinyi Xu, Alok Prakash, Daniela Rus, and Bryan Kian Hsiang Low in #ICML2024 Workshop on In-Context Learning introduces DETAIL: Task DEmonstration Attribution for Interpretable In-context Learning! Our DETAIL method: ✅ Optimizes demonstration selection & ordering in an interpretable way; ✅ Detects corrupted demonstrations; ✅ Computable in REAL-TIME! #Interpretability #PromptEngineering 📝 Paper: https://lnkd.in/gUWmP_y5 🕹️ Github: https://lnkd.in/gSjkfciJ

14

✨ Jiayuan Hao is a second-year PhD student in biostatistics program. Hao’s area of interest is environmental statistics and environmental health. “I would like to be a researcher in biostatistics developing advanced statistical tools for complicated environmental health studies after graduation,” shares Hao. 🖥️ This award enabled Hao to train in Deep Learning models more efficiently. Similarly, accessing APIs from OpenAI broadened Hao's perspective on the application of AI techniques in health-related research. 💡 “Last year, I stepped out of my comfort zone and began genomics research. Although, I initially found it incredibly challenging, my persistence in seeking out resources and mentors helped me overcome the steep learning curve. It shattered my self-imposed limitations and instilled in me a newfound confidence. The key takeaway was a reaffirmation of my belief: with dedication and perseverance, I can excel in any field of my choosing.” 💭 Hao encourages other students to consider applying for the Student Success Award. "It represents an excellent chance to reflect on the challenges they have overcome in the past year and to set new objectives for the future. This process is not just about recognition but also about personal growth and envisioning a path forward."

10

💥💥💥 A Survey for Large Language Models in Biomedicine Chong Wang, Mengyao Li, Junjun He, Zhongruo Wang, Erfan Darzi, Zan Chen, Jin Ye, Tianbin Li, Yanzhou Su, Jing Ke, Kaili Qu, Shuxin Li, Yi Yu, Pietro Liò, Tianyun Wang, Yu Guang Wang, Yiqing Shen Abstract Recent breakthroughs in large language models (LLMs) offer unprecedented natural language understanding and generation capabilities. However, existing surveys on LLMs in biomedicine often focus on specific applications or model architectures, lacking a comprehensive analysis that integrates the latest advancements across various biomedical domains. This review, based on an analysis of 484 publications sourced from databases including PubMed, Web of Science, and arXiv, provides an in-depth examination of the current landscape, applications, challenges, and prospects of LLMs in biomedicine, distinguishing itself by focusing on the practical implications of these models in real-world biomedical contexts. Firstly, we explore the capabilities of LLMs in zero-shot learning across a broad spectrum of biomedical tasks, including diagnostic assistance, drug discovery, and personalized medicine, among others, with insights drawn from 137 key studies. Then, we discuss adaptation strategies of LLMs, including fine-tuning methods for both uni-modal and multi-modal LLMs to enhance their performance in specialized biomedical contexts where zero-shot fails to achieve, such as medical question answering and efficient processing of biomedical literature. Finally, we discuss the challenges that LLMs face in the biomedicine domain including data privacy concerns, limited model interpretability, issues with dataset quality, and ethics due to the sensitive nature of biomedical data, the need for highly reliable model outputs, and the ethical implications of deploying AI in healthcare. To address these challenges, we also identify future research directions of LLM in biomedicine including federated learning methods to preserve data privacy and integrating explainable AI methodologies to enhance the transparency of LLMs. 👉 https://lnkd.in/dRPWGtBH #machinelearning #biomedicine #innovation

2

Can we leverage the strengths of both Transformer and Mamba to excel in tasks requiring both long-term context and efficient sequential processing? The Bamba-9B model does exactly that! Bamba-9B is an inference-efficient hybrid Mamba2 model. It not only achieves competitive performance against state-of-the-art transformer models such as LLama3.1-8B but also delivers 2.5x throughput improvements and 2x lower latency! Bamba is the result of a fantastic collaboration between IBM, UIUC, CMU, and Princeton. And the best part? It's publicly available for everyone to explore and experiment with! - Tech blog: https://lnkd.in/gSgNZnhf - GitHub: https://lnkd.in/gf6M_M8E Raghu Ganti, Mudhakar Srivatsa, Davis Wertheimer

24

Congratulations to Qianlin Liang on successfully defending his Ph.D thesis titled "Resource Management for Edge AI." Qianlin's thesis looks at model-driven resource management for edge AI workloads. #edgeAI #cloudcomputing

93

7 条评论

Check out this fascinating research by Peichen Pan, Chang-Yu Hsieh, Tingjun Hou, et al. from Zhejiang University! 'A flexible data-free framework for structure-based de novo drug design with reinforcement learning' This work introduces a unqiue structure-based de novo drug design method that utilises Monte Carlo tree search alongside a carefully curated fragment library. This approach effectively overcomes the limitations of traditional atom-by-atom molecular generation methods. Read here for free via Diamond💎 Open Access: https://lnkd.in/eWtRSN9M

1

Our paper entitled "Relation-aware heterogeneous graph network for learning inter-modal semantics in textbook question answering” just accepted in #TNNLS (IEEE Transactions on Neural Networks and Learning Systems) https://lnkd.in/g_XqrhZR

1

ECE Research presentation : Wednesday April 17 2024 Join Zoom Meeting https://lnkd.in/gvN53beB Meeting ID: 972 578 6218 --- Agenda 1:00 pm Basharat, Alina. " SECURING FEDERATED LEARNING: A TRI-FOLD APPROACH FOR FAIRNESS, ATTACK RESILIENCE, AND PRIVACY PRESERVATION" 1:13 pm Boone, Ethan. "Glucose Infused Solution’s Permittivity during Varying Temperatures" 1:26 pm Ricardo Cepeda. "Feasibility Assessment of Antenna-Based Glucose Monitoring: Leveraging Blood Permittivity for Accurate and Realistic Detection of Glucose Levels" 1:39 pm Luiton, Erick "Safe and Efficient Transportation via Decentralized Multi Agent Reinforcement Learning for Connected Autonomous Vehicles" 1:52 pm Cabrera, Gabriel "A Signal Classification Solution for EEG-based Brain-Computer Interfaces" 2:05 pm Carrizales, Andrew "Utilizing Energy Assessment Analysis in Industrial Facilities to Implement Energy Efficiency by Optimizing Renewable Sources" 2:18 pm Salinas, Rogelio Thanks to Dr. Soto for organizing it .

2

1 条评论

Great learning materials on Equivariant Neural Networks put together by Wenhan Gao, along with toy code to play with. If you are interested in Group CNNs, Steerable CNNs, Spherical Tensors, Cartesian Tensors, Irreps, Tensor Product, Spherical Harmonics, Group Averaging, Frame Averaging, etc, can check this for an easy start: https://lnkd.in/enfwyenj Other topics that Wenhan is working on are here: https://lnkd.in/ebmjMwm4 #Equivariance, #Symmetries, #AI4Science, #SciML, #GeometricDeepLearning

34

Interested in a linear-complexity Cross (Attention-like) Mamba model? Come and join our poster in #NeurIPS2024 at Vancouver Convention Center in today. Congratulations to Qianxiong Xu, supervised by Prof Cheng Long, and co-supervised by Prof Guosheng Lin from Nanyang Technological University Singapore and me. This research is supported by our generous industrial research grant from S-Lab. - Paper: https://lnkd.in/dD_v9WsE - Github: https://lnkd.in/d9PgcAhP Attention is famous for its ability to capture long-range dependencies, but its computational complexity is quadratic to the number of pixels (tokens). What if we want to deal with, e.g., large images, but only with normal GPUs? Try Mamba instead! This recent advance has shown competitive ability, but the computational complexity is just linear! Mamba is originally designed to capture intra-sequence dependencies, and we can call it Self Mamba. But what if there is requirement for capturing inter-sequence dependencies, i.e., Cross Mamba? For example, in the task of Few-Shot Segmentation (FSS), the features of a query image should be fused with the features of k annotated support images for segmentation. Existing methods deploy quadratic-complexity Cross Attention to achieve this goal, so can we have a linear-complexity Cross Mamba? Of course, we can! A naive way is to use a standard Mamba to scan the support features first, and the support features would be compressed into the hidden state of Mamba. Then, use this hidden state to continue scanning query image, so the support features can be fused to query features via hidden state. Unluckily, two issues will arise: (1) Support forgetting: With the scan on query, the finite hidden state will turn from pure support features to the mixture of support and query features. As a result, the support information cannot be sufficiently utilized by many query pixels, especially for those pixels at end; (2) Intra-class gap: Although query and support objects belong to the same class, they can still be visually different. Therefore, query features may prefer not to fuse support features from the hidden state, but focus more on themselves. In our work, we design two Mamba modules to address them: (1) Support Recapped Mamba: We adopt an ingenious strategy to periodically recap (i.e., re-scan) support FG features when scanning query features; (2) Query Intercepted Mamba: We intercept the mutual interactions among query pixels when propagating the hidden state (with pure support FG) to each query pixel, so each query pixel will have no choice but to inevitably fuse support features. For more details, please refer to our paper, or come and join us! #FewShotSegmentation #Mamba #InterSequenceDependency #LinearComplexity #NeurIPS2024 #Vancouver #DeepLearning #MachineLearning #ComputerVision

24

Do you want to know how a particular knowledge graph embedding perform on a graph without running any downstream task? This seems an impossible question! Check our latest paper, ReliK, to be presented at #TheWebConf2024! A paper of which I am particularly proud. Work with my PhD student Maximilian Egger and Wenyue M., Francesco Gullo, Aris Anagnostopoulos, Ilaria Bordino, Panagiotis Karras Bonus: Code and data available 👇 #kge #embeddings #ml

22

Interested in solving combinatorial #scheduling problems with differentiable models and gradient descent on GPUs, TPUs, or other accelerators? Coming at ICML 2024 #ICML, Mingju Liu (Mingju's website: https://lnkd.in/gbyWAwT9 ) present our paper, "Differentiable Combinatorial Scheduling at Scale," where we introduce a fully differentiable modeling approach for combinatorial scheduling. This new formulation allows for a fully differentiable encoding of SDC-scheduling #sdc #scheduling, which can be extended to broader combinatorial problems under linear programming (LP) models. Great thanks to the scheduling/HLS expert Prof. Zhiru Zhang from Cornell for many insightful discussions and advice, and congrats to our leading student authors Mingju Liu Yingjie Li. If you are interested in solving discrete combinatorial problems with differentiable models using ML infrastructures, please check out our paper and GitHub repository, and feel free to reach out! Paper: https://lnkd.in/gFCyGXZX Code: https://lnkd.in/gUik3NuA #ICML2024 #Scheduling #Opensource

125

2 条评论

A new publication titled “Deep Learning-Based Superpixel Texture Analysis for Crack Detection in Multi-Modal Infrastructure Images” is now available on NDT MDPI. #ndt #imaging #deeplearning #textureanalysis #cracks #multimodalimages #infrastructure

7

Congrats to Ruokai Yin and Youngeun Kim on paper acceptance to International Symposium on Microarchitecture (MICRO) 2024, titled "LoAS: Fully Temporal-Parallel Datatflow for Dual-Sparse Spiking Neural Networks". LoAS targets the dataflow optimization in spiking neural networks with dual sparsity (sparse matrix-sparse matrix operations) towards ultra-lightweight execution. With weight sparsity on top of spike sparsity (dual), existing dataflows and corresponding accelerators are suboptimal, exhibiting excessive memory accesses. LoAS parallelizes all time steps after direct coding, thus formulating a fully temporal parallel (FTP) dataflow. LoAS scrutinizes both membrane potential accumulation and neuron firing and equips an FTP-friendly spike compression mechanism and a novel SNN-tailored inner join circuit. This work is a joint effort between Priyadarshini Panda at Yale University and me. #neuromorphiccomputing #deeplearning #computerarchitecture #dataflow #spikingneuralnetwork #domainspecificacceleration

81

4 条评论

Our next presentation within the IBM University of Illinois Urbana-Champaign Discovery Accelerator Institute is coming up in two days: Speaker: Deming Chen, UIUC Title: SaiS - Security for AI and AI for Security Join here on Friday, 10/18, 11 am Eastern Time (or watch recording): https://lnkd.in/gESqbTUh Abstract: Artificial intelligence (AI) is transforming various industries and aspects of human life, including healthcare, finance, retail, autonomous vehicles, national security, and smart grids. Consequently, securing AI systems has become imperative. At the same time, AI can significantly enhance system security, for instance, by using machine learning to detect and respond to cyber threats in real-time or employing AI tools to prevent financial crimes. This dual focus—securing AI and leveraging AI for security—is essential, as these two aspects are deeply interconnected. In this talk, we will discuss the unique security challenges faced by AI systems and explore strategies such as Trusted Execution Environments (TEEs) to protect AI models and data, especially when using AI accelerators. We will also introduce AI-based techniques for detecting malicious activities in cyber-physical and IoT systems, concluding with a vision for utilizing TEEs, SmartNICs, and a new concept called 'safe-domains' to securely manage diverse AI workloads in hybrid cloud environments.

8

🚨 New Paper Alert! 🚨 ✨ Our paper entitled “Repeatability of Fine-Tuning Large Language Models Illustrated Using QLoRA,” has been published by IEEE Access! ✨ 📜 In this paper, we study the challenges of repeatability in fine-tuning four LLMs using two public datasets with QLoRA. To assess repeatability, we fine-tune each LLM seven times under identical hardware and software conditions. Our findings underscore the impact of non-repeatable fine-tuning of LLMs in obtaining reproducible LLMs models and in understanding the behaviour of LLMs. A huge thanks to the great team Prof. Larry (Lawrence) Hall, Prof. Dmitry Goldgof, and Prof. Peter R. Mouton, Ph.D. and to USF, Nvidia, NU and NSF for the computing resources and funding support! 🙏 Read more here: https://lnkd.in/dzpEx9qe #Research #AI #DeepLearning #LLMs #IEEE #QLoRA #Repeatability #Reproduciblity

35

2 条评论

💡 It is not a secret that when it comes to some domain-specific expertise, models trained with targeted fine-tuning of the smaller models are superior to #LLMs in both performance characteristics and cost. 🎯 📝My colleague Jennifer Jie Xu and I have recently published a study, titled “Entity recognition from colloquial texts” (DSS, vol. 179)  in which we evaluated fine-tuning strategies for recognizing symptom descriptions in colloquial texts. The paper describes distinguishing characteristics of colloquial symptom entities found in patient forums, and presents design choices for effective fine-tuning of symptom-labeling models. 🔑 Key Findings: 🔖 Colloquial symptom mentions differ from those in clinician-written texts by using a greater number of words, frequent use of non-medical terms, and figurative language. 🔖 Adding colloquial text to the training data improves model’s ability to correctly identify longer symptom descriptions. 🔖  Models fine-tuned on a mixture of formal medical and colloquial texts perform best overall, when used on both colloquial and formal texts. Interestingly, they also are on par or better for formal texts than models trained on formal texts alone.  🔖 Fine-tuned Roberta and Deberta models worked on par or better than pretrained biomedical models fine-tuned in the same way . 🎯The study of symptom identification in colloquial texts was motivated by the need to improve patient-doctor communication. ⁉ Question: Do you know other domains, in which colloquial texts may contain similarly important information?🤔 #MachineLearning #ModelFineTuning #LLM #NaturalLanguageProcessing #HealthcareIT #DigitalHealth

13