Skander Hannachi

These findings motivate our design of a novel framework to investigate hidden representations in LLMs with LLMs, which we call Patchscopes. The key idea behind this framework is to use LLMs to provide natural language explanations of their own internal hidden representations. Patchscopes unifies and extends a broad range of existing interpretability techniques, and it enables answering questions that were difficult or impossible before. For example, it offers insights into how an LLM's hidden representations capture nuances of meaning in the model's input, making it easier to fix certain types of reasoning errors. While we initially focus the application of Patchscopes to the natural language domain and the autoregressive Transformer model family, its potential applications are broader. For example, we are excited about its applications to detection and correction of model hallucinations, the exploration of multimodal (image and text) representations, and the investigation of how models build their predictions in more complex scenarios. Google DeepMind

9

🚀 **Just Published: Unlocking the Power of Large Language Models with Chunking Techniques!** 🚀 🔍 Ever wondered how large language models (LLMs) handle long documents while keeping context intact? The answer lies in **chunking**! In my latest blog, I explore the key chunking methods that boost LLM performance, covering everything from **simple sentence splits** to **adaptive AI-based chunking**. With each technique, I’ve included **code samples** to help you put these methods into action! 💡 Whether you're working in **Generative AI**, **NLP**, or **text summarization**, this guide will help you unlock new potential with LLMs. 📈 **Curious to know how it works? Check out the full blog!** https://lnkd.in/gniDQYwW --- #GenerativeAI #AI #MachineLearning #LLM #NLP #AIResearch #DataScience #Chunking #TextProcessing #AIEngineering --- For maximum engagement, tag a few companies or groups if possible, like: - **@OpenAI** - **@Google AI** - **@Hugging Face** - **@DeepMind**    Excited to share these insights and can’t wait to hear your thoughts! #llm #openai #genai #chunking #langchain

12

2 Comments

Can LLM preference tuning degrade Multi-Agent system performance? In a recent tweet, Andrej Karpathy questioned why all LLMs sound similar, prompting Sebastian Rascha to speculate about the uniformity of alignment datasets. Aligning user preferences is a last-mile step in training large language models. Regardless of the pre-training datasets used, could the preference datasets used during alignment compromise the effectiveness of LLMs? This issue could adversely affect multi-agent systems incorporating multiple LLMs in their design.

7

1 Comment

FlexAttention: Simplifying Attention in PyTorch _________________________________________________________________ FlexAttention is a new PyTorch API that combines the flexibility of PyTorch with the performance of optimized attention methods. It allows researchers to implement various attention mechanisms easily without needing to write custom kernels. Key Points: - Flexibility with Performance: -- Traditional attention methods offer high performance but limited flexibility. -- FlexAttention provides a flexible API that supports many attention variants with minimal effort. - How It Works: -- Standard Attention: Calculates attention scores using queries, keys, and values, normalizes them, and computes the weighted sum of values. -- FlexAttention: Allows for modifications to the attention scores through a flexible scoring function, enabling customization for different types of attention mechanisms. - Examples of Attention Variants: -- Relative Position Encoding: Adds positional information to attention scores based on relative positions. -- ALiBi Bias: Applies a bias to attention scores to account for position differences, enhancing model performance. -- Causal Mask: Masks future positions in the sequence to ensure the model only attends to previous positions, useful for autoregressive models. Handling Sparsity: -- BlockMask: Efficiently manages sparse attention masks, such as those used in causal attention, by creating masks that prevent unnecessary computations. Learn More: https://lnkd.in/g58pxB4A

8

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

𝐒𝐩𝐢𝐧𝐐𝐮𝐚𝐧𝐭: 𝐍𝐞𝐰 𝐒𝐎𝐓𝐀 𝐟𝐨𝐫 𝐋𝐋𝐌 𝐐𝐮𝐚𝐧𝐭𝐢𝐳𝐚𝐭𝐢𝐨𝐧 We have now open-sourced the code for our work on 𝐒𝐩𝐢𝐧𝐐𝐮𝐚𝐧𝐭. This is a broadly applicable method to reduce outliers in activations by rotating activation or weight matrices. SpinQuant establishes a new SOTA for LLM quantization. It narrows the accuracy gap with full precision model to merely 2.9 points for the Llama-2 7B model on zero-shot reasoning tasks. Code: https://lnkd.in/gFQ_PRG2 Paper: https://lnkd.in/g-fppehV

170

LLM as a solver Software involving Combinatorial optimization techniques is ubiquitous. How does LLM fare compare to a brute force method? We prompt the LLM with, 1. A distance matrix, the distance between the cities 2. Starting node/city. And expect the shortest path between the cities starting from the given start city and the distance covered. https://lnkd.in/enMkT92W 1. Create a python dataclass to describe the input and out of a TSP problem. 2. Write a prompt template to describe the input, output and few-shot examples. 3. A quick brute force TSP problem generator for n-shot examples. 4. Prompt the LLM for answer.

18

4 Comments

Interesting paper about the limitations of the generative AI. Actually Yann LeCun talks about this limitation as inherited problem in the existing models for long time. Knowing the limitation helps to align expectations. https://lnkd.in/dCid9B_J #generativeai #llm #artificailintelligence #ai

7

3 Comments

Types of Models for Time-Series Forecasting: (Back from the buzz of GenAI and LLMs :-)  Time-Series forecasting is crucial in many applications such as forecasting demand in supply chain management, forecasting load in the energy sector, forecasting price in e-commerce, forecasting sales for capacity planning among many others. While early statistical methods such as moving averages and exponential smoothing came up in the early 20th century (and are still used widely!), over a period of time there are tons of techniques that came up for forecasting. How does one navigate this landscape of techniques for time-series forecasting? This short video talks about the broad categories of techniques for time-series forecasting. In the subsequent videos of the series, I hope to dive into some examples of each category… https://lnkd.in/gHmYqD8c #ml #datascience #timeseries

61

8 Comments

[research update] use a Transformer's multi-head attention activations to fetch RAG style, to augment multi faceted problem breakdown for LLMs in ."Multi-Head RAG: Solving Multi-Aspect Problems with LLMs": 1. Enhances LLMs by integrating document retrieval (RAG) into the LLM context. 2. Challenge: Existing RAG solutions struggle with queries needing multiple documents with varied contents due to distant embeddings. 3. Multi-Head RAG (MRAG): Introduces a novel approach using Transformer's multi-head attention activations to fetch diverse documents. 4. Activation Utilization: Different attention heads capture varied data aspects, improving complex query retrieval. 5. Improved Relevance: MRAG shows up to 20% improvement in relevance over standard RAG baselines. 6. Evaluation Methodology: Provides metrics, synthetic datasets, and real-world use cases to demonstrate effectiveness. 7. Seamless Integration: MRAG can be integrated with existing RAG frameworks and benchmarking tools like RAGAS. 8. Multi-Aspect Problem Solving: Designed specifically to handle multi-aspect queries more effectively. 9. Data Stores Compatibility: Compatible with various data store classes. 10. Potential Applications: Benefits scenarios requiring diverse information retrieval, enhancing LLM responses. Experiment The paper introduces MRAG and evaluates it using synthetic datasets and real-world data. It compares MRAG's performance against standard RAG models. The experiment focuses on the model's ability to retrieve relevant documents across various query aspects, demonstrating up to a 20% improvement in relevance over baselines. Use Case MRAG is tested on real-world scenarios like customer support, where queries often require information from multiple documents with diverse contents. The model's capability to retrieve and integrate multi-aspect data enhances the quality of responses.

81

5 Comments

Understanding Conversation Summary Memory vs Conversation Summary Buffer Memory in LangChain For detailed sample code, check out the Google Colab notebook(https://lnkd.in/gRsarpNX) In the evolving field of conversational AI, memory mechanisms play a critical role in maintaining coherent and contextually relevant interactions. Two key approaches, Conversation Summary Memory and Conversation Summary Buffer Memory, offer distinct ways of handling conversational context. While Conversation Summary Memory focuses on summarizing entire dialogues, making it ideal for long conversations where historical context is essential, Conversation Summary Buffer Memory retains a rolling window of recent exchanges, suited for short, focused conversations. Understanding the differences between these memory types and their appropriate use cases can significantly improve chatbot performance. For a deeper dive into how these memory systems function and their practical applications using LangChain, you can explore the full pdf article below. #ConversationalAI #AI #LangChain #Chatbots #ArtificialIntelligence #MemoryMechanisms #MachineLearning #NLP #OpenAI #AIChatbot #Technology

92

one of the challenges with RAG is evaluating it. check out this code from Amazon Science if you are building a RAG based system Romina Sharifpour, Ph.D. Baichuan Sun

23

5 Comments