Nima Nejatti, PhD, MBA

𝐏𝐨𝐬𝐢𝐭𝐢𝐨𝐧 𝐀𝐯𝐚𝐢𝐥𝐚𝐛𝐥𝐞: 𝐏𝐡𝐃– 𝐀𝐈 𝐒𝐭𝐚𝐫𝐭𝐢𝐧𝐠 𝐃𝐚𝐭𝐞: 𝐀𝐬 𝐒𝐨𝐨𝐧 𝐀𝐬 𝐏𝐨𝐬𝐬𝐢𝐛𝐥𝐞 We are #hiring a #PhD candidate to join our research team and contribute to the development of the next generation of AI systems. Despite significant advancements in artificial intelligence—particularly in foundation models and LLMs—current AI systems still fall short in a crucial aspect of intelligence: the ability to learn continuously and adapt to dynamic environments without losing previously acquired knowledge. Current methods, such as RAG, are employed as a stopgap due to the lack of effective dynamic knowledge integration and scalable memory-augmented models. The successful candidate will work on pioneering techniques that push the boundaries of continual learning, dynamic knowledge representation, and memory architectures, enabling AI systems to learn and reason more effectively over time. Our team regularly publishes in top-tier venues such as 𝐍𝐞𝐮𝐫𝐈𝐏𝐒, 𝐈𝐂𝐋𝐑, 𝐈𝐂𝐌𝐋, 𝐚𝐧𝐝 𝐀𝐀𝐀𝐈. A selection of our recent publications can be found here: https://lnkd.in/eXhtCU5V. If you are passionate about advancing the field of AI and are interested in joining us, please send your CV to b.zonooz@tue.nl #ai

77

I think AI agentic machine translation has huge potential for improving over traditional neural machine translation, and am releasing as open-source a demonstration I'd been playing with as a fun weekend project. Using an agentic workflow, this demonstration (i) Prompts an LLM to translate from one language to another, (ii) Reflects on the translation to come up with constructive suggestions, (iii) Uses the suggestions to refine the translation. In our limited testing, this is sometimes competitive with, and sometimes worse than, leading commercial providers. But it gives a highly steerable translation system where by simply changing the prompt, you can specify the tone (formal/informal), regional variation (do you want Spanish as spoken in Spain or as spoken in Latin America?), and ensure consistent translation of terms (by providing a glossary). This is not mature software. But I hope the open-source community can make agentic translation work much better. Given how a simple reflection workflow already gives decent results, I think there's significant headroom to make agentic translation much better. Releasing an early software prototype like this is something new I decided to try to see if it is helpful to the developer community. I'd love any feedback on this. Thanks to Joaquin Dominguez, Nedelina Teneva, PhD and John Santerre PhD for help with this. https://lnkd.in/gjGANH6H

5,304

163 Comments

🔍 Quantum AI Series: Modeling Sequences with Quantum States Picking up where I left off in my #QuantumAI series, let’s talk about Modeling Sequences with Quantum States, with insights from a fascinating paper on the topic. Classical probability distributions can be modeled using quantum states—particularly entangled states that retain information from subsystems. This approach allows quantum models to efficiently represent sequences without losing essential details. Tensor networks play a key role in optimizing the process, providing a compact, low-rank factorization of data. As I've shared previously, Singular Value Decomposition (SVD) is foundational to these models, as it helps represent the complex correlations present in sequential data. Matrix Product States (MPS) capture these rich interactions by using reduced densities to efficiently compress sequence representations while preserving their structure. This is the essence of keeping representation while building efficient lightweight models. In summary, quantum probability models allow us to go beyond classical limits in sequential tasks, contributing to information retention. Sharing the paper for a deeper dive if you'd like so: https://lnkd.in/d5aqXyyS #QuantumAI #TensorNetworks #SVD #QuantumComputing #AI

5

This new article discusses our efforts in delivering tensor parallelism, weight-only quantization, vLLM integration, and the RBLN model zoo. It is definitely worth reading! 👉 https://lnkd.in/gRnG8_pR #TensorParallelism #WeightOnlyQuantization #vLLM #ModelZoo

84

Handling a high volume of language processing tasks is rather token-intensive. See the performance impact of accelerated AI inference with NVIDIA NIM, when deploying apps at scale.

4

Announcing our latest paper sPhinX: Sample Efficient Multilingual Instruction Fine-Tuning Through N-shot Guided Prompting TL;DR: We propose a recipe to create a new synthetic IFT dataset covering 50 languages, and finetuned phi-3-small and Mistral with significant improvements over several multilingual benchmarks spanning reasoning, reading comprehension and QA without compromising on English performance! Preprint: https://lnkd.in/gFNiTqfX Abstract: Despite the remarkable success of LLMs in English, there is a significant gap in performance in non-English languages. In order to address this, we introduce a novel recipe for creating a multilingual synthetic instruction tuning dataset, sPhinX, which is created by selectively translating instruction response pairs from English into 50 languages. We test the effectiveness of sPhinX by using it to fine-tune two state-of-the-art models, Phi-3-small and Mistral-7B and then evaluating them across a comprehensive suite of multilingual benchmarks that test reasoning, question answering, and reading comprehension. Our results show that Phi-3-small and Mistral-7B fine-tuned with sPhinX perform better on an average by 4.2%pt and 5%pt respectively as compared to the baselines. We also devise a strategy to incorporate N-shot examples in each fine-tuning sample which further boosts the performance of these models by 3%pt and 10%pt respectively. sPhinX also outperforms other multilingual instruction tuning datasets on the same benchmarks along with being sample efficient and diverse, thereby reducing dataset creation costs. Additionally, instruction tuning with sPhinX does not lead to regression on most English benchmarks. with Kumar Tanmay, Sanchit Ahuja, Hardik Chauhan, Barun Patra, Kriti Aggarwal, Luciano Del Corro, Arindam Mitra, Tejas Indulal Dhamecha, Ahmed Awadallah, Monojit Choudhury and Sunayana Sitaram

108

Scaling and more data alone won't achieve human-level intelligence in LLMs. Yann LeCun sees JEPA as a key step. Our new article covers: - How JEPA works - How it differs from transformers - Models based on JEPA: I-JEPA, MC-JEPA, V-JEPA https://lnkd.in/ehqXW6T7

1

Leverage StarCoder 2, a powerful LLM for code generation, completion, summarization, and documentation. Try the now available NVIDIA-hosted StarCoder 2 API, packaged as NIM inference microservice for free from the NVIDIA API catalog. https://dy.si/Zf8AD

8

Thanksgiving update 🦃 -- the Qwen team released QwQ-32B, a new AI model that matches OpenAI's latest o1 model in reasoning. tl;dr: Open-source AI shows that it can continue to keep up with closed-source performance on reasoning tasks (like competition math and coding) with smaller architecture. 🔍 Key achievements: Outperforms o1 in mathematical reasoning (MATH-500: 90.6% vs 85.5%) Matches top-tier performance in graduate-level science problems Excels in coding challenges with real-world applications 🧠 What makes it special: QwQ-32B achieves this through "deep reflection" - essentially teaching AI to double-check its work and question its assumptions, much like human experts do. 💡 Industry Impact: This breakthrough shows that efficient, open-source models can rival larger proprietary systems, potentially democratizing advanced AI capabilities. This follows recent advances from Deepseek-r1 and Llava-o1, suggesting a renaissance in open-source AI development. Blog: https://lnkd.in/eDEDHwgz HuggingFace link to weights: https://lnkd.in/eMpwUPpk #AIresearch #MachineLearning #OpenSourceAI

29

🌟 2024: The Year of Generative AI Deployment! 🌟   If 2022 was the year the world discovered the potential of generative AI (GenAI) and 2023 marked breakthroughs in large-scale training, then 2024 is set to revolutionize how organizations deploy these powerful systems. Companies are now actively testing GenAI in production, unlocking unprecedented benefits. However, this shift isn't without its challenges—managing complexity and ensuring rapid response times are crucial for a seamless, human-like experience.   🚀 Introducing NVIDIA NIM™   NVIDIA has launched NIM™ inference microservices, designed to streamline GenAI deployment and significantly reduce time to market. Learn more about NIMs here https://lnkd.in/dyFkBfbi.   But what if the specific model you need isn't available as a NIM yet? You can still be extremely performant, and we’ll show you how.   📚 Unlocking Potential with Optimized Inferencing   In our latest blog , we explore how to leverage NVIDIA's cutting-edge inferencing technologies—Triton Inference Server and TensorRT-LLM—to enhance large language model (LLM) inferencing in production. We showcase the DictaLM-2.0 model with a custom tokenizer (which has recently led the Hugging Face Open Leaderboard for Hebrew LLMs) and demonstrate how to create optimized engines for both FP16 and quantized INT4 versions, all served through Triton Server. This setup allows for efficient interaction with the LLM, delivering streaming outputs with minimal latency.   🔗 Dive Deeper!   Check out our blog and explore the accompanying Jupyter notebook that details each step of the process. We’d love for you to try it out and share your feedback! Blog - https://lnkd.in/d68QYepb Unofficial Jupyter notebook - https://lnkd.in/digcgqQh   Big shoutout to the incredible team behind this initiative and the guys over at Dicta for the amazing collaboration: Shaltiel Shmidman Amit Bleiweiss Zvi Lapp   #GenerativeAI #MachineLearning #NVIDIA #TritonInferenceServer #TensorRT #LLM

18

1 Comment

Llama 3.2 Optimized for High Performance on NVIDIA #GPUs 🚀 AI at Meta's latest Llama 3.2 vision language models (VLMs) and small language #models (SLMs) have achieved great performance enhancements through NVIDIA's comprehensive optimizations, ensuring efficient deployment across NVIDIA GPUs, from data centers to edge devices. Key Highlights: - Multimodal Capabilities: Support for both text and image inputs, with context lengths up to 128K tokens. - Advanced Optimizations: Cutting-edge TensorRT and FP8 quantization unlock higher throughput and lower latency without compromising accuracy. - Achieve up to 2,646 tokens/sec throughput on #NVIDIA H200 GPUs. These improvements let developers build AI solutions that work faster and cost less, setting new standards for AI inference performance. Learn more: https://lnkd.in/gfchxZQN

78

Hello All! It is great to be back with another article in my series “Neural Networks Intuitions - 18. Generative Pretrained Transformer(GPT) Series". The past 1.5-2 years have been very much dominated by Large Language Models(LLMs) ever since the release of GPT-3.5 and GPT-4, their fantastic ability to solve wide range of natural language tasks — right from question answering, summarization, visual QA and even code generation. And anyone who has followed this recent trend would know about the talks regarding the emergence of AGI which led to two school of thoughts — one set of people who truly believe that Language Models can lead us to AGI, while the other set sharply denies this statement by arguing that although there are a few signs of reasoning, there is no true extrapolation as such and all of it comes down to the training data(and its scale). The latter group's criticism is that unless there is visibility into the training data distribution and having a test set(without contaminating the test data with train set) along with a proper evaluation methodology, it is unfair to say that these LLMs can actually lead to AGI. Now in order to truly validate some of these closed source GPT models(such as GPT-4, Gemini, Claude etc), we need to be aware of the training data distribution — which is highly unlikely. But another way to understand if LLMs or more specifically instructed tuned LLMs can actually generalize to novel distribution or not is by actually digging deep into the evolution of these architectures/algorithms — more specifically the evolution of GPT models. In this article, I will be going over a series of GPT papers - GPT-1, GPT-2, GPT-3 and InstructGPT, and try to understand what makes these models truly generalizable or universal. Do read and comment your thoughts/opinions :) https://lnkd.in/di8_VpJA #neuralnetworks #largelanguagemodels #GPT #incontextlearning #nlp

24

1 Comment

We have been saying this for over a year, using massive models in a business context when model augmentation is required (e.g. RAG) is misguided and wasteful. While massive models have a purpose, one size does not fit all and much smaller models can be far more effective and efficient: https://lnkd.in/ggY-p3Zc

148

11 Comments

🎉 New scientific release! 🎉 I’m excited to share our latest publication on AVR (Advancing Video Retrieval), a groundbreaking framework that enhances video concept detection through an optimized fusion of visual and semantic features. Our research shows impressive improvements in retrieval accuracy, positioning AVR as a significant step forward compared to existing systems. 🔍 Publication title: "AVR (advancing video retrieval): A new framework guided by multi-level fusion of visual and semantic Features for deep learning-based concept detection" Published in Multimedia Tools and Applications (#SpringerNature, Q1) 📄 Length: 63 pages This work opens up new possibilities for improving multimedia data retrieval, and I'm proud to collaborate with #Sonia_Lajmi from the #University_of_Albaha, #Saudi_Arabia, and #Maryem_Jalouli from the #University_of_Sfax on this project. 📖 A must-read for anyone interested in advancing video retrieval and concept detection with deep learning. Thanks to my co-authors for their invaluable collaboration! 🙏 🔗 Full article here 👉 https://lnkd.in/dUTa5WYf #Publication #Research #AI #VideoRetrieval #MachineLearning #DeepLearning #Multimedia #Innovation #UniversitéLimoges #XLIM #3iL #SpringerNature

27

1 Comment

📣 Machine learning developers: Check out this very technical webinar by our very own Milos Puzovic where he details how to get the most out of Llama on Arm-based AWS Graviton instances.. #onarm #ml Ed Miller

37

Does CPU speed significantly affect GPU inference in LLMs like llama.cpp? The answer may surprise you. 🔹 Our benchmarks with RTX 4080/4090 reveal CPU speed’s limited role in smaller model inference. 🔹 Intel vs. AMD showed comparable results, meaning fewer worries about CPU choice! 🔹 Key takeaway? Focus GPU inference optimizations elsewhere. Get the full benchmarking breakdown in our latest article. https://hubs.ly/Q02SrSVb0

5

We just released our first pretrained draft model that we use to accelerate Llama 2 7B and CodeLlama 2 7B on Ryzen AI. #AMD #AI #ML

173

1 Comment