Himanshu Raja

A couple of recent college graduates from India against AI finetuning giants like Anyscale and Run:ai? I met Arko C at a NASSCOM event last October. He was building Xylem AI (YC S24) , a platform that allows you to train and deploy LLMs in production. So naturally most Indian VCs rejected him. Because how could a team of three recent college grads compete with the likes SV startups like Anyscale that have raised hundreds of millions of dollars, have teams of experienced AI researchers and a stash of H100 GPUs? Arko hustled and closed two Fortune 500 companies for paid contracts. I personally know unicorns that haven't managed to do this. He then got into YC in the last round in May. There's a lot to still be proved out, but Arko exemplifies how valuable grit is in building startups. I’m doing a webinar with him tomorrow on challenges of using LLMs in production. Link to register in the comments. Disclaimer: I'm a small investor in Xylem.

153

5 Comments

We are excited to back Morphing Machines. As Dr. Ranjani, CTO and co-founder says: “For the first time in India and globally, we have a processor that is truly runtime reconfigurable, catering to emerging workloads.” While Nvidia’s growth and high valuation are driven by robust demand for “infra-compute” in high-performance GPUs, spurred by Gen AI, there is a distinct need for innovative solutions tailored to a new wave of reactive applications. Enter Morphing Machines. It addresses this demand with its unique product designed for next-gen telecom, autonomous vehicles, collaborative robots, avionics, and online data analytics. This product is specifically engineered to handle heterogeneous workloads and diverse tasks with varying performance demands simultaneously, ensuring a high level of scalability. As we move from an institutionally-backed research team at IISC to productization, Golden Sparrow is collaborating with Speciale Invest and others to support Deepak Shapeti, who joined as CEO three years ago after earning his Masters at Stanford. He brings not only technical expertise but also the business acumen necessary to guide the product to commercial success. Venkat Raju Afreed Faizan Anand Rao Ritik Singh Michael Marmor

68

5 Comments

The 2024 edition of the Stanford AI Index is out, and highlights PaLM-E and RT-2 as this past year's research spotlights in robotics: https://lnkd.in/gemZZzXJ

48

1 Comment

I am a big fan of Andrej Karpathy's work (DNN & LLMs). Here are his tips for going from hacking to changing the world (at the recent keynote at UC Berkeley AI Hackathon) : 1) Snowball your projects 2) Put in your time (10K hours) 3) Keep the dopamine flowing https://lnkd.in/duv3PZZh (4:10 to 22:20)

46

Just witnessed the most epic impromptu panel discussion take-over by none other than John Chen (VP of Technology at NVIDIA) at Monte Jade West Science & Technology Association talk with Brandon Wang, Wen H. Hsieh, Ph.D., Jay Lee, Achin Bhowmik and Wei Li: GPUs are far ahead of all the rest of the infrastructure behind AI/LLM datacenters. It’s time to invest in the next frontier of AI infrastructure folks! Not just Cloud compute, hybrid cloud+edge, networking, power, edge RL, edge inference and edge firmware orchestration just to name a few areas we’re excited about!

75

5 Comments

With Wizerr AI, as a vertical AI solution fine-tuned for the electronics industry and an early adopter of NVIDIA NIMs, enterprises can maximize their AI investments. Using accelerated infrastructure with NVIDIA NIMs, and an industry specific model, 3X Generative AI tokens could be produced with the same compute resources - boostings efficiency and allowing enterprises to get the most out of their AI infrastructure. Let the AI Games Begin!

51

At Corgnit Research Lab, India, we are excited to share our latest findings on the convergence of Reinforcement Learning and Federated Learning and their implications for advancing AGI. Key Insights: - 𝐒𝐜𝐚𝐥𝐚𝐛𝐢𝐥𝐢𝐭𝐲 𝐚𝐧𝐝 𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐜𝐲: The distributed nature of FL enables scalable and efficient training of RL models, significantly reducing computational overhead and accelerating convergence. - 𝐏𝐫𝐢𝐯𝐚𝐜𝐲-𝐏𝐫𝐞𝐬𝐞𝐫𝐯𝐢𝐧𝐠 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: By ensuring that sensitive data remains local, FL facilitates the development of AGI in privacy-sensitve domains, such as healthcare, without compromising data security. - 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐯𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠: Multiple RL agents can share knowledge through FL, enhancing overall learning efficiency and robustness without the need to share raw data. - 𝐏𝐞𝐫𝐬𝐨𝐧𝐚𝐥𝐢𝐳𝐞𝐝 𝐀𝐝𝐚𝐩𝐭𝐚𝐭𝐢𝐨𝐧: The integration of FL allows AGI models to be tailored to individual users or environments, combining global intelligence with local customization. - 𝐑𝐨𝐛𝐮𝐬𝐭𝐧𝐞𝐬𝐬 𝐚𝐧𝐝 𝐑𝐞𝐬𝐢𝐥𝐢𝐞𝐧𝐜𝐞: Aggregating diverse experiences through FL results in more generalized and resilient AGI models, mitigating the risk of overfitting. Benefits for Researchers: - 𝐑𝐞𝐬𝐨𝐮𝐫𝐜𝐞 𝐎𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧: Leveraging distributed computational resources lowers costs and enhances the feasibility of large-scale AGI experiments. - 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐂𝐨𝐥𝐥𝐚𝐛𝐨𝐫𝐚𝐭𝐢𝐨𝐧: Model updates can be shared instead of raw data, promoting cross-institutional and interdisciplinary research. - 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐞𝐝 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Faster training times and rapid iteration lead to quicker advancements in AGI. - 𝐄𝐭𝐡𝐢𝐜𝐚𝐥 𝐀𝐈 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭: Prioritizing user privacy and data security aligns with ethical standards and regulations, fostering responsible AI research. - 𝐑𝐞𝐚𝐥-𝐖𝐨𝐫𝐥𝐝 𝐀𝐩𝐩𝐥𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬: Immediate feedback and improvements from real-world data enable practical deployment in sectors like smart cities, healthcare, and IoT. Our research at Corgnit underscores the transformative potential of merging RL and FL in the pursuit of AGI. We believe these insights will catalyze further innovations and collaborations in the AI community. #ArtificialGeneralIntelligence #ReinforcementLearning #FederatedLearning #AIResearch #CorgnitResearchLab #PrivacyPreservingAI #ScalableAI #EthicalAI Corgnit, Corgnit Research India, Reinforcement Learning, Federated Learning Consortium

20

The keynote speech by Trevor Cai from #OpenAI at #HotChips 2024 was interesting. Trevor's presentation reiterated the ongoing hypothesis at OpenAI—that the AI models must continue to grow to achieve superior performance, with no apparent end in sight... He bolstered this argument using a log-log power law plot, which essentially plots the relationship between compute (in FLOPs) and model performance (i.e., loss) on a logarithmic scale. This approach has enabled OpenAI to accurately predict the performance of models like GPT-4 by curve-fitting the smaller models on the plot, thus demonstrating the scalability of this method. Trevor then drew a fascinating parallel to Moore's Law, which famously predicted the doubling of transistors on chips every two years. This prediction held true for decades, fueling massive advancements in semiconductor technology. He suggested that similar underlying principles might sustain the exponential growth of AI models for a significant time, though the mechanisms might evolve. However, Trevor notes that such growth isn't without its challenges. He highlighted several critical issues, including the need for advanced data centers, enhanced reliability, availability, and serviceability (RAS) systems, and more sophisticated power management strategies. The massive scale of synchronized training introduces vulnerabilities like power draw jitters and single-point failures that can cause widespread disruptions. If synchronized training causes so many challenges, I wonder if the research community is focusing on alternative algorithms for LLMs that don't require gradient synchronization after each iteration.? 🤔 💡 Trevor points out that as AI models became more intelligent, the demand for inference has grown significantly. He notes that while training infrastructure can be planned and scaled, inference capacity must be more responsive to actual usage and demands, making planning more challenging and dynamic. The technology and economics of inference will continue to evolve. While the conference is about hot chips, he has no specific recommendations on GPU/accelerator architectures, whether inference could benefit from custom hardware, etc. Overall, it is evident that openAI will continue to grow the models and datasets it trained on for each new release of GPT and will continue to challenge the AI/GPU and networking hardware industry to support this with technological advancements. At some point in the future, more than half the data on the Internet will have originated from the LLM models. At that point, how would the log-log plot look? 🤔 Wouldn’t models trained on their self-generated data collapse, and the losses start to grow? He acknowledges the problem in Q&A and says they are "investigating" techniques to overcome this…. Any thoughts? #AI #LLMs #datacenters #junipernetworks

83

8 Comments

Great work by the team producing this collection of assets on how to use NVIDIA NIM on AWS to optimize your inference performance to get state-of-the-art deployments. Running inference endpoints across Amazon's NVIDIA GPUs instances on EC2, Amazon EKS, and Amazon SageMaker has never been faster or more efficient. Here’s why NVIDIA NIM is the best solution for AI developers and enterprises: ✅ Lightning-fast Deployment: Prebuilt containers with optimized generative AI models, deployable with just a few commands, help you get started in minutes. ✅ Unparalleled Performance: Experience up to 2.5x higher throughput, 4x faster time-to-first-token, and 2.2x faster inter-token latency compared to other open-source solutions—because speed and efficiency matter when scaling AI. ✅ Seamless Integration with AWS: NIM is purpose-built to integrate with AWS managed services like EKS, EC2, and SageMaker, providing a streamlined path from development to production, no matter where your AI models run. ✅ Enterprise-Grade Security and Flexibility: Maintain full control of your data, IP, and custom models while scaling AI workloads in a secure and enterprise-ready environment. ✅ Cost-Effective and Scalable: Reduce time-to-market and operational costs, optimizing response times and throughput, all while leveraging the robust infrastructure of AWS. Whether you’re deploying large language models or domain-specific AI, NVIDIA NIM on AWS empowers you to deliver high-performance AI applications, bringing vision to reality at scale. #GenerativeAI #AIinference #NVIDIANIM #AWS #EC2 #EKS #SageMaker #AI https://lnkd.in/e722U8Np Abhishek Sawarkar Brian Pickering Guy Farber Adel El Hallak Jiahong Liu Timothy Ma Saurabh Trikande Vara Bonthu

117

3 Comments

Great to see testing of AI systems gaining traction! Right now the focus is on the very ML concept of Evaluation. The future, imo, lies in the merging of XP-style Test Driven Development and ML-style evals. Probabilistic tests, written by the developers themselves, to enable rapid iteration and improved reliability of AI apps.

11

5 Comments

If you're designing very big chips like specialized wafers for deep learning, you're probably running into some very computationally challenging workloads during manufacturing. Designing chips with EDA software is extremely time and compute intensive. While it'd be nice to run verification workloads on cloud spot instances and save several million dollars, it seems impossible. Particularly in the following phases: 1. Defect Detection and Yield Management - Advanced scan chain diagnostics - Yield analysis - Redundancy implementation All use extensive computational lithography, DFM (Design for Manufacturability) analysis, and fault-tolerant design techniques. These can require thousands of CPU cores (possible GPU supplement) and weeks to months of continuous runtime. 2. Static Timing Analysis (STA) and Timing Closure As chips get larger, the sheer size makes the following analyses vital: - Clock tree synthesis (CTS) - Clock domain crossing (CDC) verification, and - Timing margin analysis Timing sign-off processes are computationally expensive and can require terabytes of RAM and several months of analysis. 3. Parasitic Extraction and Thermal Modeling - Advanced RC extraction algorithms - Full-chip electrothermal simulation and - Power integrity analysis Require high resolution modeling, which means very high compute requirements and even petabytes of storage. _____ Seems impossible right? Using memory checkpointing, I've been working with some customers to test checkpoint + restore in these vital verification and extraction workflows. We're starting to see some promising results and savings that don't require manual intervention. Curious to learn what chip systems engineers are seeing. DM me if you'd like to learn more! #AWS #Semiconductors #Cadence #HPC

6

Get ready for Anyscale’s 2024 Ray Summit by checking out a new piece in SiliconANGLE & theCUBE, outlining what to expect on October 1st. The article highlights Anyscale’s recent announcements like the appointment of Keerti Melkote to CEO and new partnerships that will expand its LLM and scalable AI offerings. Learn more here: https://bit.ly/3TPoNIP #Anyscale #Summit #LLM

11

🌟 Unlock the Power of Generative AI with Andrew Ng! 🌟 DeepLearning.AI has launched "Generative AI for Everyone"—a groundbreaking course designed to make generative AI accessible to all. 🚀 🎓 What You'll Learn: Core Concepts: Understand the fundamentals of generative AI and how it works. Real-World Applications: Discover how tools like ChatGPT, DALL·E, and more are transforming industries. Business Strategies: Learn how to integrate generative AI into your organization to boost productivity and creativity. Practical Insights: Hands-on exercises to help you grasp the technology, even without a technical background! 👩‍💻 No coding experience? No problem! This course is tailored for everyone—professionals, business leaders, and anyone curious about AI's potential. ✨ Ready to shape the future with generative AI? Enroll now and start your journey today! ➡️ deeplearning.ai #GenerativeAI #ArtificialIntelligence #Upskill #AIInnovation

6

Large language models can be applied to all kinds of data, not just natural language and code. In this post co-written by Hugging Face and Intel AI, we'll will show you how to work with the state-of-the-art ProtST protein language model, and how the Intel Corporation Gaudi 2 accelerator does a nice job at accelerating inference and fine-tuning versus NVIDIA GPUs 😀 https://lnkd.in/eCGcqRfF #ai #opensource

72

3 Comments

Good listen: Mark Suster, the guest host, is joined by Dan Senor to discuss the remarkable growth of NVIDIA in #Israel, which now boasts over 4,000 employees and a global CTO who is Israeli. The conversation shifts to Israel’s startup ecosystem, highlighting its impressive statistics: 500 venture capital firms, 120 accelerators, 35 incubators, 400 multinationals with major R&D operations, and 7,000 startups. These figures underscore Israel’s position as having the highest density of startups globally, attracting significant global venture capital, leading in R&D spending relative to GDP, and ranking fourth in NASDAQ-listed public companies. Regardless - I'll recommend following This Week in Startups podcasts which brings interesting conversations with leaders from the startup community... https://lnkd.in/eGv3fr-H #iwork4dell

7

1 Comment

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

The session at re:Invent “how Netflix benchmarks Foundation Models (FMs) and LLMs across hardware chipsets” by Amit Arora and Amer Ather highlighted some critical considerations for model benchmarking, including inference server configurations, resource optimization, and performance metrics across different hardware chipsets. Benchmarking and evaluating models becomes essential when users take their applications external facing. When this happens, three aspects become the top of mind for them: Inference at scale, cost to serve and model accuracy. To choose the right model for your generative AI workload, it is essential to target all three vectors: Performance efficiency (latency optimizations, token throughput), Cost optimization (dollar cost per transaction for a generative AI workload that serves a given number of users) and model accuracy (either on domain specific or open source data). In the session, FMBench is highlighted as a tool that targets all three vectors above and provides the most optimal recommendations that users can streamline their genAI decision making with. FMBench offers a comprehensive feature set, offering customization through YAML files, support for multiple inference engines (Triton, vLLM, TensorRT, DJL, HF TGI), deployment support for AWS silicon/GPU/CPU hardware and built-in analytics capabilities. What stands out is the flexibility it offers in benchmarking any model (this includes custom, fine tuned models), on open source/custom datasets, and evaluate those in a scalable way using a panel of LLM judges. You can use this tool today for your use case and benchmark for your generative AI requirements. Here are some helpful resources to get started: 1. Introductory Videos (https://lnkd.in/eXSTHeKd, https://lnkd.in/egzqbSa8): These are introductory/set up videos that contain information on FMBench, how to set it up and a walkthrough of a sample report. 2. FMBench Website (https://lnkd.in/eaEEXAPr): This website is fully searchable to ease common tasks such as installation, finding the right config file, benchmarking on various hosting platforms (EC2, EKS, Bedrock, Neuron, Docker), model evaluation, etc. 3. FMBench GitHub Repository (https://lnkd.in/eM8mmHrJ): The FMBench GitHub repository is open source. Get started with benchmarking and evaluating for your use cases today and reach out if you have any questions or ideas! Manju Prasad #AWS #MachineLearning #Benchmarking #GenerativeAI

122

8 Comments