Ravi Jain

The Token Wars of the chip startups. Why the fight for the fastest inference will influence the future of AI. „Today, Cerebras, SambaNova, and Groq are all delivering over 1,000 tokens per second, and the “token wars” have revved up considerably. At the end of August, Cerebras claimed it had launched the “world’s fastest AI inference” at 1,800 tokens per second, and last week Cerebras said it had beaten that record and become the “first hardware of any kind” to exceed 2,000 tokens per second on one of Meta’s Llama models. “ „AI models are quickly being used to power far more complex applications than just chat.“ Cerebras CEO Andrew Feldman simplifies why: „Nobody’s going to build a business on an application that makes you sit around and wait“. A piece by Sharon Goldman at Fortune. https://lnkd.in/eMs7Pxux

1

An interesting alternative emerges to transformer-based architectures, and here comes a new acronym - the LFMs...MIT spinoff Liquid debuts non-transformer AI models and they’re already state-of-the-art https://bit.ly/3TQdvnO #govtech #AI #genAI

4

Hot off the presses! A new paper with Nick Laws and Maggie Chen -- "Valuing Distributed Energy Resources for Non-Wires Alternatives" Main takeaway: Distributed energy resources (DER) such as batteries -- regardless of whether it's the utility or homeowner who owns the system -- can reduce system operating costs and delay system upgrades for distribution systems. For example, using DER might help reduce how often local transformers might need to be upgraded. "We show that when valuing DER the operating costs over 20 years can be reduced by over 40%." Let's deploy some batteries, y'all! #batteries #futureofenergy #distributedenergyresources #DER Walker Department of Mechanical Engineering, The University of Texas at Austin Webber Energy Group

80

11 Comments

The General Services Administration's recent announcement of a new class of Presidential Innovation Fellows dedicated to #AI is a significant step forward. By focusing on AI, these fellows can drive advancements while ensuring that the technology is used ethically and responsibly to protect both individuals and organizations. Initiatives like these help bridge the gap between rapid technological innovation and robust regulatory frameworks, ensuring that AI serves the public good. https://bit.ly/3RLNNzD

4

I continue to be alarmed at the progress of proposed California regulation SB 1047 and the attack it represents on open source and more broadly on AI innovation. As I wrote previously, this proposed law makes a fundamental mistake of regulating AI technology instead of AI applications, and thus would fail to make AI meaningfully safer. I’d like to explain why the specific mechanisms of SB 1047 are so pernicious to open source. To be clear, there are routes that regulators should pursue to improve safety. For example, I would welcome outlawing nonconsensual deepfake pornography, standardizing watermarking and fingerprinting to identify generated content, and investing more in red teaming and other safety research. Unfortunately, the proposed bill pursues a less beneficial and more harmful path. SB 1047’s purported goal is to ensure safety of AI models. It puts in place complex reporting requirements for developers who fine-tune models or develop models that cost more than $100 million to train. It is a vague, ambiguous law that imposes significant penalties for violations, creating a huge gray zone in which developers can’t be sure how to avoid breaking the law. This will paralyze many teams. You can read the latest draft of the law online. I’ve read through it carefully, and I find it ambiguous and very hard to follow. Developers who try to navigate the law’s complex requirements face what feels like a huge personal risk. It requires that developers submit, under penalty of perjury, a certification of compliance with the requirements of the law. But when the requirements are complex, hard to understand, and can even shift according to the whims of an unelected body (more on this below), how do we ensure we are in compliance? For example, the certification must include many different sections. One is an analysis of “the nature and magnitude of critical harms … the model might reasonably cause or enable.” But given that even leading AI researchers aren’t sure what harms models might cause or enable, how is a team of developers supposed to figure this out and declare — under penalty of perjury — that they meet this requirement? Further, some developers will be required to implement “protections to prevent … misuse of, or unsafe post-training modifications of, the covered model and all covered model derivatives … that are appropriate in light of the risks associated with the covered model, including from advanced persistent threats or other sophisticated actors.” Even leading AI researchers don’t agree on how best to “protect” AI models against these supposed risks, or what would be “appropriate.” So how are developers supposed to figure out how to comply with this requirement? [Reached LinkedIn length limit. Rest of text here: https://lnkd.in/g_NhQpni ]

2,721

202 Comments

Lately, I have been inspired by the neural operator work and research coming out of Caltech led by Anima Anandkumar. I was reflecting today that perhaps we aren’t approaching the energy storage and density dilemma in the right way. We tend to only think in terms of battery chemistry or fossil fuels, but what if there’s a third category that we haven’t yet discovered? And what if we could unlock that discovery using AI-driven methods? I don’t think most people fully grasp the staggering difference in energy densities between batteries and fossil fuels. If we want to fully transition to renewable energy, we need to make significant investments in advancing battery chemistry or discovering an entirely new approach. Batteries - Alkaline (zinc-manganese dioxide): 0.4-0.59 MJ/kg - Zinc-carbon: 0.13 MJ/kg - Nickel Cadmium (NiCd): >0.14 MJ/kg - Lead Acid: >0.14 MJ/kg - Nickel Metal Hydride (NiMH): >0.36 MJ/kg - Lithium-ion batteries 0.36-0.875 MJ/kg (theoretical limit 43.2 MJ/kg) Fossil Fuels - Coal 24 MJ/kg - Gasoline 46 MJ/kg - Uranium-235 3,900,000 MJ/kg (this is crazy!) What excites me about Anima and her team's work is how using neural operators they've been able to make monumental increases in modeling speeds. For instance, AI-based modeling using their neural operators is 45,000x faster than current weather forecasting models. There is so much more we could learn about how different elements and natural factors play roles in areas such as electromagnetism, electrostatic forces, subatomic electronic charge, and much more. #AI #energy #modeling #batteries #research https://lnkd.in/enE-U3uN

4

Groq, known for their high-performance AI hardware and software, released two new open-source models specifically designed for tool use: Llama-3-Groq-70B-Tool-Use and Llama-3-Groq-8B-Tool-Use, built with Meta Llama-3. Tools are utilized by AI assistants to respond to user prompts. Typically, a tool is an API call to an external system, like Google search. The results from the tool are then analyzed by the LLM and incorporated into the response. Thus, the LLM's performance in using tools is paramount. These models promise significant advancements in processing speed and efficiency. Llama-3-Groq-70B-Tool-Use boasts 70 billion parameters, while Llama-3-Groq-8B-Tool-Use features 8 billion parameters, both optimized for rapid deployment in various applications. By leveraging Groq’s advanced hardware, these models aim to set a new standard in AI tool use, focusing on enhancing both performance and accessibility.

1

💡 Transforming Transformers - the Transformer architecture powers the most popular public and private AI models today. We wonder then — what’s next? Is this the architecture that will lead to better reasoning? What might come next after transformers? 🔎 In addition to the dominant Transformer architecture, we’re now seeing production grade State Space models (SSM), hybrid SSM-Transformer models, Mixture of Experts (MoE) and Composition of Expert (CoE) models. These seem to perform well on multiple benchmarks when compared with state of the art open-source models. ⚡ With a simple sketch and multi-modal models, non-technical people can build simple applications without much effort. Knowing how to use such tools can be a superpower, and it will help anyone who is looking to excel in their careers. Umesh Padval and I cover some of these topics in our latest blog on enterprise AI (link in comments). Please let us know if there is something else that's keeping you from rolling out AI capabilities to your workforce? #ai #founders Thomvest

40

3 Comments

Intersection of AI and humanoid robotics presents one of the biggest opportunities for societal scale impact Perception, Sensors, and LLMs are merging. Looking out for break out ventures working at this intersection. ! #EmbodiedAI https://lnkd.in/emK-nVNx

18

5 Comments

Meta’s Fundamental AI Research (FAIR) team, VSPARTICLE (VSP) and the University of Toronto (UoT) have published a sizable open-source experimental catalyst #database as a ground-breaking first step towards fighting #climatechange. This initiative represents a turning point in clean energy research by accelerating the identification of key ingredients for #decarbonization using #AI and #nanotechnology. VSP used its spark ablation-based VSP-P1 nanoprinter to make hundreds of different materials by vaporizing them and placing them as nanoporous films.  “Due to the high temperature of the spark, the solid material evaporates, forming the smallest particles you can make on Earth,” Aaike van Vugt, co-founder and CEO of VSP, said in an interview with EE Times. Larry Zitnick, research director at Meta AI Facebook , shared how Meta’s AI models are advancing the identification of electrocatalysts—materials essential for clean energy applications like water splitting for #hydrogen production.

77

In my new podcast episode, I chatted to Lemurian Labs CEO Jay Dawani about the gap between supply and demand for AI compute, and how much of the deficit will be made up by hardware, software and model advancements. A fascinating conversation which is well worth a listen, and stick around to the end to find out what Lemurian, a former AI chip company, have pivoted to. Thanks Jay for the always-intriguing conversation! #ai #aimodels #aihardware #aichips #aiinfrastructure #aisoftware https://lnkd.in/ekB-nB-x

54

2 Comments

Robotics and AI will touch all aspects of life and society. This one “could” be a path in reducing or eliminating the herbicides used on crops. On the technical side, from the article, this is astounding. Who would have thought even 10 years ago GPUs would be utilized in this manner. “Powered by 24 NVIDIA GPUs, Carbon Robotics’ vehicles use deep-learning-based computer vision models to autonomously identify and eliminate weeds via CO2 lasers. LaserWeeder, as the product is called, processes 4.7 million high-resolution images per hour, zapping 5,000 weeds per minute, according to the company. To date, its dataset includes 25 million labeled plants and more than 30,000 crop and weed models” #AI #agritech #robotics #nvidia

3

1 Comment

🌟 Geoffrey Hinton Awarded Nobel Prize in Physics 2024 This year, the Nobel Prize in Physics goes to two pioneering minds—Geoffrey Hinton and John Hopfield—whose groundbreaking work with neural networks laid the foundation for modern AI and machine learning. Their methods, inspired by physics, have helped revolutionize how machines can recognize patterns and even "learn." Yet, there’s an irony here. Hinton, once a vocal advocate of AI’s potential, has significantly shifted his stance. Where he once championed AI, now he cautions us about its rapid, unchecked progress. He warns of job loss, societal disruption, and even the risks of artificial general intelligence (AGI) gaining self-preservation instincts. The acceleration of AI’s capabilities—what he initially predicted might take 50 years—has shocked even him. But in this shift, we can find balance. 🚀 While Hinton raises alarms, I remain cautiously optimistic. If AI is the "airplane of the mind" 🌍This is what I mean in this linkedin pos 🔗t: https://lnkd.in/gsxpNm_U , it can take us to new heights of innovation and connectivity. ✈️Like aviation, we need regulations and ethical frameworks—but the potential for transformative impact is vast, from healthcare to global collaboration. #NobelPrize #AI #GeoffreyHinton #MachineLearning #Innovation #FutureOfWork #EthicsInAI https://lnkd.in/gVGnxB4F

8

“You know, you can trust things that you can articulate” As part of our efforts on Trusted AI-assisted programming, the paper demonstrates effectiveness of LLM in generating formal postconditions, can be executed or analyzed with several use cases to be presented in FSE’24 in Brazil with Madeline Endres, Sarah Fakhoury, Shuvendu Lahiri. Our ambition is to equip the AI assisted programming system enough capabilities to fomally analyze (and at the extreme verify) the AI generated code, so that the programmer does not have to blindly trust the system. First step to verification is formalizing the specifications. This paper shows that the specifications (e.g., postcondition of a method) can be effective formalized with AI agents. In future work, we also explore the proof part of the verification with assistance from LLMs. No debate, LLMs do show tremendous emergent capabilities. Modern LLMs are able to answers questions (such as writing a syntactically correct code in a completely made up language, only with few examples), that are impossible to imagine 5/6 years ago. We aim at harnessing such capabilities, while ensuring that we get exactly what we want by imposing constraints, such as specifications, verification tools, etc.

45

Last week I had the unbelievable opportunity to meet Jensen Huang. His visionary approach building NVIDIA and insights into the future of technology provided a powerful perspective that resonates deeply with our opportunities, challenges and ambitions in the international development and humanitarian aid sectors. Here are three pivotal lessons I took away from our discussion: 1. 𝗜𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻 𝗮𝘀 𝗮 𝗥𝗼𝗮𝗱𝗺𝗮𝗽 - In sectors like ours, where technology adoption may lag, Mr. Huang's journey with NVIDIA serves as a blueprint for integrating cutting-edge solutions that can scale effectively and sustainably.  2. 𝗖𝗿𝗼𝘀𝘀-𝗦𝗲𝗰𝘁𝗼𝗿 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 - Engaging with leaders from tech-forward industries underscores the importance of leveraging cross-sector insights. This approach will catalyze our impact and help us better navigate the complexities of digital transformation.  3. 𝗖𝗼𝗿𝗲 𝗜𝗻𝘁𝗲𝗴𝗿𝗮𝘁𝗶𝗼𝗻 - At DevResults, we are inspired to go beyond using technology as an adjunct tool. Instead we're embedding it as a core element of our strategy, enhancing how we drive change and deliver value for our partners worldwide. As our industry declares,"𝘋𝘦𝘷𝘦𝘭𝘰𝘱𝘮𝘦𝘯𝘵 𝘪𝘴 𝘋𝘪𝘨𝘪𝘵𝘢𝘭", these lessons fortify our commitment to harnessing innovative technologies not merely as enhancements but as essentials for our collective mission. It's quite an exciting time to be working at the intersection of development and digital! #TechForGood #DigitalDevelopment #Leadership

98

3 Comments

Last week, I had the privilege of attending the Stanford Accelerator for Learning summit, which brought together 400 thought leaders in education technology—including educators, researchers, funders, and innovators. Kudos to Isabelle Hau and her team for orchestrating such an inspiring event. Few Key Takeaways and Observations: 1. Teacher Empowerment is Paramount: Successful EdTech tools are the ones that have placed Educators in the driver seat. Technology must support, not replace, educators. At Merlyn Mind, we believe teachers are the central force driving student engagement and learning experiences in classroom. 2. Responsible AI - Now a Standard Expectation: AI conversations have evolved beyond mere discussion of safety risks and more towards effective AI. My optimistic belief is that what was a hot topic in 2023 has now become a baseline requirement. At Merlyn Mind, we've proactively integrated responsible AI practices into our product development process. 3. Leverage data to assist: Vast amounts of data are available on Learners. AI presents great opportunities to leverage learner data to provide useful and actionable insights. By understanding interaction patterns and learning styles, we can create personalized experiences that support educators, parents, and students. 4. Evidence-based Research for Rapid Innovation: Traditional peer-reviewed research can't keep pace with new tools that are being developed. We need more dynamic, flexible research methodologies to support fast-evolving educational tools. Iterative, multi-stage evidence needed to understand impact. Involve teachers and schools directly in product development process. 5. Educational Infrastructure: Our education systems must adapt to changing learning landscapes. With technology transforming how students learn outside classrooms, we need new systems that reshape traditional learning and assessment models that can adapt to the new environment. These events also become a great venue for exchanging ideas, reconnecting with peers. Erin Mote Steven Schneider Malvika Bhagwat Chris Agnew Richard Charles, Ph.D., GSP Betsy (Elizabeth) Corcoran #EdTech #AIinEducation #FutureOfLearning #InnovationInEducation

76

3 Comments

International collaboration remains on the ground (Q1, Q2, Q3) regardless of political clamor driven by different interests. Q1: "The public testing milestone has been several years in the making. Waymo first announced plans to build an electric autonomous ride-hail vehicle with Zeekr for the U.S. market in December 2021." Q2: "The new minivan-esque vehicle, which appears to be modeled after Zeekr’s upcoming Zeekr 009 minivan, is outfitted with Waymo’s sixth-generation hardware — a more cost-effective set of sensors than previous versions that is also designed to operate in winter environments." Q3: "Meaning that until Congress or the National Highway Transportation Safety Administration comes forward with a ruling about Chinese connected vehicles, the DMV will not change the way it issues permits." https://lnkd.in/gmRq4_Nr

2

New Tesla Patent Reveals Clues to an Exciting Future of Humanoid Robots and AI Integration In a interesting discussion, Herbert Ong and Scott Walter dive deep into Tesla's recently published patent application WO2024072984A1 (link in comments). Entitled "Actuator and actuator design methodology," the patent primarily focuses on controlling robot movement through strategic actuator placement. However, the document also unveils important clues about the potential for end-to-end neural net training of humanoid robots, reminiscent of Tesla's approach to training its vehicles with FSD. Ong and Walter explore the concept of personalized humanoid skills. Imagine purchasing a bot with built-in capabilities, then having the option to augment and customize those skills through a "Skills Store" akin to Apple's App Store. This opens up a world of possibilities for tailoring robots to specific tasks and preferences. The discussion also touches on the potential for various robot form factors, each designed to excel at particular skills. Picture a tennis-playing robot that comes pre-trained with basic volleying instructions, but requires a "Skills Upgrade" to perform at a master level. This raises questions about the need to manufacture and train a diverse range of robot body types and sizes, mirroring how humans possess ideal physiological characteristics for different activities. Later, Ong and Walter discuss a scenario in which Tesla builds the hardware – including automobiles, humanoids, and other bot form factors – while xAI develops the brains. This arrangement opens up opportunities for licensing agreements between the two companies, that may help the companies navigate future antitrust concerns as these technologies are deployed at scale. The convergence of advanced robotics and artificial intelligence holds the potential to catalyze a profound societal transformation. As Tesla and xAI continue to push the boundaries of what's possible, I can envision a future where intelligent, adaptable robots become integral to various aspects of our lives – from manufacturing and transportation to healthcare, recreation, and beyond. The seamless integration of cutting-edge hardware and software will revolutionize the way we live and work, ushering in an era of unprecedented efficiency, productivity, and innovation. While there are still challenges to overcome and ethical considerations to address, the prospect of these combined technologies driving positive change on a global scale is awe-inspiring. #FrontierFrontier #Humanoids #AI #Robots

7

1 Comment