Karan Goel

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

55

2 Comments

Excited to be invited to give a guest lecture at BITS Pilani, on "Building AI Agents in Production." Here’s what I’ll be covering: 1. What is an AI Agent? How does it differ from an LLM? 2. Autonomy in AI Agents: Exploring how planning and reasoning enable autonomy. 3. Solving a Task Traditionally: We'll tackle a task manually to understand the challenges. 4. Building an AI Agent Together: We'll code an AI Agent from scratch and let it autonomously solve the task. No prior knowledge of any specific framework is needed—we’ll build an AI agent from scratch! 📅 Date: Tuesday, November 26 ⏰ Time: 7:30 PM IST / 6:00 AM PT / 3:00 PM CET BITS has generously opened the event to external participants as well! So, even if you're not a part of BITS, you're more than welcome to join. This session will be particularly insightful for anyone eager to truly understand what an AI agent is and learn how to build one—cutting through the hype to focus on real-world implementation. The session will be remote. Event details are in the comments. Hope to see you there! 🚀 Thank you very much Dhruv for the invite!

121

14 Comments

Over last few months, I am increasingly leveraging LLMs to write code. From 10% to nearly 78% currently (based on lines of code) of my code is from LLMs. I have my own pipeline to fine tuned LLMs to augment my needs. Observations - 1. I am producing more code --> MORE IMPACT (approx 2.3X) 2. I am reading more code --> MORE MAINTAINABLE CODE 3. I am learning more as now I can start from concept and expand out ---> MORE LEARNING 4. I am incorporating best practices --> BETTER CODE 5. Lesser typing --> LOWER REPETITIVE STRAIN INJURY #AIBUDDY

76

8 Comments

Here's why prompt engineering is one of today's most valuable skills. VentureBeat recently published an article by Deven Panchal of AT&T Labs. I applaud Deven for his vision. He says, "A well-crafted prompt can significantly impact the quality and relevance of the output of LLMs." So many are refusing to accept this. Mainly because prompting uses natural language. They are misguided, assuming AI can expertly prompt for them. Even if it can someday (which I doubt), it can’t today. The closest thing is SmartGPT 2.0, which still can’t compete with an expert prompt engineer. I recommend learning and practicing prompt engineering. 🟢 If you don't know where to learn, comment "START" and I'll point you in the right direction. --------------------------------------------- See comments for a link to the full article. 🔗 Follow 🔔Mark Fulton for more on #AI #Prompts #AIAgents

27

18 Comments

Loved all the thoughtful audience interaction at my "Deep Learning with PyTorch and TensorFlow" workshop at ODSC East yesterday. Halfway through, today's session on "Generative A.I. with LLMs" has been a blast so far too. Hope to meet you at Open Data Science Conference (ODSC) East in Boston this week if I haven't already! #datascience #odsc #odsceast #machinelearning #ai #generativeai

86

5 Comments

We had a great session about NVIDIA NIMs with Anshul Jindal and Mark Moyou, PhD yesterday. 🚀 From Mark: We got lots of info on inferencing with NIMs and different types of NIMs like embedding models, generative models, reranking models.. 🎨 I walked through how you can use NIMs for embedding, retrieval and generation with #Haystack 🚀 And finally, Anshul did a live coding session where he showed how you can deploy and scale your own self-hosted NIMs. If you missed it, or don't even know what I'm talking about but you wanna learn, the recording is up (adding to the comments👇) All resources, including the example repos and guides are in the description. #AI #LLM #Nvidia #opensource

36

2 Comments

Evaluations remain a persistent pain point in generative AI. As Anyscale builds out its own LLM-powered features, co-founder and CEO Robert Nishihara noticed what will be a growing problem: how to evaluate generative models. This challenge compounds as customized and fine-tuned models proliferate. With traditional machine learning evaluation was straightforward: → Start with your test data set → Run the model against it → Get an accuracy score That's impossible with natural language, where outputs are open-ended and benchmarks are not absolute. How do you determine if one response is better than another? And what defines "better"? Most existing evaluation frameworks come from academia and don't map to business use cases. The subjectivity of generative outputs makes them difficult to evaluate systematically. This presents a major opportunity for startups heading into 2025.

55

14 Comments

My bridge-building days taught me something about building product that even YC did not: The 'The 20/50/80 method:' design process. (It's really 30-60-90, but who's counting) SOORAJ PREMKUMAR and I modified it a little for our use. The exact approach I used to build Zoko's premium product from day one—so we never had to rebuild it. Here's how it works: 20% Design: throw sh*t at the wall • We mock up basic concepts • Get feedback from customers and internal users • Make sure all the moving parts and pieces are included (doesn't need to work well with each other) 50% Design: start getting serious • We've got a solid figma prototype • Again show different types of customers • We watch where they click, what confuses them • This is where the real insights happen • Engineers go to war with it - "A popup over a popup? Are you insane?" "That swipe is only for tinder, that shit wont work on desktop" "Auto-refresh here will nuke my cloud bills" 80% Design: final tweaks • Engineering review • Last chance to catch any f*ck-ups • Sometimes it can even be kicked back to the drawing board • Then we ship it Real example: We had this feature where you could switch between agent chats. Initially, we just had a down arrow. Guess what? Users didn't get it. So between 60% and 90%, we added the word "Change" next to the arrow. Boom. Problem solved. So adding words instead of icons work right? Another one: We had "Unread" and "Unreplied" filters. Customers asked "Isn't "Unreplied" the same as "Unread"? They did not get the "Unreplied" was "Read but not replied". Everyone was sufficiently confused! The point is: you can't predict this shit. You have to observe, iterate, and sometimes admit you were wrong. "But Arjun, this sounds like it takes forever!" Yeah, and rebuilding your product because you shipped garbage takes longer. Choose your pain. Battle Rules: → Get bloody feedback early → Figma battles are cheaper than code wars → Kill your darlings before they kill your product Try to build software like the best bridge builders in the world - they make sure it wont break before they let people drive on it. Your users deserve the same respect.

59

8 Comments

Prof. Rao's post is a good starting point for LLM practitioners to become familiar with classic (i.e., predating LLMs) AI concepts and theories, for two reasons: 📌 To understand that some things are theoretically beyond the reach of LLMs and not waste resources chasing them. 📌 To hedge against the possibility that LLMs may be replaced by other technologies and/or undergo fundamental technological revamps, which would render all existing investments in LLMs obsolete. #artificialintelligence #machinelearning #deeplearning

41

3 Comments

🚀 Day 18 of My 25-Day Generative AI and LLM Challenge 🤖🚀 🌟 How Variational Autoencoders (VAEs) Work: Encoding & Decoding 🌟with my friends HARSH JAVIA and Maanan Jagani! Variational Autoencoders (VAEs) are a type of generative model that aim to learn the underlying structure of data. Here's a simplified breakdown of how they work, using a bit of professional flair and emojis to keep it engaging! 🔍 1. Understanding the Basics🔍 At a high level, VAEs consist of two main components: 1. Encoder 🧠: - Takes input data (e.g., images, text) and compresses it into a lower-dimensional space known as the **latent space**. - Think of this as summarizing the essence of the input data into a compact representation. 2. Decoder 💡: - Takes the encoded representation from the latent space and tries to reconstruct the original input data. - This process helps the model learn to generate new data that is similar to the input data. 🚀 2. Step-by-Step Process 🚀 Step 1: Encoding 📥 - The encoder network receives the input data and transforms it into a mean (μ) and a standard deviation (σ). - From these, a latent vector \( z \) is sampled using the reparameterization trick: \( z = \mu + \sigma \cdot \epsilon \), where \( \epsilon \) is a random variable from a standard normal distribution. Step 2: Latent Space 🌌 - The latent space is a lower-dimensional representation that captures the essential features of the input data. - This space allows the model to explore various possible data representations. Step 3: Decoding 📤 - The decoder network takes the latent vector \( z \) and attempts to reconstruct the original input data. - The quality of the reconstruction indicates how well the model has learned the data's underlying structure. 🌟 3. Why VAEs are Powerful🌟 - Data Generation 🎨: VAEs can generate new, unseen data that is similar to the training data, useful in creative applications like art and music. - Dimensionality Reduction 📉: They reduce the complexity of data, making it easier to visualize and analyze. - Anomaly Detection 🔍: VAEs can identify anomalies by recognizing data that doesn’t fit the learned patterns. 🏆 4. Professional Takeaway 🏆 Variational Autoencoders are a powerful tool in the machine learning toolbox, enabling efficient data compression, reconstruction, and generation. Their ability to model complex data distributions makes them invaluable for various applications, from image generation to anomaly detection. #MachineLearning #AI #DataScience #DeepLearning #GenerativeModels #VariationalAutoencoders #VAEs #NeuralNetworks #DataInnovation #TechTrends #ArtificialIntelligence #MLApplications #DataGeneration #TechInspiration #Innovation #AIResearch #DataAnalytics #DeepLearningModels #TechCommunity

6

Along the same vein, I also want to mention Prof. Rao's (Subbarao Kambhampati) argument that many folks may have misinterpreted LLMs' *approximate retrieval* as reasoning. See https://lnkd.in/e5qeYcdn Also, Prof. Rao's group did rigorous work to show that LLMs cannot plan. See https://lnkd.in/exR8Yb5h https://lnkd.in/eJSS7k-H https://lnkd.in/e62WA7e7 #artificialintelligence #machinelearning #deeplearning 

3

LinkedIn team (Juan Pablo Bottaro Karthik Ramgopal) shared a helpful deep dive into their deployment of an LLM-powered job search assistant. Highly recommend this read! Few highlights:  💡 Relatively easy to setup RAG-based pipeline. Achieved 80% of the basic experience in 1 month, then 4 more months to reach 95% mark. 💡 Evaluating the quality of application outputs is hard. I hear this from many enterprises developing LLM applications. (check out Okareo by Matthew Wyman Boris Selitser) 💡 Balancing latency with quality is important. While certain techniques like Chain of Thought could improve accuracy and reduce hallucinations, it will increase response times, which can affect user experience. https://lnkd.in/gtGGwf5P #largelanguagemodels #RAG #EBR #LLM #ml #ai #hallucinations #evaluations #

4

The best things come in small packages 👝 🚨 Pay attention to the rise of Small Language Models (SLMs)! They will become important going forward: 1️⃣ Efficiency: SLMs achieve comparable or superior performance to LLMs with fewer parameters. Microsoft's new Phi-3 series outperforms larger models on benchmarks like MMLU & MT-bench (Google: https://lnkd.in/gn5zYZHg Microsoft: https://lnkd.in/ghK8fm7V Meta: https://lnkd.in/gkbM7pGs) 2️⃣ Specialization: Trained on curated, high-quality datasets. With 3.8B-14B parameters & advanced datasets, these transformer decoder models match GPT-3.5's capabilities in a compact form. Accurate benchmarking/eval will be important. 3️⃣ Interpretability: Neurons in SLMs are easier to analyze, improving model transparency. 4️⃣ Cost-effective: Less resource-intensive (vs training large models) I have a chance to meet a few exciting startups here. Reach out if you are working on this space 🙋🏻‍♂️ #startups #llm #slm #smalllanguagemodels #phi3 #largelanguagemodels #gpt4 #google #gemma #llama3 #ai #ml #edgeai

24

4 Comments

Ensemble of smaller LLMs (PoLL) is less biased, faster, 7x cheaper when evaluating LLMs performance. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796

9

Can you really build a company the same way you build an engineering team? I recently touched upon this on the ML Street podcast with Tim Scarfe but let me break it down further. If you’re Jensen Huang running NVIDIA, the answer is yes.  With one of the flattest structures in comparison to company size in the world, Jensen manages 50+ direct reports.  Jensen’s centralized approach to management works as employees aren’t deemed more powerful because of the information they possess. This creates a culture of transparency and freedom of information that… 1. Shows people how and why their work impacts overall business objectives 2. Motivates employees when they know what kind of impact they’re making 3. Aligns everyone to company objectives, reducing friction 4. Builds trust as leadership is pressured to be candid and accountable 5. Fosters innovation through information gathering and collaboration across the company At CentML, we’re a blend of all of the above, where we believe in honesty, craftsmanship, and collaboration, not only within our engineering teams, but our company as a whole. Although, we were in fact founded by a group of engineers, so perhaps a bit of the engineering team rubs off on how we’ve grown the company after all. #ai #engineering #hiring #company #nvidia #centml #management #collaboration #honesty #craftsmanship

44

1 Comment

Had the privilege to join a Fireside Chat with Sudivya from the Google Cloud team at IIMA Ventures. Key takeaways: LLM democratization: Making custom LLMs accessible to non-tech users. AI in action: Impactful use cases like RAG and domain-specific automation. Build vs. leverage: Pre-trained models offer speed and flexibility; fine-tuning ensures precision. Tech challenges: Managing data at scale, ensuring compliance, and balancing the economics of GPU-intensive workloads remain complex challenges. Balancing data efficiency with cost-effective GPU scaling via distilled LLMs. Why GCP?: Powerful AI tools, stellar startup support, and global scalability. #DataNeuron #AI #FiresideChat #LLM #GoogleCloud #Innovation #Startups

41

🚀 The need for India-focused GenAI platforms has led to a host of indigenous developments. Let's take a look at 8 Made-in-India LLMs that are shaping the country's future. 🔸 Recognizing the socio-cultural gap in existing systems, entrepreneurs and developers from India are working on a GenAI ecosystem tailored for Indian consumption, yielding better contextual outputs across all formats. 🔸 A handful of homegrown conglomerates, such as Reliance (Jio), Tata Consultancy Services, Infosys, and Mahindra & Mahindra, are already working on a slew of GenAI projects. 🔸 Indigenous developments like BharatGPT, Ola Krutrim, and Project Indus (a Tech Mahindra venture including 40 different Indic languages), are addressing the language challenges pan-Indian users face in real life and now in the AI world. According to Inc42 data, India’s GenAI market is expected to grow exponentially in the next few years, surpassing $17 Bn by 2030 from $1.1 Bn in 2023, growing at a CAGR of 48%. With BharatGPT and its ilk taking charge and cementing their positions in India’s conversational AI landscape, India is well-prepared for a new era of digital empowerment where language barriers are dissolved and access to information is democratized. 🌟 Thoughts on this trend: Cultural and Linguistic Relevance: These platforms address India's diverse socio-cultural and linguistic nuances, ensuring inclusivity and effectiveness. Local Innovations and Entrepreneurship: Investments by companies like Reliance, TCS, Infosys, and Mahindra & Mahindra highlight India's growing tech ecosystem and global AI positioning. Economic Growth and Opportunities: The rapid expansion of the GenAI market signifies substantial economic opportunities, leading to job creation and skill development. Digital Empowerment and Accessibility: By providing AI-driven solutions in multiple Indian languages, these platforms enhance digital literacy and inclusivity. Addressing Real-World Challenges: Indigenous AI platforms deliver solutions closely aligned with the needs and preferences of the local population, improving customer service, education, and healthcare support. The emergence of India-focused GenAI platforms is a promising development driving technological, economic, and social progress in the country. Let's look forward to a digitally empowered and inclusive future! 🚀🌐 #GenAI #AI #India #DigitalTransformation #Innovation #Technology #BharatGPT #OlaKrutrim #ProjectIndus #DigitalEmpowerment #FutureTech #AIinIndia

28

3 Comments

So excited to see more biology open-source models for real positive use-cases of AI (think curing diseases & improving everyone's health!). Chai by Chai Discovery does structure predictions at AlphaFold3 levels of accuracy and able to handle multi-peptide or peptide-ligand complexes rather than just single chains. All in Apache 2.0 on Hugging Face: https://lnkd.in/ez_zgmWD Let's go!

82

6 Comments

Hiring full-time NLP engineer for fast-growing and funded generative AI startup! At Arrowhead, we’re becoming the AI partner for enterprises’ customer support and sales teams. We are working with enterprises globally and are targeting close to $1 mn in ARR by end of the year. We’re building proprietary in-house models that will create 100x value compared to what we can do today. At Arrowhead, you’ll experience a culture where:  🚀 You will have ownership and empowerment to take decisions, even as an intern 🚀 You will see what you’re building directly in the hands of users, and be able to iterate on a weekly basis based on user feedback 🚀 You will be challenged and pushed to do some of the best work you’ve ever done! If you are interested, please fill out the google form in the comments below 👇 #hiring #NLP #startuphiring

139

11 Comments