Samartha Chandrashekar

Large language models are wonderful at ingesting large amounts of content & summarizing. Uploading an academic paper and I can pester it with an infinite list of questions & it will respond with equally infinite patience. In comparing the two most recent Microsoft earnings calls, Claude highlighted: - faster than expected Azure growth (29% vs 27%) - AI contributing 8% of Azure revenue up from 7% - higher CapEx spending & greater capacity constraints for data centers - much better commercial bookings growth : 17% vs 13% Excellent analysis. Benn Stancil described LLMs as great averagers of information. But what if I don’t want the average? If I seek the data point two standard deviations out? If I’m channeling my inner Anthony Bourdain & I’m seeking the fermented shark or cobra heart within my query? At Google, we ranked web pages & ads with many signals. An engineer taught me that of them found rarity of a word across a set of documents. For example, if there’s a document containing 10 instances of the word ghotiacross a collection of documents, where the average document has 0 ghotis, that document is likely the best to answer a search about fish. I haven’t found a way to goad an LLM to produce the rare result. Why did one user decide to use a particular piece of software when 10 others did not? Maybe I haven’t learned how to prompt an LLM well. Ideogram launched a feature called Magic Prompt that expands a basic prompt into richer instructions that elicit better results, marrying the language of a user & a computer. A user might write : A boy with a dog in a park. Magic Prompt replies : A heartwarming scene of a young boy playing with his happy, wagging dog in a lush, green park. The boy has a playful smile on his face while he tosses a bright yellow ball for his furry friend to fetch. The park is filled with beautiful flowers and trees, creating a serene atmosphere for this bonding moment between boy and dog. Magic Prompt surprised me in three ways. First, unlike Google search where terse queries tend to work very well, LLMs relish verbosity. Less Hemingway & more Faulkner. Second, maybe there’s a prompt for rarity. If you know what it is, please share with the rarity shibboleth. Third, I wonder what other prompt techniques are out there that have yet to be discovered to wrest the best from an LLM. This technique is called TF-IDF. It means Term Frequency Inverse Document Frequency.

21

6 Comments

Did you hear? Anthropic's upgraded Claude 3.5 Sonnet model is now on Vertex AI! When building with the upgraded Claude 3.5 Sonnet on Vertex AI, you get a platform to take your workloads from concept to production with fully managed infrastructure, robust security and privacy, and advanced developer tools. Get all the details, here →

1

LLMs are going to be commoditized akin to cloud services in the early 2010s. Big companies and startups alike are going to start building agents on top of these offerings. Whoever can solve the latency, guardrails and security issues will come out on top.

11

$2 H100s: How the GPU Bubble Burst Last year, H100s were $8/hr if you could get them. Today, there's 7 different resale markets selling them under $2. What happened? The market has flipped from shortage ($8/hr) to oversupplied ($2/hr), because of reserved compute resales, open model finetuning, and decline in new foundation model co’s. Rent instead. https://lnkd.in/gpQEidmC

2

1 Comment

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

16

3 Comments

When greentech unicorn Enpal needed to automate its solar panels sales process, it turned to custom AI software specialists dida Machine Learning. Taking an abstract mathematical approach to complex ML models, dida used Google Cloud to build an ML solution that cut Enpal’s process time by 87%, saving the company time and reducing costs while improving its customer experience. Learn more on the blog ⬇️

3

1 Comment

Will HTAP eat OLTP & OLAP? What is HTAP, anyway? Recently, Timothy Chen and I sat down with two of CedarDB co-founders (Christian Winter & Moritz Sichert) to discuss the future of HTAP, whether OLTP/OLAP can converge, and why Cedar DB is the answer. https://lnkd.in/dNcFXEgT

21

3 Comments

Task-specific models and routing architectures are the future. Robert Nishihara, co-founder and CEO of Anyscale, is seeing hybrid strategies emerge, where businesses are evaluating different types of models for different use cases. With the rapid advancement of AI tech, it won't be long before open-source models are "good enough" for the majority of business tasks. The natural progression from here is to develop smaller, task-focused models using proprietary company data. As Robert puts it: "You're going to need small, fast models. And task-specific models have a huge advantage when it comes to cost and speed because you can achieve the same quality with a smaller model if it's specialized." Another interesting development: the growing use of routing architectures. Imagine a classifier as a kind of traffic controller for your AI queries. It assesses the complexity of each query and directs it to the most suitable model: simpler requests are handled by smaller models, while complex ones are passed on to larger, more sophisticated models. “If you do it right,” Robert says, “you can achieve something like the cost of the smaller model with the quality of the more advanced model.” Dive deeper into AI architecture trends in my full discussion with Robert on the B2BaCEO podcast here: https://lnkd.in/ghigvERn

46

3 Comments

sharpening writing skill is one of killer skill!

2

Come and build a smart shopping assistant designed to answer customer questions, guide product discovery, and tailor search results. Combine the power of AlloyDB, Gemini (Large Language Model) and Vertex AI Agent Builder to quickly bootstrap an intelligent conversational assistant.

7

I wrote an article with David Yuan and Diego Z. from Tidemark, where we dove into how AI is shaking up the data game. For years, the big players—those with massive datasets—had an edge because their data acted like “gravity,” pulling customers in and keeping them there. But AI is changing that. Now, with tools to integrate and make sense of data across systems, startups can compete without needing to play by the big guys’ rules. It’s a new era of agility where data orchestration may just be the next big advantage. Exciting times for those rethinking their data strategy! https://lnkd.in/eqGDACYW #AI #DataStrategy #Innovation #Partnership

24

2 Comments

Ensemble of smaller LLMs (PoLL) is less biased, faster, 7x cheaper when evaluating LLMs performance. Works great on QA & Arena-hard evals arxiv.org/abs/2404.18796

9

What is #Raft? Raft is a consensus algorithm designed for managing replicated data in distributed systems. The goal of Raft is to ensure that a group of nodes in a distributed system can work together to maintain consistent and reliable data changes. Raft is an alternative to the more complex #Paxos algorithm. It includes leader election, where one node in a cluster is elected as the leader. The leader manages the replication of data entries to other nodes. If the leader fails or becomes disconnected, a new leader is elected through a distributed consensus process. This approach eliminates the need for a #Zookeeper-like system and provides automated high availability and strong consistency guarantees. Raft vs Paxos? Both of these algorithms function like way for a group of friends to make decisions together, even when some are not present simultaneously. The key differences lie in their complexity and ease of implementation. Raft has gained popularity specifically for its clarity. Raft has been widely adopted and implemented in various distributed systems and platforms. Notable users and implementations of Raft include Redpanda Data (based on #Seastar), #TiKV database, RedisRaft (under development), etcd, RethinkDB, The Apache Software Foundation's brpc (based on Baidu, Inc.'s braft), and others. In Arium, we are using Raft to orchestrate our gaming machines. For C++, #RaftLib is a good implementation, and alternatives include eBay's #NuRaft and Redis's Raft (developed base on Willem-Hendrik Thiart's work) are good as well. libraft: https://lnkd.in/d9AJd_fa redislabs: https://lnkd.in/dz5Cdany NuRaft: https://lnkd.in/dJW8sydM #tech #programing #distributed #system #cpp

62

I remember when this all started. Faris - invited me to catch up at our favorite spot around Blok M. I had no idea he’d just wrapped up a procedure at Pondok Indah Hospital! (He really knows how to add drama to our catch-ups.) We’d been on separate paths and projects since "the winding down project", but there he was, reminding me of our dream to ride this AI wave together again. It was a plan we’d hatched with Sahal Zain & Gurnoor Dhillon when the world of AI was still mostly theoretical and jargon-heavy. Somehow, the conversation continued with Gurnoor Dhillon while he was waiting for his meeting with someone who would eventually become one of CalvinBall’s key early adopters. It felt like everything was coming together, like a series of seemingly random dots connecting to form a bigger picture. Seeing how far we’ve come as a team at CalvinBall Technologies is truly exciting. What started as theoretical discussions has now evolved into practical Gen AI solutions that are solving everyday problems and hold endless possibilities to shape the industry. 👨‍🚀 👩‍🚀 Here’s to pushing boundaries and achieving new milestones together! Gurnoor Dhillon, Sandeep Ramesh, Faris -, Sahal Zain, Eva Zuliya, Ratu Nadiah Khairunnisa 🚀 #GenAI #AIInnovation #TechJourney #ArtificialIntelligence #TechTransformation #CalvinballTech #Calvinballers

55

2 Comments

The most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with "regular" programming mixed in. This leads to more capable, more reliable, and more interpretable AI systems. Most people building with AI already know this. But (unless you work for Google) the main barrier to realizing multi-step AI workloads is an infrastructure one. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. This state of affairs is stifling progress. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. Substrate is the first inference API optimized for multi-step AI workloads. With Substrate, you can write less code, run more inference, and build high performance AI applications with zero infrastructure to manage. No tooling, no infrastructure – just elegant abstractions. Excited to support Ben Guo, Rob Cheung and the amazing Substrate team on their mission to democratize how AI applications are built.

41

4 Comments

🔍 Unlocking the secret sauce to efficient AI models! 🧩 Puzzle by #NVIDIAResearch isn't your average jigsaw puzzle - it's a mind-bending framework transforming over-parameterized LLMs into hardware-optimized masterpieces! 🌟 Let's put the pieces together for smarter, faster, and greener AI magic! 🚀 #AI #Innovation #PuzzleSolvers 🧠✨ #innovation

3