Bharath Rao’s Post

How to take an LLM into production Taking an LLM or agent (the latest buzzword in the AI bubble) into production is a critical skill that many people overlook—especially when it comes to launching a service that is both scalable and reliable. That’s why I’ve published a quick introduction covering the basics of how to achieve this. While the article provides a fairly comprehensive overview, this is just the starting point. The topic is complex and can be expanded much further, but my goal was to lay down the fundamentals. In the article I use Microsoft Azure, as they have an amazing platform that we have been able to use in Hippocrates AI after joining Microsoft for Startups. Link to the article: https://bit.ly/40bujJt

Microsoft is set to invest €4 billion in cloud and AI infrastructure in France as part of the 'Choose France' summit. This strategic investment aims to train 1 million people and support 2,500 startups by 2027, enhancing France's tech ecosystem. #Microsoft #AI #CloudComputing #TechInvestment #ChooseFrance

Getting to play with bleeding-edge tech 💻 is honestly the coolest part of being a technical founder. The pace of AI innovation? It's wild, but I wouldn't trade the constant learning for anything. Google's LLM day in Seattle was a blast. We got hands-on with new AI architectures, learned how surprisingly smooth it is to productionize AI with Google Cloud (which, huge deal for smaller teams like ours), and wait for it... 'jailbreaking' LLMs 😲 . (shout out to Vinesh Prasanna Manoharan for that awesome workshop on jailbreaking LLMs!) Turns out, with the right approach, there's way more flexibility than I realized. Biggest product takeaway => It's easy to get caught up in the sheer awesomeness of AI, but the real magic is finding those pinpointed use cases where it'll meaningfully transform your product. Biggest technical takeaway => Integration with Retrieval Services: Google Cloud offers built-in integration with retrieval solutions (e.g., document stores, vector search) that are essential for LLM's RAG's (Retrieval augmented generation) performance and development speed. This simplifies the pipeline from data source to model, meaning faster speed to market A.I solutions for startups, like mine. That's the kind of transformative insights we're constantly after at my startup, Adauris. #googleai #LLM #founderlife #startups #audio #googlecloud #adauris

#AWS's Strategic Shift under Matt Garman #CloudInnovation Matt Garman, the new CEO of AWS, is steering the company through a transformative era, focusing on generative AI, open source, and recalibrating services. His approach emphasizes maintaining a stronghold in innovation while keeping startups at the forefront. Garman stresses the importance of not just resting on AWS's extensive array of services but pushing the envelope on innovation and security. Generative AI is a major focus. AWS's introduction of Bedrock demonstrates a deliberate strategy to offer a flexible platform for custom and proprietary models, instead of rushing chatbots to market like some competitors. Garman aims to make AI more accessible by reducing costs with the new Trainium chips, set to launch soon. AWS is also refining its service offerings by shutting down lesser-used services to focus on more promising ventures. Additionally, Garman is committed to strengthening ties with the open-source community, highlighted by AWS’s contributions to the OpenSearch Foundation. In what ways do you think AWS’s focus on generative AI and open-source collaboration could reshape the cloud industry? #AWS #GenerativeAI #OpenSource #MattGarman #CloudComputing #Innovation #SaaSverse

Google announced a collaboration with MeitY Startup Hub to train 10,000 startups to harness the power of AI. This includes providing up to $350,000 in Google Cloud credits to eligible startups, enabling them to invest in the necessary cloud infrastructure and computational power. Through Project Vaani, in collaboration with the Indian Institute of Science (IISc), Google plans to capture the diversity of India’s spoken languages. The project has already collected 14,000 hours of speech data across 58 languages. To further support developers, Google is introducing IndicGenBench, a benchmark for Indian languages, and open-sourcing the CALM framework, allowing for the creation of more powerful and nuanced language models. https://lnkd.in/egB2_TZu #AIHWEdgeAISummit #AI #EdgeAI #AIHardware #Partnership

🚀 𝗥𝗲𝗱 𝗛𝗮𝘁 𝗘𝘅𝗽𝗮𝗻𝗱𝘀 𝗜𝘁𝘀 𝗔𝗜 𝗩𝗶𝘀𝗶𝗼𝗻 𝘄𝗶𝘁𝗵 𝗡𝗲𝘂𝗿𝗮𝗹 𝗠𝗮𝗴𝗶𝗰 𝗔𝗰𝗾𝘂𝗶𝘀𝗶𝘁𝗶𝗼𝗻! Red Hat, under IBM, is stepping up its AI game by acquiring Neural Magic, a startup that’s revolutionizing AI model optimization. Neural Magic’s technology speeds up AI model performance on standard processors, eliminating the need for specialized chips and making AI more accessible across cloud environments. 🔥 This acquisition strengthens Red Hat’s open-source AI stack, particularly with vLLM model serving for seamless deployment and scalability on Red Hat Enterprise Linux AI and OpenShift AI. Red Hat’s commitment to hybrid cloud AI innovation now includes tools to help businesses better deploy and optimize AI—meeting rising demands for flexibility and efficiency. ☁️✨ Is Red Hat on the path to becoming the “Red Hat of AI”? Let’s discuss! #technology #innovation #redhat #startups #ai #NeuralMagic

🇲🇾 #AWSUntukMalaysia #akandatang #HereatAWS #AWSforMalaysia #cloudcomputing Read the Amazon Web Services (AWS) blog https://lnkd.in/gkraBsZc

In the bustling world of MLOps, a new star emerges: VESSL AI from South Korea has just clinched $12M in Series A funding to innovate how we manage GPU expenses. Imagine slashing those costs by up to 80%! At the heart of this innovation is VESSL AI’s multi-cloud strategy that seamlessly combines on-premise and cloud infrastructures. This approach not only reduces costs but alleviates the notorious GPU shortages plaguing AI development. With 50 major enterprise clients already on board, including Hyundai and TMAP Mobility, and strategic partnerships with Oracle and Google Cloud, VESSL AI is poised to make a significant impact on the MLOps landscape. Founded in 2020 by a dream team with stints at Google and PUBG, VESSL AI is focused on solving the complex challenges of machine learning deployment and operation. Their impressive platform offers automated training, real-time deployment, streamlined workflows, and optimized resource usage. Could this be the future of cost-effective AI model training? Keep an eye on VESSL AI as they continue to reshape the MLOps industry! Source: TechCrunch #AI #MLOps #Startups #Cloud #Funding

👾 The new #AWS Region in #Malaysia will launch this year~ 💵 The MYR 25.5 billion #investment by Amazon Web Services (AWS) in Malaysia is significant as it represents AWS's largest investment in the country to date. 🎯 It aims to support Malaysia's digital transformation by 👉 establishing data centers and 👉 fostering innovation in technologies like #generativeAI, #machineLearning, and the #InternetOfThings (#IoT). 🙋🏻♀️ Follow me to receive more updates on technology, learning and diversity~ #Amazon Amazon #HereAtAWS #ASEAN #Cloud Jeffrey Kratz

This is a slide from my presentation on "Generative AI on AWS" last year at Persians in Tech Berlin United! where I discussed how startups with limited knowledge in model selection can leverage Bedrock to pick the most suitable foundational model for their dataset and specific requirements. The thing is Large language models are trained on vast datasets, which is why they excel at specific tasks such as reasoning or summerization but may not perform as well in others. It's crucial for startups and enterprises to assess their specific needs based on their use case before selecting a foundational model for production. However, if a startup team lacks expertise in choosing the right foundational model, Amazon Bedrock can step in to help. With Amazon Bedrock's model evaluation, customers can bring their dataset to upload it for evaluation and comparison of foundational models. Based on their dataset and the specific evaluation method they choose, Bedrock identifies the most suitable foundational model for them. There are different predefined metrics like accuracy, robustness, and toxicity that can be selected for evaluation based on your use case. Amazon Bedrock then conducts evaluations and generates a report, allowing you to easily understand how the model performed against the selected metrics and choose the right one for your use case. #ModelEvaluation #TechStartups #generativeai #GENAIForStartups #aws #amazonwebservices