Qi Li

New Blogpost: Fine-tuning LLMs for Multi-turn Conversations In this deep dive we cover: 🎛️ How to fine-tune with conversation data. 🎭 Implement loss masking to improve model performance. 🐍 Full code example fine-tuning Llama 3.1 8B on CoQA dataset. Read here: https://lnkd.in/gFxbpia6

20

What’s new with Redis Vector Library (RedisVL)? We’ve introduced new features and functionality that support LLMs at scale. You’ll find a streamlined set of commands to focus on building your app with optimal performance and cost-effectiveness, including: - Streamlined tools to build GenAI apps with our benchmark-leading vector database - LLM memory to store and retrieve chat history - Semantic caching to save on LLM costs and accelerate responses - Semantic routing to send user requests to the best tool, model, or knowledge base Read more about how this helps you build faster here: https://lnkd.in/gbcpZ42U

38

Fully-local function calling w/ llamacpp 🤖 Function calling allows an LLM to connect with external tools, which is central to building agents. But, tool calling with local LLMs has been a challenge. Improved open weights LLMs, such as Llama 3, and fine-tuning efforts are now yielding tool calling LLMs that can run locally. We've updated our integration with llama.cpp to support tool calling and structured outputs. Here we quickly walk through its usage with a fine-tuned 8b parameter llama3 model (from @NousResearch). Video: https://lnkd.in/gva_YmMm Docs: https://lnkd.in/gRDKahXn Model: https://lnkd.in/gehzmfx8

400

13 Comments

LLaMA CPP update: Local function calling is now possible with LLaMA 3 and fine-tuning. • This allows external tool integration, a key step in building agents. • Updated integration with llama.cpp to support tool calling and structured outputs. Check out the demo with a fine-tuned 8B LLaMA 3 model. #AI #Langchain #LLama3

3

Tomorrow at Confluent #Current24: Marc Farnsworth leads the Building Real-time Analytics Apps with Kafka and #Druid session. - Ingest streaming data natively at scale - Enhance and align data from multiple topics - Automatically detect data types and ingest into scalable tables - Execute SQL queries that combine real-time and historical data - Create interactive dashboards to visualize event data - Embed these dashboards into web applications https://bit.ly/4gvOkAH

5

One of my favorite features of Mirascope - use the tool_fn decorator to clean up a poorly documented function so you can use it for function calling in LLMs. And once you have your model provider tool object, pass it in directly to your call :)

1

EXCITING NEWS! We've rolled out some stellar updates at WhoisXML API last quarter. Plus, we’ve boosted our #developer resources with new scripts and tutorials to make your integration even smoother. Check out all the details here: https://lnkd.in/dewg7c28 #productupdate #devops

7

🛠️ RAG On-Prem. Built on Kubernetes, fine-tuned open source models, and battle-tested on 2B+ tokens. 🧿 GroundX has made waves with clients like Air France and Dartmouth, and so far, we’ve processed over 2 billion tokens of data! And now, with Enterprise Grade RAG On Prem— it's designed from the ground up for secure, closed environments. Our API's offer the same high-performance RAG (Retrieval-Augmented Generation) capabilities but fully deployable behind your firewall. We built this for organizations where security isn’t optional but essential—think finance, healthcare, and defense. Are you exploring AI solutions for secure environments? Let’s talk about how GroundX On Prem can meet the most demanding requirements. Learn More: https://buff.ly/2DiomCf #EnterpriseAI #RAG #kubernetes #onprem

5

Looking for a Redshift alternative that just makes sense? I once… ...got asked if Spark can replace a data warehouse like Amazon Redshift. One alternative to Redshift that is out there and that makes sense to use is Databricks. Because Spark itself is just the processing, right? But what they did on Databricks, which I really like, is that you can actually build a warehouse yourself. You can start creating tables, you can ingest data for instance from files, run ETL jobs all within Spark. You can also put the data into tables and then you can either make it accessible through the compute cluster to external clients or you can even generate a separate data warehouse with separate resources for clients to come in and actually create reports of the data. So, Databricks it is ;) Try it out! In my Academy you find a great course to dive right into it. #dataengineer #dataengineering #datascience #bigdata #databricks #spark

5

Startups oftentimes face a huge existential challenge right out of the gate: Choosing a data infrastructure that supports their products as they scale, but that also provides the flexibility and performance they need to build, iterate, and grow. Redis for Startups is our new program to help early-stage companies get access to the Redis solutions and resources they need to build better apps. Every rocket ship needs help getting off the ground. Come see how Redis can be your booster: https://redis.io/startups/ Read more here: https://lnkd.in/g-gaP6Yx

146

5 Comments

when using structured outputs with llms, mirascope makes it easy to retry on validation error and insert the error into the subsequent call this feature ensures end users maintain full control over the retry logic while keeping it simple ti implement docs: https://lnkd.in/gUFY2x-P

4

The Common Table Expression (CTE) materialization framework in Presto was open-sourced by Uber and Meta and helps to minimize query-level redundant computations in queries. Jay Narale gives a technical deep dive into the CTE framework in his latest blog, and also shares some performance results from Uber Engineering by running Presto with Heuristic CTE materialization: 📊 38% reduction in CPU usage 📊 40% less data read 📊 7% improved query latency https://lnkd.in/gEud4RaA

66

1 Comment

Is Postgres all you need? Is a vector just a data type? That's a tough question. On one hand, dedicated vector databases have a lot of catching up to do in terms of almost 30 years of functionality. On the other, there are advantages to reimagining the architecture factoring in all we know in 2024. The good news with txtai is that it's capable of working with multiple setups. It can persist data to Postgres. It can store data in Faiss + SQLite. It can also integrate content with other vector databases. The idea is to have everything needed to get started fast and be flexible to change as the requirements and landscape evolves. https://lnkd.in/dxWDeey

13

We’re big on the LLM ETL stack. We’ve made two big releases in LLMs + structured outputs this week 💫 1. Open-source structured outputs with async + streaming: https://lnkd.in/gpHSTbCb 2. LlamaExtract: https://lnkd.in/gVg5gahy We want your input 📣 - what are the biggest enterprise use cases that you’re using structured extraction and output capabilities for? Whether you’re using a framework like LlamaIndex or Instructor or a tool like Deasie. Is it for a standalone use case or to make an e2e pipeline like RAG better? Some thoughts: 1. Add metadata for RAG 2. Invoice processing 3. Report generation 4. Form filling Let us know your thoughts 👇

233

9 Comments

We just released a major update to our inference stack that's set to revolutionize Enterprise RAG. Do you want to know what has changed?🤔👇 Key highlights: - Up to 3.5x throughput improvement across most workloads - Up to 8x latency reduction for RAG applications - Day-one support for Llama 3.1 and much more... This release represents a significant leap forward in AI inference technology, offering substantial performance improvements that directly translate to cost savings and enhanced capabilities for enterprises. Whether you're looking to optimize existing workflows or embark on new AI initiatives, Takeoff 0.16.0 provides the power and flexibility you need to stay ahead in the rapidly evolving world of AI. Do you want to dive deep into these exciting new features and learn how Takeoff can transform your AI operations? Read our latest blog post in the comment section or contact us to schedule a demo today! #EnterpriseAI #EnterpriseRAG #AIInference #TitanML

16

1 Comment

🏗️ 𝗣𝗼𝘀𝘁𝗴𝗿𝗲𝘀 𝗔𝘂𝗿𝗼𝗿𝗮 𝗗𝗕 𝗺𝗮𝗷𝗼𝗿 𝘃𝗲𝗿𝘀𝗶𝗼𝗻 𝘂𝗽𝗴𝗿𝗮𝗱𝗲 𝘄𝗶𝘁𝗵 𝗺𝗶𝗻𝗶𝗺𝗮𝗹 𝗱𝗼𝘄𝗻𝘁𝗶𝗺𝗲 🏗️ Our payment platform team had the unique challenge to upgrade our Aurora Postgres DB from v10 to v13. This DB was responsible for storing transactions within Lyft and contains ~400 tables (with partitions) and ~30TB of data. Upgrading the database in-place would have resulted in ~30 mins of downtime. Such significant downtime is untenable — it would cause cascading failures across multiple downstream services, requiring a large amount of engineering effort to remediate. Through native replication between the two Postgres major versions and a blue/green deploy, we were able to reduce the downtime to ~7 mins! https://lnkd.in/g7fpXy8z #DataEngineering #DataArchitecture #AWS #DatabaseMigration

2