Wei Teck Low’s Post

Not a damn thing is being reinvented here: LLMs are not programmers and have a very limited (if any) concept of semantics and meaning. The problem here is that many human idiots use the tool without understanding what it is. It's like translators - they are extremely useful for people who actually know two target languages and use them to speed up the process of translation or quickly look up word if they forgot accurate equivalent. Good luck putting complex text into a translator, expecting to receive an accurate translation.

In our recent experience of working with text-to-speech models has teached us about using TTS for downstream tasks in other languages via finetuning.  Text-to-Speech (TTS) technology that is transforming how we engage with digital content, bridging written and spoken communication. At the forefront of this, the most used LLM is Facebook's MMS model, capable of handling TTS, Speech-to-Text (STT), and Language Identification (LID) tasks across 1,100 languages. We've been hard at work leveraging this powerful model to develop a highly accurate TTS system, especially focusing on Arabic. As we started to see the results of MMS model, there were a few accuracy issues related to numbers. However, we understood that we need to fine-tune this Facebook's MMS model. These are few issues that we faced for fine-tuning. Lack of Official Fine-Tuning Framework: Facebook hasn't provided an official framework, but there is collaborative efforts on GitHub by individuals that yield promising results. Voice Cloning Accuracy: Achieving high accuracy in voice cloning is challenging, especially with multiple speaker models. We've utilized the VITS TTS model, fine-tuned with the LJ Speech dataset format, to improve clarity and consistency. Due to these reasons we also opted for other open source models as well. The models that can be retrained easily than that of Facebook's MMS. How we accomplished this is via VITS.

 🎈 LLM Benchmarks Update! tl;dr: do not depend on benchmark leaderboards to choose your "chatbot" model! (Especially for non-English languages.) First of all, I'm discontinuing the Open #Dutch #LLM Leaderboard (https://lnkd.in/eFnsaFR6). It will stay online for now, but I urge the use of the #ScandEval leaderboard instead (https://lnkd.in/etKMqTnS) by . It contains more tasks, has better reproducibility and statistics (CI) and a flexible back-end library (`scandeval`) to run your own benchmarks with. As part of project "Leesplank" (cc ) we recently added GPT-4-1106-preview scores to add a good "target" to the leaderboard. An important note here is that benchmark leaderboards are not a golden truth. Especially evaluating generative models is hard. You run into issues like prompt engineering (and sensitivity of models to one or other prompt), structured output generation, and - quite simply - "how to automatically evaluate open-ended generation". 💡 Another important but under-discussed facet is the discrepancy between models' capability of understanding vs. generating *in different languages* (so the NLU part of NLG benchmarking). In other words: some of the listed models score really well on, e.g., MCQ benchmarks but are not suitable to use as DUTCH chat bots. Interestingly, some of these models seem to understand questions in Dutch and are able to pick the right answer (because they have good knowledge or reasoning skills), but generating fluent and grammatical Dutch is something else entirely! This is perhaps also true for humans: it's easier to sort-of grasp the meaning of a new language and answer with "Yes" or "No", but answering fluently in the language is much harder! Yet, your language production fluency does not necessarily say anything about your knowledge and reasoning skills. My wish is that some day we can set up a chat arena like https://chat.lmsys.org/ for Dutch. User feedback is (almost?) the only feedback that truly matters.

As promised, just finished documenting building a simple LLM agent-based translation app! Translation of the jargon? I finally wrote up one of the times I iterated building something using generative AI. This time, I allocated different and distinct roles to AI models. Building has helped me better understand how LLMs should be more effectively governed, and what's at stake. Clearer code and clearer writing (legal, especially) also go hand in hand. In previous posts, I covered how the idea was conceived, how this is the basic building block of a more comprehensive localisation tool, and how to orchestrate agents on Langgraph. (It took a bit longer than expected: I got a tad distracted by #adventofcode) LLMs - with a super generous dollop of validation and verification - and technologies like FastAPI, Docker, and Cloud Run let you iterate faster and learn on the fly. I’m more convinced than ever that small, ground-up projects are the way forward for legaltech and civic-tech builders. Next up in the new year: 🤖 a toy/mini (but nevertheless working) eDiscovery tool; 🧪 experimenting with the L4 domain specific language for law (based at the SMU Centre for Computational Law); and 💻 modelling legislation (I'm thinking the Singapore Criminal Procedure Code or the Rules of Court 2021) as code (Rules-as-Code). If you would like to keep exploring, iterating, and most importantly, learning, stay tuned, and stay in touch! Let me know if you have any ideas for what you would like to see built 😊 P.S. If you have substantial business language learning/translation and eDiscovery needs assisted by AI, please check out the veritable experts Bluente and Litigation Edge Singapore. #LegalTech #FastHTML #GoogleCloud #BuildWithMe #AgentBased #FutureOfLaw #2025

https://lnkd.in/dnPww58R << ...In the evolving world of Large Language Models (LLMs), crafting effective prompts has become an essential skill. That's why I've created this collection, showcasing the most impactful prompts of the year across various intriguing domains... >>

Will miss #acl2024 this time. Please check our papers: 1. xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval (https://lnkd.in/ghbrAX7p) 2. On Context Utilization in Summarization with Large Language Models (https://lnkd.in/gEubck9H) 3. Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts (https://lnkd.in/gP_jGNhm) 4. ChartInstruct: Instruction Tuning for Chart Comprehension and Reasoning (https://lnkd.in/g4nMXTR2) 5. Data Augmentation using LLMs: Methods, Learning Paradigms and Challenges (https://lnkd.in/gB_G76tN) 6. XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags (https://lnkd.in/gthvuePq)

Can AI-generated Content Be Original in Different Languages? AI’s intervention in language and content creation is not just a technical challenge but also an ethical one. #aigenerativecontent #ailanguages #languagetranslation https://bit.ly/4bD8lmm

Can AI-generated Content Be Original in Different Languages? AI’s intervention in language and content creation is not just a technical challenge but also an ethical one. #aigenerativecontent #ailanguages #languagetranslation https://bit.ly/4bD8lmm

Can AI-generated Content Be Original in Different Languages? AI’s intervention in language and content creation is not just a technical challenge but also an ethical one. #aigenerativecontent #ailanguages #languagetranslation https://bit.ly/4bD8lmm

🎉 Week 2 Update: Major Milestone Achieved! I'm thrilled to share some wonderful news on our Translate Buddy journey! 🚀 🌟 Feature Implementation: Text-to-Text Translation I have successfully implemented one of the core features of our app: text-text translation! With support for an incredible 96 languages, users will be able to translate between a catalogue of different languages. This milestone marks a significant step forward towards the vision of the app. To power this feature, I have been hard at work on the backend using #Python and creating a robust system that interacts with the scripts and GPU's to run the models efficiently. 📈 What's Next? As I continue building Translate Buddy, the next steps are rolling out the other features including voice to voice translation and text to voice translation. Stay tuned for more updates as we keep pushing the boundaries of what's possible with language translation! #TranslateBuddy #AI #ReactNative #Translation #BuildInPublic