Liam Atkinson

Help support the amazing work being done by a very good friend of mine as he continues to innovate within the blossoming field of breathwork science 🔬 If you have benefited from a mindfulness, yoga, or meditation practice, then you already know just what kind of impact breathing can have on one’s wellbeing🧘‍♀️🧘‍♂️🧘 Guy and colleagues in his field are not only helping us to understand the science behind why breathing exercises can have such powerful positive impacts on the mind and body, but are trying to go beyond, striving to leverage said science so we can more easily attain and extend the benefits of breathwork 💪 Have a look, consider contributing, and sponsor the next major advance in this growing field 🌬️

7

I wrote a blogpost a few years ago, accidentally crossed by Hopfield Network as one of notable milestones of Artificial Intelligence. The Nobel Prize to Physics this year, awarded to John Hopfield and Geoffrey Hinton, is all about that network. The formulation is simple, yet the idea borrows to deeper layers of how consciousness store and retrieve information. Let's deep dive in! #AI #NobelPrizePhysics2024 https://lnkd.in/eUKi-Gnc

15

pleias releases world’s first LLMs trained on exclusively open data Training large language models required copyrighted data until it did not. Today we release Pleias 1.0 models, a family of fully open small language models and their RAG-tuned counterparts. Pleias 1.0 models come in three sizes: 350m, 1.2b and 3b parameters. Our models are the first to be fully EU AI Act compliant, thus establishing a clear precedent that truly ethical and socially acceptable AI is possible. What Makes pleias 1.0 Special Our models are: *  trained exclusively on open data, meaning data that are either non-copyrighted or are published under a permissible license * multilingual, offering strong support for multiple European languages *  safe, showing the lowest results on the toxicity benchmark *  frugal (<3b) yet performant for key tasks, such as knowledge retrieval *  able to run efficiently on consumer-grade hardware locally Specialised pretraining This release is not just about open and ethical data. We’re publishing two specialized models for knowledge retrieval with strong performance: Pleias-Nano (based on Pleias-1b) and Pleias-Pico (based on Pleias-350m) are competitive with the best existing models (Llama, Qwen) ten times their size. They're like the espresso shots of the model world: small, strong, and they get the job done without making a fuss. The small size of our Pico and Nano models makes it possible to create fully local RAG applications running on CPU. The models are an integral part of our scientific assistant scholasticAI we are demoing today for Mozilla Builders. Not only open source but open science models The Pleias 1.0 are not just open weights but open science models. The models were trained on Nanotron, the open pretraining library from Hugging Face and we’re releasing our code variant and configurations. Along with Common Corpus we are going to release our training data recipes. The 3B model was trained with support from Etalab - Direction interministérielle du numérique (DINUM) on the GENCI Jean Zay H100 NVIDIA Eviden partition, as part of the Grand Challenge aligned with the European strategy for establishing AI factories through the EuroHPC Joint Undertaking (EuroHPC JU) aimed at supporting startups and providing open source models to the community. We developed our Nano (1.2B) model with the support and expertise from TractoAI, a serverless AI platform for running data and compute-intensive workloads at scale. Use our models The whole family of models is available for use through HuggingFace. We release the Pleias 1.0 models under a permissive Apache 2.0 license, meaning that models are available for use, distribution, and modification for any purpose. In the end, this is about more than just models – it's about showing that the "impossible" might just be a matter of perspective. And maybe, just maybe, the ethical path isn't as boring as everyone thought. https://lnkd.in/eZieYVpQ

252

49 Comments

I'll be giving a talk on "Protein folding and what it means for drug discovery" at PyData London 2024 on Sunday, June 16th. I'll be giving the talk from the perspective of someone who's worked as a pharmacologist and as a software engineer rather than from the perspective of a bioinformatician or a cheminformatician. I'm planning to work backwards from a well-known receptor/ligand relationship and then see how this could have been discovered by using virtual screening, protein folding and the like and hopefully give attendees a sense of how to practically go about this. Here's hoping I add something to this exciting field of in silico drug discovery! Find out more here — https://lnkd.in/eG8ZPzyu Image citation: Interactions of Salbutamol docked in the agonist binding site. H bond and pi-pi networks of salbutamol in (A) wild and (B) T164I β2AR. Black discontinuous lines are H bonds, blue line represent pi-pi interactions. https://lnkd.in/gJUapiWz  #pydata #pydatalondon #insilico #drugdiscovery

21

2 Comments

I gave a talk on Protein folding and what it means for drug discovery at PyData London. I, like a lot of folks, have been interested in the developments of protein folding and thought it would be fun to have a go at folding a protein and then doing something with it like looking at on-target and off-target drug bindings for a well known receptor, the Beta-1 adrenergic receptor, which is important in stroke, coronary artery disease, and congestive heart failure. Activating the receptor increases your heart rate, and contractility and conversely blocking the receptor decreases your heart rate, and contractility. In the talk, I use open-source tools like RDKit, Biopython, PyMol, dockstring, Autodock Vina and AlphaFold to fold a protein (the Beta-1 Adrenoceptor) and then pose the drug propranolol to it and compared it to the endogenous agonist noradrenaline; "endogenous" means "made by the body", and "agonists" are chemicals that activate a particular receptor. I then compared this to another drug, simvastatin, to see how you could compare a drug that is known not to affect this receptor to demonstrate that while other drugs are energetically favourable to interact with this receptor that it doesn't interfere with where the agonist binds. I'm hoping that this talk will be useful to those in drug development that are interested in the applications of protein folding. It certainly gave me some ideas about how we could use these techniques to screen compounds in silico. YouTube: https://lnkd.in/eWZSu9TH GitHub: https://lnkd.in/ehvEJfjy #pydata #pydatalondon #pydatalondon2024 #alphafold #rdkit #pymol #drugdiscovery #drugdevelopment #insilico #pharma #pharmacology

37

3 Comments

On one hand, finetuning on new facts encourages hallucinations. On the other hand, retrieval (RAG) is mostly coarse and imprecise. 😦 😧 😧 There are quite a lot the research community needs to figure out, especially when these LLMs start to run on private and domain-specific data. https://lnkd.in/gTe558K4

10

announcing my next line of work how to elect a committee of expert LLMs to select a jury of LLMs via indirect & iterative prejudiced optimisation in the space of context-agnostic hallucinations, that is provably drug-addict. 😂

90

2 Comments

This paper https://lnkd.in/eGaCaD7s shows that calibrating LLMs to the distribution of their training set always leads to hallucination, at a probability lower bounded by Good-Turing frequency estimation. Good-Turing frequency estimation https://lnkd.in/ewrh3RZC was developed to estimate the frequency of discovering a new species. Assuming the discovery of new species follows i.i.d., then its frequency can be reasonably estimated by the historical discovery rate, which is N_1 / N​, where N_1​ is the number of species observed only once (a.k.a., new discoveries), and N is the total number of observations. Now, calibrating LLMs forces them to emit only in-distribution facts (known species) and zeros out the probability of out-of-distribution facts (unknown species). Therefore, calibrated LLMs will hallucinate when the question is OOD, at a rate of N_1 / N​​. And there is no cure, as non-calibrated LLMs can be even worse. #artificialintelligence #machinelearning #deeplearning

208

18 Comments

Working with some Large Language Models (LLMs) at Trionic Technologies LLP, Vatsal Patel asks a very interesting question: "Can we really exercise some influence or exploit LLMs to get focused answer from multiple choices?" On digging into it, we found a very interesting research paper that highlights the same! This paper, "Manipulating large language models to increase product visibility," shows how "strategic text sequences" could increase the visibility of the product in LLM-driven searching aids. Think about dropping a purposely designed message into a product description and seeing that one rise in the recommendation listing to the very top. Not only is it possible, but it's also now proven! Their research in optimizing these sequences employs techniques such as the Greedy Coordinate Gradient (GCG) algorithm to ensure that the very products, even if in real spaces, which were previously shrouded on digital catalogs, shine bright on the virtual shelf. This is almost revolutionary in SEO for AI-run search services in the sense of really changing the perception and priority of products by algorithms. The potential of businesses to harness this ethically is not less impactful and innovative in generating competitive edges. Let's see how strategic text sequences can help in innovations of digital marketing and e-commerce. Link: https://lnkd.in/dPE8zFtm #AI #MachineLearning #LLMs #DigitalMarketing #ecommerce

15

As efficient prioritization of molecules for synthesis requires a lot of expertise and things to consider, it is great to see emerging computational approaches aiming to facilitate this process. Jenna Fromer and Connor W. Coley recently published SPARROW - an algorithmic framework that prioritizes compounds for synthesis considering information about the cost of starting materials, synthetic routes, common intermediates, information gain, and other properties that can influence the final decision. The molecules are evaluated as a set, capturing the non-additive synthetic costs and other details associated with batch synthesis.     Big thanks to the authors for using the Chemspace API to get real-time compound pricing information.   Blog post: https://lnkd.in/ehXTxUCN Original paper: https://lnkd.in/exUyEXB4  GitHub: https://lnkd.in/e-pbgRr6  

62

1 Comment

It's tonight! Please join the PyData Cambridge community to welcome Mayank P. (Astra Zeneca) with his talk "Image Registration: How to align images" and Ehecatl Antonio del Rio Chanona (Imperial College London) with "Data-driven optimization for expensive problems: a chemical reactor case study". This event is free and in-person only. Please signup on Meetup (link below) and reach out if you have any questions or wish to contribute in any form. We are always looking for speakers, attendees, and funding (company sponsorships, personal donations, etc.). Please also follow us and share this event! - MeetUp (event signup): https://lnkd.in/eT3FRzt6 - LinkedIn: PyData Cambridge - Twitter: https://lnkd.in/eCr9BNT5 - Location (Map): https://lnkd.in/evutQgnY #llm #ai #datascience #dataengineering #cambridgeuk #sponsorship

15

I was (rhetorically) asked why should anyone care about anything else than single file implementations (SFI) in RL research, so here is my take: First let me acknowledge that SFI such as CleanRL are great and this lib in particular has contributed to RL in an unprecedented manner. I could only wish I had the skills to make something as impactful as that! Having it all in one file that you can put side to side with the paper is super useful: helps you understand what is going on in the algorithm, and lets you identify very precisely where you can tweak the precise line that you think needs improvement. It’s not only clear but also efficient: since each file on out considers a specific scenario (limited environments and hyper parameters) you can safely over optimise for that one. So if your research is about changing that a bunch of lines in the DQN algorithm and show that you get an improved SOTA result in Atari results, go for it! Same if you want to show/test that your task can be solved by a regular SAC implementation and it looks more or less like that one in the example file. Now one scenario where do SFI fall short is whenever you need anything that is not strictly accounted for by the implementation: eg you need to use a stateless environment, you need some specific replay buffer etc. Another issue is that the “proper” way of recycling these scripts for research is to copy paste part of the script. That means that you are isolating a bunch of code that is undocumented and take responsibility for what you’re doing with it. Comparatively, a modular framework will provide you with documented and tested chunks of code (classes or functions) which responsibility and coverage is clearly defined. If the collaborator has done his job properly, you should also have an example of how to run this primitive (In fact, SFI are mostly not uni-tested, the macro-tests they achieve mostly comes from proven reproducible learning curves). Regarding efficiency, sure a SFI is very fast at doing what it does, but you won’t be able to scale it a lot: if you need hugely distributed solutions, or highly engineered ways of offloading RAM/GPU usage on disk these repos will provide a limited help and guidance. Finally, I think this isn’t a black or white situation: one can perfectly recycle a SFI and replace the buffer or data collection part with a modular implementation (such as TorchRL ‘s) #reinforcementlearning #research

29

1 Comment

Finally - achieving and surpassing the benchmarks that have long been held to mark a tipping point in quantum computing. Today's announcement, taken together with other disclosure in the past few weeks, extends our leadership in the quantum computing sector and brings reliable quantum computing out from the lab into the real world. This achievement should be one that all of us in the sector can enjoy and appreciate. I look forward to building on these developments as we get to the point where it becomes impossible for classical computers to simulate quantum processors. A few weeks ago we demonstrated our ability to scale our quantum computers. Then we disclosed stunning results with Microsoft on logical qubit development. And today we provide details of our world leading quantum volume numbers (exponentially better than literally anyone else) and for the first time, 2 qubit gate fidelities above '3 nines' across all our zones in our fully connected quantum processor. I could not be prouder of my colleagues. There is so much more to come as we continue to set the high bar in the industry and lead by example in exemplifying excellence. https://lnkd.in/esMQfmNt

227

17 Comments

A couple of weeks back I released an R package, ggsmc, on CRAN. It can be used to visualise the output of IS, SMC and ensemble-based algorithms. https://lnkd.in/egP2Jy3F 1/4 Whilst I am a big admirer of dplyr/ggplot2, I don't use them regularly enough to remember what I need to do to convert Monte Carlo output (which I usually store in a matrix) to the tidy format needed for plotting. 2/4 I wrote this package to help me with this, and to provide an easy interface to construct the types of plots I commonly use. There is a vignette for the package. https://lnkd.in/eHvw5KgA 3/4 Over the next few days I'll post some details about some of the functions. For now, I'll highlight the matrix2tidy function, which can be used for converting from matrix format to the tidy format required for plotting. https://lnkd.in/eveHHqm3 4/4

31

I've reached a point where I no longer document my code myself—I let AI do it for me. It not only handles the documentation but also fixes naming conventions, assigns meaningful variable names, and even refactors my code at times. It's no exaggeration to say that my role has shifted to critical thinking, mathematical modeling, and rapidly writing code as ideas come to me, while AI acts as my copilot, handling the cleanup. Have I become lazy with coding? Perhaps. But has this freed up my time to sharpen my critical thinking and mathematical skills? Absolutely—by at least threefold. My productivity for writing complex code has increased significantly. I surmise that in few years I might even stop coding and just managing the critical process of directing what needs to be done on a mathematical canvas. And probably in a 5-10 years, I might not have to do that either! 🤷‍♂️ #coding #AI

5

2 Comments

I’ve probably generated 6M tokens of fake data and up over the past 2 weeks. My conclusion - it’s more of an art than a science at this point. You’ll expect to throw away 60% and up of generations, especially at the early phase. There’s no easy way around it though, because how else would you get a vibe for what makes sense and what doesn’t. I wrote about my key takeaways here : https://lnkd.in/gwwXXUFq

22

2 Comments

Some positive Sunday news for those suffering from ME/CFS 🌞 A team from the The University of Edinburgh, involving both scientists & mathematicians, has published preliminary results from a huge study of blood samples from ME/CFS patients (1,400+ patients, 130,000+ health controls) Their findings show a clear 'biological signal' in the blood of patients 🩸 What is especially interesting is that it wasn't just one molecule that enabled differentiation (although that would be great), but 116! Something enabled by the large sample size taken from the UK Biobank. As stated by Prof Chris Ponting on X: "Such a large number of replicated and diverse blood biomarkers that differentiate ME cases from controls should now dispel any lingering perception that ME is psychosomatic" We need & deserve serious investment in this space! #MECFS #InvisibleIllness #LifeSciences #BioTech

68

17 Comments

I’m looking for a Senior Algorithm Researcher to join our team, and if you’re into Deep Learning, Computer Vision, and Python, I’d love to hear from you! We’re working on something pretty cool at LiteBC. Our focus is on developing non-invasive blood count technology with our flagship device, the HemoScope. It’s a handheld device that can monitor blood parameters like red and white blood cells in real-time—without ever needing a needle or a big fancy laboratory. All you do is place it on the patient's lip, and 30 seconds later sophisticated optical and imaging techniques supply you with precise blood count results. We’re breaking new ground in blood analysis, aiming to transform complete blood counts into a real-time, non-invasive process. The potential here is massive—imagine a doctor being able to provide your blood results while you’re still sitting in the office. With immediate insights into critical blood parameters like red and white blood cell counts and hemoglobin levels, this technology could radically change diagnostics, making healthcare more efficient and personalized. If this excites you, we’d love your help in taking it even further! What You’ll Be Doing: - You’ll help us build and refine deep learning models that process complex imaging data from our device, giving us insights into blood composition. - You’ll stay on top of the latest computer vision research and apply it to make our models more accurate and scalable. - You’ll work on designing robust data pipelines to handle large-scale clinical data and ensure our tech can scale with ease. - You’ll collaborate with a smart, multidisciplinary team of physicists, software engineers, and medical experts to turn raw data into actionable medical insights. What I'm Looking For: - MSc or PhD in a relevant field, and at least 5 years of hands-on experience in Deep Learning, Computer Vision, and Numerical Analysis. - You should feel extremely comfortable around Python and have a solid grasp of software engineering practices. - Experience with deep learning frameworks like PyTorch and a knack for building scalable, efficient data workflows. Why You’ll Love Working with Us: We’re working on tech that has the potential to change healthcare worldwide and make blood testing more accessible and vastly more available, especially for underserved populations. If you like working with smart people, solving real-world problems, and making a real impact, this is the role for you! 👉 If you’re intrigued, let’s chat! https://lnkd.in/dMzaqKcP

25

1 Comment