Why gzip Just Beat a Large Language Model
A paper has shown that a compression algorithm – gzip – outperforms some large language models (LLMs) in some tasks. This has the NLP community in uproar. In this article, I dissect what has just happened, and what it means for language modeling at large.
Messianism in the Age of Artificial General Intelligence
You may have already stumbled upon them: "longtermists" who believe that sentient AI is basically around the corner and will kill us all. In today’s piece, I want to portray this belief as messianist, as a variation of the apocalyptic Christian perspective. Viewed as such, the insistence of these people to warn about impeding doom makes sense, as does the inability of critics to address them.
Swedish Researchers Rise Up Against Current Ethical Review Practices
Every country has its own approach to ethical vetting of research. While many countries have no real prescriptions on that, Sweden decided to do it proper and wrote ethical vetting requirements into its legal code. This has a set of drawbacks, however, and right now, Swedish researchers are rising up against the sometimes detrimental effects that the law can have.
Large Language Model Inference with PyTorch on Apple Silicon
More than two years ago, Apple began its transition away from Intel processors to their own chips: Apple Silicon. The transition has been a sometimes bumpy ride, but after years of waiting, today I feel the ride is coming to an end. In this article, I reflect on the journey behind us.
An Era of ‘Artificial Fake Truth’? On the Effects of Large AI Models
Yes, I'm still talking about large AI models. But today I want to highlight an aspect that has many people worried: what could be the effects of these models going forward? Luckily there is already a debate going on that focuses on these issues.
“Pause Giant AI Experiments”: An Open Letter Full of Straw men
In a recent Open Letter, AI scientists and entrepreneurs are demanding a moratorium on the training of large AI models. In this article I argue that the letter is full of straw man arguments and does not significantly bear on the dangers emanating from AI.
Core.js: Open Source is not Broken
A recent incident surrounding the JavaScript library Core.js has seen many arguing that Open Source is broken. However, looking at the longer history of incidents within Open Source, here I argue that we need to stop saying Open Source was broken, and instead focus on the real problem: A lack of institutional funding.
Selecting Documents for Active Learning (My Supervisor was Right)
Today's article is about a (relatively) new technique called Active Learning, that aims to annotate large corpora with as little effort as possible. Since it utilizes machine learning, decisions regarding metrics are of utmost importance. In this article I dive into deciding on a metric for resampling documents. It turns out that, depending on what situation you face, there are multiple valid options.
How to Use ChatGPT Productively
It’s been several weeks now since OpenAI debuted it’s new toy, ChatGPT, and users have experimented with it to find good use-cases. Today I want to focus on those use-cases, and why LSTMs may be a better choice for some of these tasks.
I get your excitement about ChatGPT, but …
… it's time for some realistic reflection (again). ChatGPT is neither a breakthrough, nor anything extraordinary. It is but a mere highly capable machine with which you can chat — for the lulz, not for real. So don't fall into OpenAI's trap and give them free advertisement. ChatGPT has the same problems as all other AI systems before, and here I list them adapted to the situation around ChatGPT.