Essays

I recently read through The Bitter Lesson, an essay by Richard Sutton, widely considered to be the “father” of reinforcement learning, about why the incorporation of human knowledge into AI has been unhelpful or possibly even detrimental to its long-term development. A bitter lesson indeed—nobody wants to believe themselves obsolete, least of all the very people working to bring about Artificial Superintelligence (ASI). I certainly didn’t want to accept it when I first read Sutton’s essay, but the more I looked back over my past experiences with machine learning, the more I realized he might have a point.

Yep, I’ve finally lost it and decided to dip my toe into the radioactive cesspool of American politics. This is a terrible idea. Let’s do it.

Ever since the groundbreaking release of ChatGPT, I’ve been wanting to look into these “large language models” (referred to from here on as LLMs). LLMs, at their core, are autoregressive transformer-based machine learning models scaled up to be able to learn from vast collections of data scraped from the internet. The central concession of an autoregressive model is that it cannot have infinite memory; instead, it takes the prior $n$ tokens as input to generate the $n + 1$th token, and discards the earliest token in memory to replace it with the most recently generated one in a sliding-window fashion, before passing the result back into the model to generate the $n + 2$nd token. While one wouldn’t expect intentionally forgetting the earliest inputs would make for an effective language model, results ever since OpenAI’s Generative Pre-Trained Transformer (GPT) have proven otherwise. Combined with major advancements in other areas of NLP like Google’s SentencePiece tokenization, researchers have been able to achieve record-breaking performance on many natural language tasks using autoregressive language models. The most recent iteration of OpenAI’s GPT, GPT-4, can even perform better than most human specialists in legal and medical exams.

For nearly the internet’s entire history, websites have been at war with automated requests (known as “bot” requests) that attempt to exploit their services. The war has resulted in an arms race of websites trying to develop increasingly effective Turing tests and bots getting increasingly effective at defeating them. From the very first instance of distorted characters being used on AltaVista’s search engine to confound optical character recognition systems and protect the service from bots, to the original CAPTCHA (which stands for Completely Automated Turing test to tell Computers and Humans Apart)’s distorted text with added markings, to today’s Google reCAPTCHA V3, bot detection technology has advanced and been defeated by competing advances in bot technology.

I dedicate any and all copyright interest in this software to the public domain. I make this dedication for the benefit of the public at large and to the detriment of my heirs and successors. I intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law.