Filtered large language models open minds & inspire
Large-language-models (LLMs) such as GPT-3 (from OpenAI) and LaMDA (from Google):
- Created a renaissance in Natural Language Processing and Understanding (NLP/U) research beginning in late 2017.
- Will dramatically impact the vast preponderance of technology applications going forward.
What are they?
LLMs are neural networks consisting of hundreds of billions of parameters tuned by algorithms that are fed trillions upon trillions of characters of text. Here’s a useful analogy between LLM training and playing the game MadLibs.
It’s an exaggeration to say LLMs are heading towards the point of being trained on all text in the known universe, but that provides a good sense of their ultimate destination.
LLMs are trained to predict words.
Once they’ve been trained, feed them a string, and they’ll begin building their response, adding one word at a time, based on everything they’ve been trained with, word by word but also across words, so the content context matters.
LLMs sound intelligent and like they understand the content they’re reading and generating. Wrong on both counts. Lacking common sense, they aren’t intelligent and they don’t understand what they’re reading or saying.
Some LLM-generated conversations or content suddenly or slowly goes off the rails, makes no sense, appears to hallucinate, shows socially harmful biases, and regurgitates hate speech.
That’s a genuine problem! But it doesn’t mean we ignore the useful applications of LLMs right now.
In May, I wrote a novelty piece about GPT-3 related work illustrating how LLMs can make writers more creative even though GPT-3 hallucinates from time to time: https://www.nasdaq.com/articles/unblock-creativity-and-stop-lousy-decision-making.
Read it to get a fine-grained exposure to how all this works.
The piece makes the point that, in the right hands, hallucinations in the content created by LLMs are not a problem. It also provides an introduction to some of the techniques you might use to explore the application of LLMs to inspire content creators.
Do not take LLM-created content as intelligent or rational.
Do not publish unfiltered LLM-created content.
Do ask the LLM repeatedly the same questions (or pose ideas for it to react to) and use small twists in wording. Collect the responses and filter away! Throw out the garbage. There’s good material you’ve just mined from the vast reaches of the internet.
You don’t need to build your own LLM. You can use, for example, GPT-3. And you don’t even have to write any code. Just go here and follow the instructions.
Iterate and look for non-garbage outcomes that you didn’t expect. Don’t be shocked. This is an opportunity to learn the value (and limitations) of this exciting technology.
Do it! And get back to me about what you discover.
© 2022 – Tom Austin — All Rights Reserved.
This research reflects my personal opinion at the time of publication. I declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.