Immersive storytelling, customer-centricity, and smarter programming are just the beginning
By Nicolás Rodríguez
OpenAI, the San Francisco-based AI lab, just released the third iteration of its GPT (Generative Pre-trained Transformer) model, or GPT-3 for short. After investing around $4.6 million, the program has shaken up every corner of the Internet, generating a mix of excitement and trepidation. But what is GPT-3, exactly?
Generally speaking, it’s a language model based on automated learning, predicting word(s) based on a certain entry or text, building on what it has learned from previous data. This learning process, called training, is focused on “feeding” the algorithm with as many examples as possible, to allow it to identify patterns in data. It can then estimate the most likely answer based on the rules it has inferred from the information and context it has ingested
GPT-3 has been trained with data from five sources: 60 percent comes from Common Crawl, a global search engine crawler; 22 percent from ebText2, a text collection extracted from different web pages and curated by people; and 18 percent from books and Wikipedia, with more than 6 million articles. Altogether, they gather around 500 billion tokens or word sequences, usually separated by a comma.
With content from diverse sources including novels, blogs, and code lines in different programming languages, GPT-3 offers depth and complexity that is still being explored and developed.
Understanding its origins
When thinking about AI’s transformative powers, you must first define intelligence. Most definitions agree it involves the ability to learn, analyze, and comprehend information in a way that can be understood as knowledge and be applied in adaptive behavior. In this sense, we humans have developed specialized“life skills”, primarily for decision-making problem-solving, creative and critical thinking, communications, and interpersonal skills. Can we say the same about Artificial Intelligence?
For a start, AI represents the ability of a machine to imitate intelligent human behavior. This means that behavior can look intelligent even if it doesn’t include comprehension, let alone knowledge. The key question: Is GPT-3 really intelligent, or just pretending to be intelligent?
Historically, “robots” were based on traditional programming, and told what to do, when, and how. In due course, machines could walk, keep up a conversation, and classify products, with rudimentary behaviors limited by the rules or code created by humans. Increased sophistication and complexity were directly proportional to the massive human effort to codify such behavior.
Everything changed in the ’60s when American psychologist Frank Rosenblatt created the Perceptron, the first artificial neuron. This was a turning point for automated learning as a discipline whose goal is to develop learning among machines, making computers able to identify underlying rules or patterns in the information and act accordingly. Rosenblatt’s investigation was overlooked for almost 20 years, mainly because of unfounded criticism from MIT professors Marvin Minsky (an AI pioneer who won the Turing Prize in 1969) and Seymour Papert (a fellow computer scientist). They saw the Perceptron model as limited and practically useless, which impeded Rosenblatt’s progress.
Fast-forward to the ’80s, when the rise of neuronal networks research and the principles of Perceptron gained relevance while being applied to more complex structures. This gave birth to a still-relevant trend of interconnected multilayer artificial neurons. As a result of these decades of research, most of our modern world is ruled by increasingly complex systems. Without them, we would live in a world with self-driving cars, Facebook recognizing your friends, or voice assistants making recommendations about where to eat or what to buy.
Competing with human intelligence
Considering this technology into the equation, it is possible to think of an analogy between how we humans think and what machines do, thanks to automated or machine learning.
One could say that our intellectual capacity comes from the complexity emerging from our brain, with synapses (the link or communications between neurons) as the biological enabler of perception and thinking. Thanks to developments in the deep learning field, machines now have similar elements to neurons and synapses called artificial neural networks (layers with interconnected artificial neurons) and parameters, respectively.
Interaction between these components allows a model to learn how to recognize human faces, generate text, or identify the sentiment expressed in a tweet. That justifies the interest for models to be increasingly deep, with more neuron layers and parameters.
GPT-3 excels in this regard: it has 175.000 million parameters, a hundred times more parameters than GPT-2, and nine times more than Turing-NLG, the second biggest model in the world. That number of components is key to its performance, only that number of parameters allows it to adapt, generalize and consume the great amount of data collection for its training giving, as a result, a highly flexible, comprehensive, and surprising AI. However, is all this enough to speak about an approach to general artificial intelligence, capable of performing virtually any human task?
GPT-3 can solve tasks related to languages, such as writing a text or answering a question, and also solve math problems (such as adding and subtracting), as well as identifying a sequence or rudimentary code. At first sight, it seems that its path for general intelligence is reduced to achieving deeper models with more parameters. After all, GPT-3 is nothing more than GPT-2 on steroids.
However, if we look at human evolution, we find a completely different answer. Human beings are not animals with the highest number of neurons, and we don’t excel for our synapses. What makes our brains truly unique is our frontal lobe: the center of command where most of what we consider intelligence originates from and that is more developed than most complex animals. All our executable functions (the ability to select the most effective option according to context) and flexibility (ability to reach correct answers according to a situation), or going through updates (ability to acquire and manipulate new useful information for solving innovative problems), are only possible thanks to our frontal lobes, which have advanced and been perfected for the past million years.
Maybe AI is also about quality rather than quantity, meaning that more efficient, complex, and deeper algorithms are still waiting to be developed in the pursuit of human-like intelligence.
The effort to get out of the lab and into the streets
So what does all that history mean for AI? When thinking about its uses and applications, we can make the mistake of undervaluing the effort of integrating models into our chaotic and complex world. In fact, one of the major criticisms of GPT-3 is the sexist and racist content it can generate, which indicates that it is not ready for mass adoption. This is less of criticism on its developers and more of a recognition of the tension between research and application. One has the objective of expanding limits of human knowledge in the widest way, while the other tries to solve specific and practical problems.
If we think of another example like the automotive industry, this distinction becomes clearer. Is the engine the essence of a sports car? Yes, of course. Is it enough to know about engines to manufacturing a sports car? No way; even if we are able to design and manufacture an engine, we need the entire car body to support its features. Can we use the same engine in other vehicles? Absolutely, as long as we respect certain guidelines. Can we manufacture cars without manufacturing engines? Yes, indeed; consider McLaren, a prestigious super sports brand that has used Mercedes-Benz engines. In this sense, GPT-3 is nothing else but a V12 with 3,000 horsepower, a monster with a tremendous brute performance able to go from zero to 100 in three seconds. The engine is provided by OpenAI, and it’s in our hands to adapt it and provide it with value as part of a comprehensive solution.
At R/GA, we understand these subtle but key differences. We are a creative company focused on driving Business, Experience, and Marketing transformation for our clients and partners, converging multiple skill sets and talent: from creatives, designers, programmers, and marketers to strategists, technologists, and data scientists. No single team has a monopoly on the development or practical data application of emerging technology such as GPT-3.
We are specialized in building custom “automobiles”, where models based on AI are a means to an end: providing solutions to complex problems with added value for our clients. Of course, we also manufacture “engines” when the project demands it, and the challenge invites us to go further. It is in our DNA to explore, test, and apply emerging technologies, but most of all we are committed to our delivery, aligning our values and capabilities in an industry increasingly filled by empty promises. That is why we have been hiring the best talent, searching expansively and inclusively to recruit and engage people who understand the world through data and see the possibilities through a human-first lens
From telecom to retail, healthcare to financial services, and beyond, all industries today are being challenged and renewed thanks to increasingly intelligent systems — and at R/GA, we want every day to go a step further. Artificial Intelligence is inexhaustible in the right hands: with creativity, humanity, and innovation at the center, new insights and applications emerge, leading to smarter businesses, agile systems, and truly game-changing products and services.
While AI has a long way to go, every great adventure starts with an open mindset of discovery, exploration, and purpose. So will GPT-3 live up to all the hype? We don’t know what the future holds, but we will surely keep contributing as we’ve been doing for more than 40 years at R/GA, pushing the limits of what’s known to set new benchmarks for innovation and exploration that will improve lives, organizations, and even societies. It’s what makes us leap out of bed each morning and roll up our sleeves. And if that’s an exciting prospect to your career or business too, by all means — please reach out and continue the conversation.
— Nicolás Rodríguez is Lead Data Scientist at R/GA Buenos Aires. He can be reached at nicolas.rodriguez@rga.com.
