Generative AI

Before the winter of 2023, the phrase “Generative AI” or “Large Language Model” might have elicited puzzled looks from a vast majority globally, including those within the tech sector. Attempting a Google search would likely have yielded minimal results, primarily academic research and obscure company blog posts buried in the web’s far reaches. Fast forward to the present in March 2024, and Generative AI has surged to the forefront of tech discussions, dominating headlines in specialised tech outlets and across mainstream media platforms. This surge has sparked a dual reaction of excitement and concern, with discussions increasingly centred on the implications of AI’s safe integration into our future societies. What catalysed such a swift and profound shift in the public and professional consciousness towards generative models, driving their ascent to global prominence?

This transformation can be attributed to several pivotal developments and breakthroughs in the field that have showcased the vast potential and versatility of Generative AI. Innovations have raced from experimental stages to practical applications, demonstrating AI’s ability to generate new content, solve complex problems, and even mimic human creativity in art, music and literature. As these technologies have become more accessible and their applications more visible in everyday life, public interest has naturally spiked. The dialogue around Generative AI has expanded to include its potential benefits and the ethical, privacy and security concerns accompanying its rapid adoption. This comprehensive discourse has propelled generative models from the fringes of tech conversation to a central topic of global interest, highlighting both the promise and the challenges of integrating advanced AI technologies into the fabric of society.

In simple terms, Generative AI refers to a subset of artificial intelligence that creates new content instead of merely analysing existing information. For decades, AI focused on pattern recognition, identifying objects in images, transcribing speech, and labelling emails as spam. But the latest breakthroughs enable AI systems to generate original text, images, audio and video in an increasingly humanistic output.

This generative capacity stems from fundamental advances in Natural Language Understanding driven by a technology architecture called Transformers. Developed at Google several years ago, Transformers proved uniquely skilled at modelling the nuances of human language. Unlike previous AI models, Transformers capture long-range dependencies between words and understand the significance of word placement within a sequence. This ability to deeply comprehend language syntax and semantics enables Transformers to predict probable next words and generate coherent text as if conversing naturally with a human.

Initially, the potential of Transformers went largely untapped as it produced less than impressive results when provided with small amounts of training data. When researchers at organisations like OpenAI began experimenting with Transformers and feeding in A LOT (similar to the scale of the whole internet) of information, these Transformers began to show their generative powers.

In 2020, GPT-3 (GPT stands for Generative Pre-Trained Transformer) stunned the AI community by producing remarkably human-like text for extended passages.

However, the real magic of generative AI lies in its capacity for iterative refinement based on further prompts or commands. Suppose you wish to delve deeper into specific aspects of King Henry VIII’s life, such as the intricate details of his marriages or the dramatic circumstances surrounding his death. In that case, you can guide the model with follow-up prompts like “focus on his wives” or “make the part about his death more interesting.” Each subsequent prompt acts as a directive for the AI, steering it to zoom in on these facets, enrich the narrative with more detail, or perhaps adopt a more captivating tone in recounting historical events.

This interactive process allows for a dynamic and customised content creation experience, enabling users to shape the narrative according to their preferences or the requirements of their audience. The ability to adjust and refine the AI’s output through continued prompting enhances the content’s relevance and engagement. It demonstrates the sophisticated understanding and flexibility of the AI models in catering to specific content needs.

The introduction of GPT-3.5 marked a significant leap in quality, paving the way for the creation of ChatGPT. This iteration incorporated a conversational interface into GPT-3.5, quickly capturing the public’s attention. Within just the first five days of its launch, ChatGPT attracted over a million users, and this number skyrocketed to 100 million within two months. The platform offered thoughtfully crafted responses across a broad spectrum of questions, providing a conversational experience that many found indistinguishable from interacting with a genuinely intelligent being rather than a mere computational algorithm.

The rapid ascent of ChatGPT to 100 million users in such a short span was notable not just for the sheer volume but also for the diversity of its user base. It wasn’t limited to a niche audience of tech enthusiasts or software developers; it encompassed individuals from every conceivable sector and corner of the globe. From chefs experimenting with novel recipes to office workers seeking fresh fitness routines, from students condensing extensive academic papers to translating intricate topics into accessible language - ChatGPT demonstrated its potential to revolutionise a wide array of tasks for virtually anyone. This broad applicability, coupled with the ease of using natural language to guide its operations, underscored the transformative impact of this technology across various facets of daily life and professional fields.

Tokens determine how much content AI models can produce, while parameters shape their capacity to store and utilise knowledge. These components allow the models to process vast amounts of data, identify intricate patterns, and generate coherent, detailed outputs spanning multiple sentences. Equipping models with trillions of parameters demands significant computational power and energy, highlighting a key challenge in AI development.

The creation of GPT-3, for example, was reported to have cost approximately $4 million. In contrast, the development of its more advanced successor, GPT-4, is estimated to have required around $100 million of compute to train the model. These substantial costs pose a barrier to making such advanced AI models widely accessible, as they necessitate specialised hardware for operation. While simpler models may function on standard consumer hardware, accessing more complex systems like GPT-4 often involves relying on services provided by major companies, including OpenAI, Anthropic, Google and Amazon. The soaring demand for Generative AI and its limited availability have led to restricted access to many such services.

Integrating these advanced models into practical applications also necessitates rigorous trust and safety measures to prevent the generation of harmful content. Without proper safeguards, raw Large Language Models (LLM) may generate outputs that are racist, nonsensical, or factually inaccurate. Consequently, companies must carefully curate the content produced by these models, ensuring it aligns with ethical standards and business goals, even if it means limiting the models’ capabilities.

Despite these hurdles, the vast technological promise of these LLMs and the future developments in the Generative AI space cannot be overstated. The capabilities enabled by this technology are enormous and varied, touching on numerous aspects of daily life and work, especially as they move into other modalities.

Below are some personal examples of how I utilise generative AI daily, showcasing the practical benefits and innovative potential of these technologies:

1. Converse with long-form reports and documents to extract answers to my queries
2. Generate code by describing what I want the logic to be
3. Create Images and diagrams for my articles
4. Convert transcriptions of my videos into summarised articles
5. Replace Google for many Q&A type searches
6. Creating the first draft of a piece of content
7. Expand a thought and then challenge my perspective
8. Generate b-roll videos for visual content
9. Clone my voice for audio versions of my content
10. Generate dad-style songs I sing to my kids purely for embarrassment