2.4 Generative AI

2.4 Generative AI

Generative AI (GenAI) refers to AI-based systems specialized in creating new content: text, images, video, music, or complex synthetic data. Many of the same systems also classify or predict—a model that writes a caption can also score how likely the caption is, and an image generator's internals often share DNA with image classifiers. Do not trap yourself into thinking generative means artistic only. A synthetic transaction feed for a test environment is generative. So is a draft incident report.

These systems learn from large collections of examples and then produce outputs that resemble that training world without being simple photocopies of one row (when they work as advertised). The resemblance is the feature and the hazard. It is why a marketing team can scale campaign variants, and why a fraudster can scale fake invoices.

What generation looks like in a product

A conventional report writer fills a template: Name, Amount, Date. The oracle is equality with the database. A generative writer is asked to explain the same invoice to a customer in a calm tone. There are hundreds of acceptable paragraphs and a long tail of unacceptable ones (wrong amount, invented late fee, insulting phrasing, leaked other customers). Testing shifts from byte match to constrained creativity: is the amount grounded, is the tone in policy, did the model fabricate a policy that does not exist?

The same pattern appears in images (acceptable product shots versus a bottle with eight fingers), code (a function that passes tests versus a function that deletes the wrong directory), and music (a jingle in brand mood versus a melody that copies a living artist's hook).

Core technologies: GANs, diffusion, transformers

Three technical families dominate the conversation testers hear in design reviews.

Generative adversarial networks (GANs)

A GAN uses two neural networks in competition. The generator tries to produce synthetic samples (faces, parts, time series). The discriminator tries to tell real training samples from fakes. Training is a contest: the generator improves until the discriminator can no longer tell them apart easily. The result can be highly realistic synthetic data.

Tester angles: GAN training is unstable; mode collapse can make the generator emit near-duplicates; synthetic faces and documents are dual-use. If the product uses GAN output as fake identity imagery, that is not only a model-quality issue. It is a misuse issue.

Diffusion models

Diffusion models generate content by gradually adding noise to data and then learning to remove that noise. At inference, they often start from noise and denoise toward an image (or other modality) that matches a prompt. The stepwise reverse process is why they can produce high-quality stills and, increasingly, video.

Tester angles: prompts are part of the input surface; small prompt changes can swing safety and brand; compute cost per image is a performance and sustainability concern; evaluation needs human or learned judges because there is no single correct picture.

Transformers and self-attention

Transformer models, which underpin large language models, use self-attention to weigh relationships across a sequence. That mechanism is why they can keep a pronoun consistent many tokens later, or copy a number from the top of a prompt into a later sentence—when they do not hallucinate instead. The same architecture is increasingly multimodal: text, images, and audio in a shared system that can both understand and generate across those channels.

Tester angles: context windows, retrieval, temperature, and system prompts are test configuration, not trivia. A grounded mode with retrieval and a low-temperature support bot is a different product from an open-ended high-temperature storyteller, even if the base weights are related.

Foundation models and fine-tuning

Most practical GenAI tools are not trained from a blank slate for each customer. They start from foundation models—large models trained on broad data—and are then fine-tuned (and often wrapped with retrieval, tools, or policy filters) for an application: a hospital's discharge-note helper, a bank's complaint drafter, a game studio's NPC dialogue.

Fine-tuning can make the model more useful and more dangerous. It can teach a house style. It can also teach confidential examples that should never reappear in someone else's session. Testers should treat training data licenses, retention, and isolation as quality characteristics, not only as legal paperwork. They should also test the base plus adapter plus filter as one system: a strong filter that is easy to jailbreak is a failed control.

Multimodal foundation models raise extra questions: does an image uploaded by a user get logged, trained on, or shown to another tenant? Can a text instruction override an image-safety classifier? Those are generative-system tests, not afterthoughts.

Benefits people actually buy

The same machinery has legitimate, high-value uses testers should not sneer at:

• Entertainment — interactive characters, music sketches, visual worlds
• Marketing — variants of copy and creatives, localized campaigns, brainstorming
• Education — explanations at multiple reading levels, quiz stems, tutoring dialogue (with grounding so lessons are not invented facts)
• Research — hypothesis drafts, literature maps, synthetic data to stress a conventional pipeline

When you test these uses, usefulness is real: time saved, coverage of variants, accessibility of explanations. The oracle may be a rubric (grounded, on-brand, reading level) plus sampling, not a golden file.

Societal issues that are in-scope for quality

GenAI is not only a UX feature. It concentrates several public harms that product testers will be asked about in incident reviews and in regulation.

Deepfakes, misinformation, and fraud. Models can fabricate faces, voices, news-like articles, and invoices that look ordinary. That undermines trust in digital media and enables scams. If your system can emit such content, abuse cases belong in the test plan: can an unauthorized user request a non-consensual intimate image, a cloned executive voice, or a fake bank letter? Safety filters, rate limits, watermarking (where used), and audit logs are test objects.

Employment disruption. Automating writing, design, coding, and even draft legal or medical documentation changes white-collar work. Testers are not labor economists, but they are responsible for products that silently replace a review step. If a hospital ships a note generator, the quality story includes human review paths, not only token speed. Workforce impact is also why organizations invest in reskilling; a system that hides its mistakes accelerates deskilling.

Energy and carbon. Training and running large models consume substantial compute. That means energy use and carbon impact at training time and at inference time (every user prompt has a cost). Efficiency, caching, smaller distilled models, and not calling a giant model for a tiny classification job are design choices with quality and sustainability consequences. Performance testing can include joules or dollars per thousand requests, not only latency.

Emerging regulation. Frameworks such as the EU AI Act are coming into force in stages to govern development and use of AI by risk. Testers in or selling into that environment should expect demands for documentation, risk management, logging, and human oversight on higher-risk uses. You do not need to recite every article. You do need to know that generative systems are not regulation-free toys, and that claims, transparency to users (for example, that content is AI-generated), and high-risk domain use will attract obligations. Local counsel owns legal interpretation; test evidence is what you can produce.

Why testers find oracles hard—and what to do anyway

Generative outputs are creative and probabilistic. Sample twice, get two essays. Temperature greater than zero is a deliberate lottery. Even temperature zero is not a mathematical identity across hardware and library versions.

That does not make testing optional. It changes the oracle catalog:

• Invariants — never invent an account balance; never output another tenant's data; never include banned categories
• Grounding checks — numbers and citations must appear in allowed sources
• Rubrics and pairwise preference — humans or judge models score samples, with disagreement tracked
• Metamorphic ideas — paraphrasing a benign prompt should not flip a safety decision; swapping a name should not change medical advice quality
• Regression against a pinned model — not because the old model is truth, but because silent drift is a defect
• Misuse and safety suites — jailbreaks, fraud templates, deepfake requests, prompt injection through uploaded files

flowchart LR
  Prompt[Prompt plus context] --> Model[Foundation model plus fine-tune]
  Model --> Sample[Probabilistic sample]
  Sample --> Filters[Policy filters and grounding]
  Filters --> User[User-visible artifact]
  Filters --> Risk[Misuse safety fraud energy]

If you only test the happy-path poem about cats, you have tested a demo. If you test grounding, filters, privacy, dual-use, cost, and the claim the marketing site makes, you have started testing generative AI as a system.

Keep the vocabulary straight for the exam: GenAI creates; many GenAI systems also classify and predict; GANs compete; diffusion noises and denoises; transformers attend; foundation models are fine-tuned; societal harms and the EU AI Act sit beside the benefits in entertainment, marketing, education, and research. Your professional stance is not wonder and not panic. It is oracles that match a probabilistic creator, and quality risks that include misuse and safety.