Lecture 4

Generative AI

Byeong-Hak Choe
bchoe@geneseo.edu

SUNY Geneseo

September 5, 2025

The Concepts of AI

What is AI?

  • Artificial Intelligence (AI): Techniques that enable machines to perform tasks associated with human intelligence (perception, reasoning, learning, generation, action).
  • In practice today: machine learning algorithms trained on data to make predictions or generate outputs.
  • Sub‑areas: machine learning; deep learning; generative models.

What is Deep Learning?

  • Deep learning is an advanced machine learning methodology.
    • All deep learning is machine learning, but not all ML is deep learning.
    • It combines statistics, mathematics, and neural network architecture.
  • Deep learning is particularly suited for complex tasks that involve unstructured data, such as:
    • Images 🖼️
    • Texts 📝
    • Sounds 🎵
    • e.g., Image recognition

What is a Neural Network?

  • A neural network is a method in AI inspired by the way the human brain processes information.

  • It uses interconnected nodes (neurons) arranged in layers:

    • Input layer → receives data
    • Hidden layers → transform data through computations
    • Output layer → produces the result

What is a Weight in a Neural Network?

  • Each connection between neurons carries a weight:
    • Determines the strength and importance of the input.
    • During training, these weights are adjusted to improve predictions.
      • With multiple layers, networks capture intricate connections and represent complex patterns in data.

What is a Token?

  • A token is the smallest unit of text an LLM processes.
    • Input & Output are measured in tokens, not words.
  • It can be:
    • A single character (a, !)
    • A whole word (dog, house)
    • A part of a word (play + ing)
  • Examples of Tokenization
    • "cat" → 1 token
    • "playing" → 2 tokens (play, ing)
    • "extraordinary" → 2 tokens (extra, ordinary)

What is an LLM?

  • Large Language Model (LLM): A neural network trained on vast text (and often code) to model the probability of the next token.

  • Capabilities emerge: dialogue, summarization, code generation, reasoning heuristics, tool use (e.g., ChatGPT, Claude, Gemini, Copilot, Grok).

  • Limitations: hallucinations (confidently wrong), training bias, context limits (a fixed number of tokens), lack of grounding.

Introduction: Living and Working with AI

The “Three Sleepless Nights”

  • After hands‑on use, many realize LLMs don’t behave like normal software; they feel conversational, improvisational, even social.
  • This triggers excitement and anxiety: What will my job be like? What careers remain? Is the model “thinking”?
  • The author describes staying up late trying “impossible” prompts—and seeing plausible solutions.
  • Key takeaway: Perceived capability jump → a sense that the world has changed.

A Classroom Turning Point

  • In late 2022, a demo for undergrads showed AI as cofounder: brainstorming ideas, drafting business plans, even playful transforms (e.g., poetry).
  • Students rapidly built working demos using unfamiliar libraries—with AI guidance—faster than before.
  • Immediate classroom effects:
    • Fewer raised hands (ask AI later); polished grammar but iffy citations.
    • Early ChatGPT “tells”: formulaic conclusions (e.g., “In conclusion,” now improved).
  • Atmosphere: Excitement + nerves about career paths, speed of change, and where it stops.

Why This Feels Like a Breakthrough

  • Generative AI (esp. LLMs) behaves like a co‑intelligence: it helps us think, write, plan, and code.
  • The shift is not just speed; it’s new forms of interaction (dialogue, iteration, critique).
  • For many tasks, the bottleneck moves from doing → directing (prompting, reviewing, verifying).
  • Raises new literacy needs: prompt craft/engineering, critical reading of outputs, traceability, and evaluation.

Prompt Engineering

The practice of designing clear, structured inputs to guide generative AI systems toward producing accurate, useful, and context-appropriate outputs.

Basic prompt

“Explain climate change.”

Engineered prompt

“Explain climate change in simple terms for a 10-year-old using a short analogy and two examples.”

General Purpose Technology (GPT — the economic term)

  • A General Purpose Technology = a pervasive technology that transforms many sectors (steam power, electricity, internet).
  • Reading’s claim: Generative AI may rival or exceed prior GPTs in breadth and speed of impact.
  • Adoption dynamics:
    • Internet took decades (ARPAnet → web → mobile).
    • LLMs spread to mass use in months (e.g., ChatGPT hitting 100M users rapidly).
  • Implication: Organizations and individuals must learn in real time—no long runway.

Capability Scaling & the Pace of Change

  • Model scale (data, parameters, compute) has correlated with capability jumps across domains.
  • Progress may slow, but even “frozen‑in‑time” AI is already transformative for many workflows.
  • Takeaway: Plan for non‑linear improvements and frequent tool refresh.

Early Productivity Effects

  • Studies summarized in the reading describe 20–80% productivity gains across tasks (coding, marketing, support), with caveats.
  • Contrast noted with historical technologies (e.g., steam’s ~18–22% factory gains; mixed labor productivity evidence for PCs/Internet).
  • Caution: results vary by task, data privacy, oversight, and evaluation rigor.

Beyond Work: Education, Media, Society

  • Education: AI tutors, personalized feedback, changes to writing/assessment.
  • Media & entertainment: personalized content; industry disruption.
  • Information quality: misinformation scale and detection challenges.
  • Identity & creativity: collaboration with “alien” co‑intelligence; authorship questions.

How We’ll Use AI in DANL 101

  • In our DANL 101, the use of generative AI will be allowed for coding and a project.

    • Note that exams are paper-based.

  • Treat AI as a co‑pilot for: clarifying concepts, brainstorming, code debugging, style/grammar critique.

  • Your responsibilities:

    • Verify facts, reasoning, math, and code; cite substantive AI assistance when allowed.
    • Avoid hallucination traps.
    • Respect academic integrity and any assignment‑specific AI rules.

  • Build habits: prompt → check → revise → document.

  • Q: Where do you draw the line between assistance and authorship?

LLM’s Common Pitfalls & How to Avoid Them

  • Hallucinations: Ask for sources; cross‑check; use retrieval tools where allowed.
  • Shallow prompts: Specify role, audience, tone, constraints, and evaluation criteria.
  • Over‑automation: Keep humans in the loop for judgment calls and ethics.
  • Privacy/IP: Avoid pasting sensitive data; follow policy and license terms.

The Concepts of AI (continued)

What is Labeled Data?

  • Definition: Data that comes with the correct answer attached.
  • Good labels = better learning.
  • Sources of labels: human annotators, experts, user clicks/ratings, existing records.
  • Challenge: Creating labeled data can be expensive and sometimes subjective.
  • Example: Companies like Scale AI use ML to make the labeling process faster and more consistent.

Note

  • Takeaway: Labeled data is the “answer key” that makes supervised learning possible.

What is Supervised Learning?

  • Idea: Learn from examples with answers.
  • Like studying with flashcards: front = input, back = correct answer.
  • The computer sees many input–answer pairs and learns to predict the answer for new inputs.

Examples:

  • Email → Spam / Not Spam
  • Photo → Cat / Dog
  • House features → Price ($)

Note

  • Most practical AI in business uses this approach.
  • Takeaway: Supervised learning = “learn by example + answer key.”

What is Attention Mechanism?

  • Analogy: A spotlight that highlights the most relevant words when making a prediction.
  • In the sentence “The bank by the river flooded,” attention helps link bank ↔︎ river.
  • Lets the model focus on what matters now and ignore the rest.
  • Result: better understanding of meaning & context.

Note

  • Takeaway: Attention = smart focus that makes transformers powerful.

What is a Transformer in AI?

  • A neural network design that underlies modern large language models (LLMs).
  • Processes all words in a sequence at the same time (not one by one).
  • Uses attention to learn how words relate to each other.
  • Handles long sequences more efficiently than earlier methods.
  • Encoder–decoder architecture:
    • Encoder: Reads and represents the input.
    • Decoder: Generates the output sequence step by step.

Transformers - Encoder

  • The encoder reads the whole input question:
    “What is the color of the sky?”
  • Each word is turned into embeddings (words used in similar ways get similar numbers).
  • Position information is added so order matters.
  • Using the attention mechanism, the encoder finds relationships:
    • color ↔︎ sky
  • Output: context-aware representations of the sentence that the decoder can use to generate an answer.

Transformers - Decoder

  • The decoder generates the answer step by step:
    “The sky is blue.”
  • Starts with the first token (“The”).
  • At each step, it:
    • Looks at encoder’s understanding of the input sentence
    • Uses the attention mechanism to focus on the most relevant words
      • Chooses the most likely next word (e.g., “sky” instead of “cat”)
  • This process repeats until the sentence “The sky is blue.” is complete.

What is Pre-training in LLM?

  • Phase 1: The model reads a huge amount of text to learn general language patterns.
  • Objective: predict the next token (piece of text).
  • No task-specific labels required—just lots of text.
  • Outcome: a foundation model with broad knowledge of words, facts, and patterns.
  • Think of it as “learning the language of everything.”

Note

  • Takeaway: Pretraining builds general skills before any specialization.

What is Fine-Tuning in LLM?

  • Phase 2: Further improve the pretrained model.
    • Often brings humans into the loop to rank or guide outputs—something earlier training didn’t use.
    • Can be done with smaller, targeted datasets (e.g., medical notes, legal Q&A).
    • Result: the model becomes more helpful, accurate, and better aligned with specific needs (e.g., medical notes, legal Q&A).

Note

  • Takeaway: Fine-tuning specializes a generalist—and with human feedback, it makes the model safer and more reliable.

RLHF (Reinforcement Learning from Human Feedback)

  • A form of fine-tuning.
  • Humans rank or score model answers (better vs. worse).
  • The model then learns to prefer answers humans like.
  • Goal: make outputs more helpful, safe, and aligned with expectations.

Note

  • Takeaway: RLHF = “learn from people’s preferences” to shape model behavior.