by Albert Major
This slide illustrates the different forms and directions of natural language communication, as discussed in the lecture. It shows one-to-one (blue, direct interaction), one-to-many (red, like public speaking), and recorded or written transmission (yellow icons: books, radio, TV) which can transcend space and time. It also shows language production (green, speaker to book), emphasizing that natural language can take many forms—spoken, written, or signed—but must always involve meaningful information exchange between people.
As mentioned in the lecture, these units are not always consciously known by speakers (e.g. babies use language without knowing what a morpheme is), but linguists rely on them to analyze and model language. Understanding these is essential for building language models and interpreting how language is structured and processed.
This slide shows how spoken language is processed and segmented. The waveform at the top represents the amplitudes of recorded speech (“my name is Giovanni Giorgio but everybody calls me Giorgio”), followed by three levels of transcription: phonetic (IPA), unsegmented characters, and standard writing. The key point from the lecture is that humans easily segment speech into meaningful units (like words), but this is very hard for machines—they need to be taught how to do it. This highlights the challenge of speech recognition and why most NLP systems rely on written, pre-segmented text.
This slide defines formal languages (like programming languages or logic) as systems with strict syntax and semantics. According to the lecture, formal languages use a specified set of strings and rules for combining them, allowing each sequence to be clearly classified as legal or illegal. Unlike natural language, where ambiguity and multiple interpretations are common, formal languages are deterministic—e.g., Python throws a syntax error if the structure is invalid. This contrast helps explain why natural language processing is much harder than interpreting code.
This slide highlights two core properties of **natural languages**: **ambiguity** and **lack of strict syntax**. In the lecture, it was explained that unlike formal languages, natural language allows **multiple interpretations** of the same sentence, often leading to **misunderstandings**. There’s no definitive rule set that determines if a sentence is grammatically “legal,” and **acceptability** is often judged by **native speakers**. This makes **natural language processing** far more complex than processing formal languages.
This slide lists the **three main goals of Natural Language Processing (NLP)** as discussed in the lecture: (1) enabling AI systems to **communicate with humans**, (2) **extracting knowledge** from large amounts of text, and (3) **advancing scientific understanding** of language using AI alongside **linguistics, cognitive psychology, and neuroscience**. These goals explain why NLP is central to both practical applications like chatbots and deeper research into how language works in the brain.
A language model is a probability distribution over sequences of words, characters, or other units.
It predicts how likely a string (sentence, word sequence) is in a specific language.
Example:
This slide explains what **a language model** is: a **probability distribution over strings** in a given language. As discussed in the lecture, it helps determine how likely a particular sequence is—e.g., “the brown fox” is more probable than “the fxow brwn”. Language models are trained on text corpora and can be used to **rank**, **generate**, or **correct** sequences based on their statistical likelihood, forming the basis for many NLP tasks like **text completion** and **language generation**.
This slide lists key applications of language models, from basic tasks like text completion, spelling correction, and language identification, to more complex tasks like translation, question answering, and even essay writing. As explained in the lecture, all of these tasks rely on the model's ability to estimate the probability of a string—choosing the next word or evaluating how well a sequence fits a language. These uses demonstrate how central language models are in both simple tools and advanced conversational AIs.
An N-gram model assigns probabilities to sequences of language units like words or characters based on their context.
An N-gram is a sequence of N units (characters or words).
Unigram = 1 unit at a time
Bigram = 2 units at a time
Trigram = 3 units at a time
4-gram = 4 units, etc.
Unigrams: [“The”, “cat”, “sat”]
Bigrams: [“The cat”, “cat sat”]
Trigrams: [“The cat sat”]
The model calculates:
P(“sat” | “The cat”) = probability that “sat” follows “The cat”
This slide shows how an N-gram character model works by counting character frequencies in a sentence and converting them into probabilities. As explained in the lecture, this allows the model to estimate how likely each character is, forming the basis for predictive tasks like text generation or language identification.
This slide shows a **bigram character model** built from *Poirot Investigates*, where **pairs of characters** (like “th” or “in”) are counted and converted to **probabilities**. As explained in the lecture, this allows the model to predict the next character based on the previous one, forming the foundation for tasks like **text generation** and **language detection**.
This slide explains that **N-gram character models** estimate the **probability of character sequences**, using the **chain rule** and the **Markov assumption**—which simplifies prediction by assuming each character depends only on a fixed number of previous ones (e.g. 1 for bigrams, 2 for trigrams).
This slide shows how N-gram character models use the chain rule and Markov assumption to compute the probability of a character sequence. In a unigram model, each character is independent; in a bigram, each depends on 1 previous character; and in a trigram, on the 2 before it—capturing more context with higher N.
"math" → total chars = 4