Human-Computer Interaction for CSAI

by Albert Major

1. Definition of Cognition

• HCI Perspective: Cognition is the mental action or process of acquiring knowledge through understanding, thought, and senses.
• Robotics/Systems Perspective: A system (human or machine) that autonomously achieves regularity or reduces entropy in its environment. It interprets the world to create order.
• The "Two Systems" of Thinking (Modes of Cognition):
  • Experiential Cognition: Fast, effortless, and automatic. It requires high familiarity.
    • Example: Driving a car after years of practice; you react without deep thought.
    • Context: Corresponds to Kahneman's "System 1".
  • Reflective Cognition: Slow, requires effort, planning, and deliberation. Used for new or complex tasks.
    • Example: Learning a new subject (like Autonomous Systems) or planning a schedule.
    • Context: Corresponds to Kahneman's "System 2".

2. Elements of Cognition in HCI

The lecture uses a gamer playing Minecraft to illustrate how cognitive elements work together.

• Interdependence: Attention, perception, memory, and problem-solving function simultaneously. If one fails (e.g., memory loss), the user cannot complete tasks (like finding a path) even if perception works perfectly.
• Sensory vs. Perception:
  • Sensing: The reception of raw data (e.g., a robot camera receiving pixel data, or a human retina detecting light).
  • Perception: The processing of that raw data into meaningful information (e.g., the robot recognizing "this is a ball" or "this is a teammate").
  • Design Implication: Users must not just "see" interface elements; they must understand what those elements represent meaningfully.

3. Attention

• Definition: Selectively focusing on specific aspects of the environment while ignoring others.
• Selective Nature: Attention is subjective. In a tennis match, one person tracks the ball, another tracks a specific player.
• Change Blindness / Inattentional Blindness:
  • The "Invisible Gorilla" Experiment: When people focus intensely on a specific task (counting basketball passes), they often fail to notice obvious stimuli (a person in a gorilla suit walking through the scene).
• Design Application:
  • Guiding Attention: Designers use visual hierarchy to direct the user's eye to the "Call to Action" (e.g., Google's search bar is the clear focal point).
  • Information Retrieval: Good design allows users to find information (like the cheapest double room on Booking.com) via scanning rather than deep search. Poor design creates a "soup" of numbers that fatigues the user.

4. Perception and Gestalt Principles

Humans do not see isolated dots or lines; the brain organizes visual input into coherent wholes. This is known as Gestalt Theory.

• Figure-Ground: Distinguishing the foreground object from the background. High contrast (e.g., black text on white) aids this.
• Similarity: Objects that look similar are perceived as related or sharing a function.
  • Example: Cards in a news feed with the same layout are understood to be the same type of content.
• Proximity: Objects placed close together are perceived as a group.
  • Example: An icon placed immediately next to a text label is perceived as belonging to that label.
• Continuity: The eye follows lines, curves, or sequences.
  • bad Example: Vertical text layouts (like "Back to School") that break the natural left-to-right reading flow disrupt continuity.
• Common Fate: Elements that move in the same direction are perceived as a group.
  • Example: Accordion menus where clicking different headers results in the same expansion animation.
• Closure: The brain fills in gaps to perceive a complete shape.
  • Example: The IBM logo consists of separated horizontal lines, but we perceive it as continuous letters.

5. Memory

• Sensory Memory: Instant, very short retention of sensory stimulation (e.g., seeing a trail of light when a sparkler moves fast).
• Short-Term (Working) Memory:
  • Capacity Limit: Humans can typically hold 7 +/- 2 items in working memory at once.
  • Design Implication: Do not overload menus or lists. Break information into chunks (segmentation) to respect this limit.
• Long-Term Memory:
  • Recognition rather than Recall (Nielsen's Heuristic): It is much harder for users to recall information from scratch than to recognize it when presented.
  • Example: When logging into Steam, seeing a masked email (l***@gmail.com) helps the user recognize which account to use, rather than forcing them to recall the email from memory.

6. Learning

• Definition: Acquiring new understanding, knowledge, or skills (distinct from the Machine Learning definition of performance improvement over time).
• Incidental Learning: Learning without a specific plan or intent, often through exposure.
  • Example: Gradually remembering classmates' names just by attending lectures.
• Intentional Learning: Goal-oriented, deliberative learning requiring effort and planning.
  • Example: A student planning their degree path and studying specific materials to pass an exam.

7. Physiological Context

• Brain Specificity: Specific brain regions handle specific processing (e.g., faces). Stimulating these areas can cause hallucinations (seeing faces on inanimate objects).
• Universal Cognition: Cognitive illusions affect animals too. Cats will sit in "2D boxes" (optical illusions of squares on the floor), suggesting visual processing rules like Gestalt are deeply rooted in biological vision systems.