Computer Networks + System Design + SDLC
The Final Phase

The TCP/IP model (Internet model) is the practical 4-layer model governing the internet. OSI is the 7-layer theoretical reference model used for understanding and troubleshooting.

-- Data encapsulation going DOWN the stack (sending):
Application: "GET /index.html HTTP/1.1"         (HTTP message)
Transport:   [TCP header | HTTP message]         (segment: adds port numbers)
Internet:    [IP header | TCP segment]           (packet: adds IP addresses)
Link:        [MAC header | IP packet | FCS]      (frame: adds MAC addresses)
Physical:    10101010110101...                   (raw bits on wire)

-- Going UP the stack (receiving): each layer strips its header
-- Encapsulation going down, Decapsulation going up

This is the most comprehensive network question — it tests OSI, DNS, TCP, TLS, and HTTP all at once.

1. BROWSER CACHE CHECK: Does browser have cached IP for google.com? If yes, skip DNS.

2. DNS RESOLUTION (Application layer):
   Browser asks OS resolver -> checks /etc/hosts
   OS asks recursive DNS resolver (ISP or 8.8.8.8)
   Resolver queries: Root NS -> .com TLD NS -> google.com authoritative NS
   Returns: 142.250.190.46 (Google IP)

3. TCP 3-WAY HANDSHAKE (Transport + Network layers):
   Browser -> Google: SYN (seq=x)
   Google -> Browser: SYN-ACK (seq=y, ack=x+1)
   Browser -> Google: ACK (ack=y+1)  -- Connected!

4. TLS HANDSHAKE (HTTPS -- Presentation layer):
   Client Hello: supported ciphers + random nonce
   Server Hello: chosen cipher + certificate (public key)
   Client verifies cert against trusted CAs
   Key exchange: Diffie-Hellman -> shared session key
   Encrypted communication begins

5. HTTP REQUEST (Application layer):
   GET / HTTP/1.1
   Host: www.google.com

6. SERVER RESPONSE:
   HTTP/1.1 200 OK
   [HTML body]

7. BROWSER RENDERS HTML, fetches CSS/JS/images (additional requests)

-- Switch MAC address table (learned dynamically by watching traffic):
Port 1: 00:1A:2B:3C:4D:5E  (laptop)
Port 2: 00:AA:BB:CC:DD:EE  (printer)
Port 3: 00:11:22:33:44:55  (NAS)

-- When laptop sends to printer:
Switch: dest MAC = 00:AA:BB:CC:DD:EE -> send ONLY to Port 2
-- Printer receives. Laptop and NAS do NOT see this frame.

-- Packet from laptop to google.com:
IP header:  Source=192.168.1.5 (your IP)     Dest=142.250.190.46 (Google)
MAC header: Source=laptop_MAC                 Dest=router_MAC

-- At your router (first hop):
IP header:  Source=192.168.1.5 (unchanged)   Dest=142.250.190.46 (unchanged)
MAC header: Source=router_MAC (changed)       Dest=next_isp_router_MAC (changed)

-- IP stays the same end-to-end. MAC changes at EVERY hop.
-- ARP maps IP -> MAC within each local network segment.

-- ARP: resolves IP address to MAC address within a local network
-- Needed because switches forward by MAC, but apps use IP

-- Example: laptop (192.168.1.5) wants to reach printer (192.168.1.10)
-- Laptop does not know printer MAC

1. ARP REQUEST (BROADCAST to all devices):
   "Who has 192.168.1.10? Tell 192.168.1.5"
   Sent to: FF:FF:FF:FF:FF:FF (broadcast -- all devices receive)

2. ARP REPLY (UNICAST from printer):
   "192.168.1.10 is at AA:BB:CC:DD:EE:FF"
   Sent only to laptop MAC

3. Laptop caches mapping in ARP table (TTL ~20 min):
   192.168.1.10 -> AA:BB:CC:DD:EE:FF

-- View your ARP table:
-- Windows/Linux: arp -a
-- 192.168.1.1   00-14-22-01-23-45  (router)
-- 192.168.1.10  AA-BB-CC-DD-EE-FF  (printer)

-- TCP 3-Way Handshake: establishing a reliable connection

Client                              Server
  |                                    |
  |------ SYN (seq=x) --------------->|  "I want to connect, my seq starts at x"
  |                                    |
  |<----- SYN-ACK (seq=y, ack=x+1) --|  "OK, my seq starts at y, got your x"
  |                                    |
  |------ ACK (ack=y+1) ------------->|  "Got your y. Connection established!"
  |                                    |
  |<======= DATA TRANSFER ============>|

-- Sequence numbers are random (ISN: Initial Sequence Number)
-- This prevents old packets from previous connections being mistakenly accepted

-- 4-Way Connection Termination:
Client -> Server: FIN          "I am done sending"
Server -> Client: ACK          "Got it"
Server -> Client: FIN          "I am done sending too"
Client -> Server: ACK          "Got it. Closing."
-- Client waits in TIME_WAIT state (2 x MSL) before fully closing

Why 3 steps and not 2? Two steps would let the server know the client is ready, but the client would have no confirmation the server received its sequence number. The third step (client ACK) confirms to the server that the client received the server sequence number. Both sides must confirm they can send AND receive — 3 steps is the minimum.

Flow Control: Prevents sender from overwhelming the RECEIVER. Uses sliding window.

-- Receiver advertises its window size (rwnd) in every ACK
-- Sender cannot have more than rwnd bytes "in flight" (sent but not ACKed)
-- When receiver buffer fills: Window Size = 0 (sender must pause)
-- When buffer drains: receiver sends Window Update

TCP Header: Window Size = 65535  (receiver has 65535 bytes of buffer available)

Congestion Control: Prevents overwhelming the NETWORK. TCP infers congestion from packet loss.

-- Slow Start: begin with cwnd (congestion window) = 1 MSS
-- Double cwnd every RTT until ssthresh (exponential growth)
-- Congestion Avoidance: grow linearly (+1 MSS per RTT) above ssthresh
-- On packet loss (timeout): ssthresh = cwnd/2, restart slow start
-- On 3 duplicate ACKs (Fast Retransmit): ssthresh = cwnd/2, cwnd = ssthresh

-- Effective send rate = min(rwnd, cwnd) / RTT

-- Port: 16-bit number (0-65535) identifying a specific application/service
-- Together with IP: (IP, Port, Protocol) = socket = unique endpoint

-- Port ranges:
-- 0-1023:     Well-known / System ports (require root/admin to bind)
-- 1024-49151: Registered ports (application services)
-- 49152-65535: Ephemeral / Dynamic ports (clients use these for outgoing)

-- Well-known ports (MUST know for interviews):
-- 20/21  FTP (data/control)
-- 22     SSH (Secure Shell)
-- 23     Telnet (insecure, avoid)
-- 25     SMTP (email sending)
-- 53     DNS
-- 80     HTTP
-- 110    POP3 (email retrieval)
-- 143    IMAP (email retrieval)
-- 443    HTTPS
-- 3306   MySQL
-- 5432   PostgreSQL
-- 6379   Redis
-- 27017  MongoDB
-- 8080   HTTP alternate (common for dev servers)

Idempotent: calling same request N times produces same result as calling once. GET/PUT/DELETE idempotent. POST is not — two identical POSTs create two resources.

Safe: does not modify server state. GET/HEAD/OPTIONS are safe.

HTTPS = HTTP + TLS. TLS provides: encryption (no eavesdropping), authentication (talking to real server), integrity (no tampering).

-- TLS 1.3 Handshake:

Client -> Server: ClientHello
  - TLS version 1.3
  - Supported cipher suites (e.g., AES-256-GCM-SHA384)
  - Client random nonce
  - DH public key share

Server -> Client: ServerHello + Certificate + Finished
  - Chosen cipher suite
  - Server DH public key
  - Digital certificate (contains server public key + CA signature)
  - Already encrypted (TLS 1.3 = 1-RTT handshake)

Client: verifies certificate chain against trusted CAs
  - Checks: domain matches, not expired, signed by trusted CA
  - Derives shared session key via Diffie-Hellman:
    Both sides compute same secret without ever transmitting it!

Client -> Server: Finished (encrypted)
=> Encrypted data exchange begins

-- Why DH (Diffie-Hellman)?
-- Client has: client_private + server_public -> shared_secret
-- Server has: server_private + client_public -> SAME shared_secret
-- Eavesdropper has: only public keys (cannot derive shared_secret)
-- This provides Forward Secrecy: past sessions cannot be decrypted

-- DNS Resolution: browser types "api.infosys.com"

1. CHECK LOCAL CACHE: Browser cache -> OS cache -> /etc/hosts
   If found and not expired -> done! (sub-millisecond)

2. RECURSIVE RESOLVER (ISP DNS or 8.8.8.8):
   Your system asks the configured resolver

3. ROOT NAMESERVER (13 root server clusters globally):
   Resolver: "Where is .com?"
   Root NS: "Ask .com TLD server at 192.5.6.30"

4. TLD NAMESERVER (.com):
   Resolver: "Where is infosys.com?"
   TLD NS: "Ask ns1.infosys.com (their authoritative NS)"

5. AUTHORITATIVE NAMESERVER (infosys.com):
   Resolver: "What is api.infosys.com?"
   Auth NS: "api.infosys.com = 103.22.14.5, TTL=300"

6. Resolver caches result for TTL=300 seconds, returns to browser

7. Browser connects to 103.22.14.5

-- Cost: ~50-100ms cold. ~1ms cached.

-- DNS Record Types (must know):
-- A:     hostname -> IPv4 address  (api.infosys.com -> 103.22.14.5)
-- AAAA:  hostname -> IPv6 address
-- CNAME: alias -> canonical name   (www -> infosys.com)
-- MX:    mail server for domain    (infosys.com -> mail.infosys.com)
-- TXT:   arbitrary text            (SPF, DKIM, domain verification)
-- NS:    authoritative nameservers for domain
-- SOA:   start of authority (zone metadata, TTL policies)

-- CIDR notation: IP/prefix_length
-- /24 means first 24 bits are NETWORK portion
-- Remaining 8 bits (32-24) are HOST portion

-- 192.168.1.0/24:
-- Network:  192.168.1   (24 bits, fixed for all devices in this subnet)
-- Hosts:    .0 to .255  (8 bits, 256 values)
-- Usable:   .1 to .254  (254 hosts -- .0=network address, .255=broadcast)
-- Subnet mask: 255.255.255.0

-- Common CIDR blocks:
-- /32  = 1 IP         (single host, firewall rules)
-- /30  = 4 IPs        (2 usable, point-to-point links)
-- /28  = 16 IPs       (14 usable)
-- /24  = 256 IPs      (254 usable, typical LAN)
-- /16  = 65,536 IPs   (large corporate network)
-- /8   = 16M IPs      (ISP or cloud region)

-- Private ranges (RFC 1918 -- NOT routable on public internet):
-- 10.0.0.0/8          (10.x.x.x)
-- 172.16.0.0/12       (172.16.x.x to 172.31.x.x)
-- 192.168.0.0/16      (192.168.x.x -- your home router)

-- NAT (Network Address Translation):
-- Maps multiple private IPs to one public IP
-- Exists because IPv4 exhausted (4.3B addresses not enough for every device)

-- Home router NAT example:
-- Laptop:  192.168.1.5:54321  (private, not routable on internet)
-- Router:  203.0.113.10        (one public IP from ISP)

-- Outgoing (laptop -> Google):
-- Original: src=192.168.1.5:54321  dst=142.250.190.46:443
-- Router rewrites: src=203.0.113.10:40001  dst=142.250.190.46:443
-- Router stores mapping: port 40001 -> 192.168.1.5:54321

-- Reply (Google -> router):
-- Google sends to: 203.0.113.10:40001
-- Router looks up: 40001 -> 192.168.1.5:54321
-- Router rewrites dst, delivers to laptop

NAT limitation: devices behind NAT cannot be directly reached from the internet — complicates P2P (VoIP, gaming). IPv6 eliminates NAT by giving every device a globally routable address.

-- CORS: Cross-Origin Resource Sharing
-- Same-Origin Policy: JavaScript on site-A cannot read responses from site-B
-- UNLESS site-B explicitly allows it via CORS headers
-- Origin = protocol + domain + port
-- Same: https://app.com/api and https://app.com/data (same origin)
-- Different: https://app.com and https://api.app.com (different subdomain!)

-- Simple GET request flow:
Browser -> api.infosys.com:
  GET /data
  Origin: https://myapp.com     (browser adds automatically)

Server responds if allowed:
  HTTP 200 OK
  Access-Control-Allow-Origin: https://myapp.com
  Access-Control-Allow-Methods: GET, POST
  Access-Control-Allow-Headers: Content-Type, Authorization

-- Preflight (for POST/PUT/DELETE with custom headers):
Browser -> Server: OPTIONS request first
  Origin: https://myapp.com
  Access-Control-Request-Method: POST
  Access-Control-Request-Headers: Content-Type

Server approves -> browser sends actual POST

-- HTTP (request-response): client MUST ask first. Server cannot push.
-- Problem: real-time apps (chat, live scores, trading) need server to push data

-- Without WebSockets (painful alternatives):
-- Short Polling: client asks every 1 second -> 99% empty responses, wasteful
-- Long Polling: client asks, server holds open until data -> one response per TCP connection

-- WebSocket: full-duplex persistent connection over single TCP
-- Starts as HTTP then upgrades:
GET /chat HTTP/1.1
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==

Server:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
-- Now it is a raw TCP connection, no more HTTP overhead

-- After upgrade: EITHER side can send frames ANYTIME:
Server -> Client: {"type":"message","text":"Hello!","user":"Alice"}
Client -> Server: {"type":"typing","user":"Bob"}

-- Frame overhead: 2-14 bytes (vs 800+ bytes HTTP headers per message)
-- Used in: chat (WhatsApp Web), dashboards, multiplayer games, stock tickers

-- CDN (Content Delivery Network): globally distributed servers (PoPs)
-- cache and serve content from the location closest to the user

-- Without CDN (origin in Mumbai only):
-- User in New York: Mumbai and back  ~200ms
-- User in London:   Mumbai and back  ~150ms

-- With CDN (Cloudflare: 300+ PoPs worldwide):
-- User in New York:  Newark PoP      ~5ms
-- User in London:    London PoP      ~3ms

-- Cache HIT: CDN returns cached copy (no origin contact)
-- Cache MISS: CDN fetches from origin, caches it, returns to user

-- What CDNs provide:
-- Static asset caching (JS, CSS, images, video, fonts)
-- TLS termination at the edge (encryption closer to user)
-- DDoS mitigation (absorb traffic at edge before it hits origin)
-- HTTP/3 and HTTP/2 at edge even if origin uses HTTP/1.1
-- WAF and bot detection (Cloudflare WAF)

-- Cache-Control headers:
Cache-Control: public, max-age=31536000  (1 year -- immutable static assets)
Cache-Control: no-cache                  (revalidate with server each time)
Cache-Control: private                   (user-specific, CDN must NOT cache)
ETag: "abc123"                           (content hash for conditional requests)

Layer 4 vs Layer 7 Load Balancing: L4 (TCP level): routes by IP/port, very fast, cannot inspect content. L7 (HTTP level): routes by URL path, headers, cookies — smarter routing. Examples: L4 = AWS NLB. L7 = AWS ALB, nginx, HAProxy. Use L7 to route /api/* to API servers and /static/* to CDN origin.

-- CACHE-ASIDE (Lazy Loading) -- most common:
result = cache.get("user:42")
if result is None:
    result = db.query("SELECT * FROM users WHERE id=42")
    cache.set("user:42", result, ttl=300)
return result
-- Only caches what is actually requested
-- Cache miss = 3 trips (check cache + hit DB + write cache)

-- WRITE-THROUGH: write to cache AND DB simultaneously
cache.set("user:42", data)
db.update(data)
-- Always in sync. Every write hits both cache and DB.

-- WRITE-BACK (Write-Behind): write to cache only, flush DB async
cache.set("user:42", data)
# background worker: flush to DB every N seconds
-- Fastest writes. Risk: data loss if cache crashes before flush.

-- Sharding: split data across multiple DB instances (each = a shard)
-- Required when: single DB cannot handle write volume or storage

-- HASH-BASED: shard = hash(user_id) % num_shards
-- user_id=1001: hash % 4 = 1 -> Shard 1
-- user_id=1002: hash % 4 = 2 -> Shard 2
-- Pros: even distribution. Cons: resharding moves most data!

-- RANGE-BASED: user_id 1-1M on Shard1, 1M-2M on Shard2
-- Pros: simple, easy range queries. Cons: hot spots (newest users on last shard)

-- DIRECTORY-BASED: lookup table: user_id -> shard_id
-- Pros: flexible, easy resharding. Cons: lookup table is a bottleneck

-- Message Queue: async communication between services
-- Producer puts message in queue, Consumer processes independently

-- WITHOUT queue (synchronous):
User: "Send password reset email"
App: calls email service -> waits 500ms -> responds to user
-- User blocked waiting for email!

-- WITH queue (asynchronous):
User: "Send password reset email"
App: puts {email_task} in queue -> responds "Check your inbox!" immediately
Email worker: picks task from queue -> sends email 500ms later
-- User gets instant response!

-- Queue benefits:
-- 1. Decoupling: producer and consumer scale independently
-- 2. Buffering: absorbs traffic spikes without losing requests
-- 3. Reliability: messages persist if consumer crashes (reprocessed)
-- 4. Load leveling: 10,000 req/sec burst -> queue absorbs -> process at 100/sec

-- Rate Limiter: limits requests per client per time window
-- Prevents: API abuse, DDoS, resource exhaustion

-- TOKEN BUCKET (most common, allows bursts):
-- Bucket holds N tokens. 1 request = 1 token consumed.
-- Tokens refill at fixed rate (e.g., 10/second, max 100)
-- Request: if tokens > 0, consume and allow. Else: 429 Too Many Requests.

-- SLIDING WINDOW COUNTER (accurate):
-- Track request timestamps. Count in last 60 seconds.
-- Memory-intensive but no edge-case at window boundary.

-- FIXED WINDOW COUNTER (simple but flawed):
-- Count per minute. Edge case: 100 at 00:59 + 100 at 01:01 = 200 in 2 sec!

-- Redis implementation (atomic, distributed):
def is_allowed(user_id):
    key = f"rate:{user_id}:{current_minute()}"
    count = redis.incr(key)        # atomic increment, returns new value
    if count == 1:
        redis.expire(key, 60)      # set TTL on first request
    return count <= 1000           # True=allow, False=reject (429)

-- Response headers:
X-RateLimit-Limit:     1000
X-RateLimit-Remaining: 847
X-RateLimit-Reset:     1714140000
Retry-After:           60   (on 429: seconds to wait)

An API Gateway is a single entry point for all client requests to backend microservices. It handles: (1) Request routing (URL path to correct service), (2) Authentication (verify JWT once, not in every service), (3) Rate limiting at the edge, (4) SSL termination (HTTPS at gateway, HTTP internally), (5) Load balancing across service instances, (6) Response transformation and aggregation, (7) Centralized logging and monitoring, (8) Circuit breaking for unhealthy services. Popular options: AWS API Gateway, Kong, NGINX, Traefik, Apigee, Netflix Zuul.

With simple hashing (hash(key) % N servers): adding/removing one server invalidates (N-1)/N of all cached keys — causing a thundering herd on the DB. Consistent hashing: place servers and keys on a circular ring (0 to 2′). Each key is stored on the first server clockwise from its hash position. Adding a server: only keys between its predecessor and itself migrate. Removing: only that server keys move to the next. Result: only 1/N of keys migrate (vs nearly all with modulo hashing). Virtual nodes (each physical server placed at K positions) provide better load distribution. Used in: Cassandra, Amazon DynamoDB, Redis Cluster.

Use SQL when: complex relationships and joins needed, ACID transactions required (payments, booking, inventory deduction), schema is stable and well-defined, or reporting with GROUP BY/aggregations. Use NoSQL when: massive write throughput needed (Cassandra: IoT, clickstream), schema varies per document (product catalog), simple key-based access with no complex joins, or extreme read speed (Redis: sessions, leaderboards, caches). Real examples: Bank ledger → PostgreSQL. User sessions → Redis. Product catalog with varied fields → MongoDB. Social feed → Cassandra. Most large systems use BOTH: PostgreSQL for transactional data + Redis for caching + Elasticsearch for search.

-- REQUIREMENTS:
-- Functional: shorten URL, redirect short -> long, optional analytics
-- Non-functional: 100M URLs, 10:1 read:write, <100ms redirect latency

-- SCALE ESTIMATE:
-- Write: 100M / 30days = ~40 URLs/sec
-- Read:  ~400 redirects/sec (10:1 ratio)
-- Storage: 100M * 500 bytes = 50GB (manageable)

-- HIGH-LEVEL DESIGN:
Client -> Load Balancer -> API Servers -> Redis cache
                                       -> PostgreSQL

-- SHORT CODE GENERATION (base62):
-- 62 chars: a-z (26) + A-Z (26) + 0-9 (10)
-- 6 chars: 62^6 = 56 billion unique codes
-- Generate random 6-char string, check DB for collision, retry if collision

-- DB SCHEMA:
CREATE TABLE urls (
    short_code  VARCHAR(8) PRIMARY KEY,
    long_url    TEXT NOT NULL,
    created_at  TIMESTAMP,
    expires_at  TIMESTAMP,
    user_id     INT
);

-- REDIRECT FLOW (cache-aside):
GET /{short_code}
1. Redis.get(short_code) -> HIT: return 301/302 redirect immediately
2. MISS: PostgreSQL query -> cache result (TTL=3600s) -> return redirect

-- 301 vs 302: 301=permanent (browser caches, no analytics). 302=temporary (every click tracked)
-- For analytics use 302.

-- REQUIREMENTS:
-- 1:1 messaging, group chat, online status, message history
-- 50M DAU, real-time delivery, 5-year message storage

-- HIGH-LEVEL:
Client -> Load Balancer -> Chat Servers (WebSocket)
                        -> Kafka (message queue)
                        -> Cassandra (message storage)
                        -> Redis (presence/online status)

-- REAL-TIME: WebSocket (persistent bidirectional connection)
-- Each user: persistent WebSocket to a Chat Server
-- Chat Server maintains: {user_id -> WebSocket} in-memory map

-- MESSAGE DELIVERY:
-- Alice sends to Bob:
-- 1. Alice -> Alice's Chat Server -> Kafka topic
-- 2. Message consumer: is Bob connected? Check Redis.
-- 3a. Bob online: find Bob's Chat Server -> push via WebSocket
-- 3b. Bob offline: store in DB, send push notification, deliver when online

-- CASSANDRA SCHEMA (time-series):
CREATE TABLE messages (
    conversation_id UUID,
    message_id      TIMEUUID,   -- encodes timestamp, enables time-sorting
    sender_id       UUID,
    content         TEXT,
    PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);
-- Fetch last 50: SELECT ... LIMIT 50 -- one partition read

-- ONLINE PRESENCE (Redis):
-- User connects: redis.setex("online:{user_id}", 30, "1")
-- Heartbeat every 20s: renew TTL
-- Check: redis.get("online:bob") -> not None means online

A notification system decouples event producers from notification delivery. Core components: (1) API to receive events (order placed, payment done), (2) Message queue (Kafka) to absorb events, (3) Notification workers per channel (email worker, SMS worker, push worker), (4) Template engine (render notification body from event data), (5) Delivery providers (SendGrid for email, Twilio for SMS, FCM/APNs for push). Key decisions: retry logic with exponential backoff for failed deliveries, idempotency (deduplicate notifications using event ID to prevent double-sending on retry), user preferences (respect opt-outs per channel), rate limiting per user (no spam). Use Kafka for durability — if the email service is down, notifications queue up and deliver when it recovers.

Core components: (1) Rider App -> API Gateway -> Booking Service (match rider to driver), (2) Driver Location Service (drivers send GPS every 5s -> stored in Redis with geospatial index), (3) Matching Service (find nearest available drivers within radius using geospatial query), (4) Pricing Service (surge pricing = demand/supply ratio), (5) Real-time tracking (WebSocket for live map updates). Key tech choices: Redis GEOADD/GEORADIUS for driver location (O(n) radius search), Kafka for trip events, PostgreSQL for trip history + billing, WebSocket for live location streaming. The hardest part: exactly-once driver assignment (prevent two riders from booking same driver simultaneously) — use Redis atomic SETNX or DB row-level locking.

See Q30 for the complete API Gateway answer. In design problems: always add an API Gateway when you have multiple microservices. It prevents the client from knowing about individual service URLs, handles cross-cutting concerns (auth, rate limiting, logging) in one place, and provides a stable contract to clients even when backend services are refactored. Without a gateway: every client must know every service URL, every service must implement auth and rate limiting redundantly, and changing a service URL breaks all clients.

Core: in-memory hash map (O(1) GET/SET/DELETE). Persistence: WAL (append-only log) + periodic snapshots. Replication: primary accepts writes, replicas stream WAL. Expiration: store TTL with each key, background thread sweeps expired keys. Eviction: when memory full, use LRU to evict least recently used keys. Cluster: consistent hashing to distribute keys across nodes. Why in-memory? RAM access ~100ns vs disk ~1ms (10,000x faster). The core data structure is just a HashMap — Redis uses hash tables with open addressing and incremental rehashing to avoid pauses.

Functional vs Non-Functional: Functional = WHAT ("user can log in with email + password"). Non-Functional = HOW WELL ("login completes within 2 seconds at P95 for up to 10,000 concurrent users"). Both must be testable — "the system shall be fast" is NOT a valid requirement.

Feasibility study is done BEFORE committing resources. It is much cheaper to discover "we cannot technically build this" at the feasibility stage than after 6 months of development. Output: feasibility report recommending go/no-go decision to stakeholders.

Testing pyramid: Many unit tests (fast, cheap, isolated) + moderate integration tests + few E2E tests (slow, brittle, expensive). Inverting the pyramid (many E2E, few unit) creates slow, fragile test suites that slow down CI/CD.

The Agile Manifesto (2001) states four core values:

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

"While there is value in the items on the right, we value the items on the left more." Agile does NOT eliminate documentation or contracts — it prioritizes the left when there is a conflict between the two.

Definition of Done (DoD): Shared team agreement on what "Done" means. Example: Code written + reviewed, unit tests passing (>80% coverage), integration tests passing, deployed to staging, documentation updated, no critical bugs. Without DoD, "Done" means different things to different people — Scrum requires a formal shared DoD.

User Story format: "As a [type of user], I want [goal] so that [reason]." Must have testable Acceptance Criteria. Example: As a registered user, I want to reset my password via email so that I can recover my account. AC: email arrives within 2 min, link expires in 24h, used link shows clear error.

-- STORY POINTS: relative measure of effort + complexity + uncertainty
-- NOT hours! A 3-point story is roughly 3x harder than a 1-point story
-- Common scales: Fibonacci (1,2,3,5,8,13,21) or T-shirt (XS,S,M,L,XL)

-- PLANNING POKER:
-- Facilitator reads user story
-- Each team member privately picks a card (story point value)
-- All reveal SIMULTANEOUSLY (prevents anchoring bias)
-- Outliers explain their reasoning (1 vs 13 = valuable discussion!)
-- Re-estimate until consensus or average

-- VELOCITY: average story points completed per sprint
-- Sprint 1: 34, Sprint 2: 28, Sprint 3: 32 -> velocity = 31
-- Used for: 200 points of backlog / 31 = ~6.5 sprints = ~13 weeks

-- Why Fibonacci? Gaps grow with size, reflecting uncertainty:
-- "1 or 2?" = small, easy to estimate
-- "13 or 21?" = large uncertainty, probably needs to be split!
-- Rule: story > 13 points -> SPLIT it (too big, too uncertain)

-- ANTI-PATTERNS:
-- Management using velocity as KPI -> teams inflate estimates
-- Comparing velocity between teams -> meaningless (different baseline)
-- Velocity as commitment -> it is a planning tool, not a contract

1. Set the stage: Check-in question. Create psychological safety (what happens in retro stays in retro).
2. Gather data: What went well? What was painful? Concrete, specific observations.
3. Generate insights: Why did problems occur? Root cause (5 Whys technique).
4. Decide what to do: Select 1-3 improvement actions MAX. Specific, assigned to person, measurable.
5. Close: Each person shares one word about the retro itself.

Most common failure mode: Teams discuss problems but leave with no action items. Or create 20 items and do zero. The SM tracks previous retro actions in the next retro — were they completed? A retro without action is a waste of time.

-- TDD: write the TEST FIRST, then write code to make it pass

-- RED: write a failing test (feature does not exist yet)
@Test
void testCalculateTax_standardRate() {
    double tax = TaxCalculator.calculate(1000, "standard");
    assertEquals(180.0, tax);  // FAILS: method does not exist
}

-- GREEN: write MINIMUM code to pass the test
class TaxCalculator {
    static double calculate(double amount, String type) {
        return amount * 0.18;  // just enough to pass
    }
}
// Test NOW PASSES

-- REFACTOR: improve code quality, tests must still pass
class TaxCalculator {
    private static final Map RATES = Map.of(
        "standard", 0.18, "reduced", 0.05, "zero", 0.0
    );
    static double calculate(double amount, String type) {
        return amount * RATES.getOrDefault(type, 0.18);
    }
}
// All tests still PASS. Code is cleaner and extensible.

TDD benefits: Tests serve as executable documentation. No code without a test (prevents untested code). Forces modular design (untestable code = bad design signal). Enables fearless refactoring (tests catch regressions instantly).

-- CI (Continuous Integration): merge code frequently, automated build+test every time
-- CD (Continuous Delivery): code always in deployable state, one-click release
-- CD (Continuous Deployment): automatic deploy on every successful CI run

-- TYPICAL 8-STAGE PIPELINE:
Developer pushes code to GitHub
        |
[1. SOURCE TRIGGER] -- webhook fires
        |
[2. BUILD]
   Compile: mvn package / gradle build
   Resolve dependencies
        |
[3. UNIT TESTS]
   Run all unit tests
   Fail fast: stop pipeline if any test fails
        |
[4. CODE QUALITY]
   Static analysis: SonarQube, Checkstyle
   Coverage gate: must be > 80%
   Security scan (SAST): Snyk, Checkmarx
        |
[5. BUILD ARTIFACT]
   Create Docker image
   Push to container registry (AWS ECR, JFrog)
        |
[6. DEPLOY TO STAGING]
   Automatic deploy to staging environment
        |
[7. INTEGRATION / E2E TESTS]
   API tests (Postman/Newman)
   Selenium E2E tests against staging
        |
[8. DEPLOY TO PRODUCTION]
   Manual approval (CD) or automatic (CDeployment)
   Blue-green or rolling deployment strategy
   Smoke tests post-deploy
   Monitor: error rate spike -> automatic rollback

DORA Metrics (measure DevOps maturity): Deployment Frequency (how often you deploy), Lead Time for Changes (commit to production time), Change Failure Rate (% of deployments causing incidents), Mean Time to Recovery (how fast you recover). Elite teams: multiple deployments per day, <1hr lead time, <15% failure rate, <1hr recovery.

-- Bug Life Cycle:
New -> Assigned -> Open (investigating) -> Fixed
                                             |
                               Retest: PASS -> Closed
                               Retest: FAIL -> Reopened -> Open again
-- Other states: Deferred (next release), Rejected (not a bug), Duplicate

Shift-Left Testing: Moving testing activities earlier (to the left) in the SDLC, rather than testing only at the end. Instead of: code → code → code → TEST, do: code+test → code+test → code+test.

Benefits: Defects found early are 10-100x cheaper to fix. A bug found in requirements costs 1x. In design: 5x. In coding: 10x. In testing: 15x. In production: 100x. Shift-left prevents the expensive production defects.

Continuous Testing in CI/CD: Every code commit triggers automated tests. Unit tests on every commit (fast: <1 min). Integration tests on every merge to main (medium: 5-15 min). Full E2E suite on every release candidate (slow: 30-60 min). No manual testing gates before deployment — automation provides the confidence.

TDD + BDD: TDD (unit level) + BDD (Behavior Driven Development, using Gherkin: Given/When/Then scenarios) combine to create a full automated test suite that serves as living documentation of system behavior.

Technical Debt: The cost of shortcuts during development that make future changes harder. Like financial debt: manageable in small amounts, crippling if accumulated. Fix it during dedicated refactoring sprints or "boy scout rule" — always leave code cleaner than you found it.

Verification: "Are we building the product RIGHT?" — checking that the work products conform to specifications. Did we implement the design correctly? Static analysis: code review, inspections, walkthroughs. Example: verify code matches the design document. Validation: "Are we building the RIGHT product?" — checking that the product meets actual user needs. Dynamic testing with real execution. Example: UAT where actual users confirm the system solves their real problem. V&V together ensure both correctness (verification) and fitness for purpose (validation). A system can be perfectly verified (matches specs exactly) but fail validation (specs did not capture actual user needs — the requirements gathering failed).

A test case is a specific set of inputs, preconditions, execution steps, and expected results designed to verify a particular aspect of a system. Key components: (1) Test Case ID (unique identifier), (2) Test Case Name (descriptive), (3) Preconditions (what must be true before executing), (4) Test Steps (numbered actions to perform), (5) Test Data (specific inputs to use), (6) Expected Result (what should happen), (7) Actual Result (filled during execution), (8) Status (Pass/Fail/Blocked). Example: Test ID: TC-LOGIN-001, Name: Valid login with correct credentials, Precondition: User registered with email test@example.com password Test123!, Steps: 1. Go to /login 2. Enter email 3. Enter password 4. Click Submit, Expected: Redirected to /dashboard, welcome message shown.