Networking Reference for Developers

What actually happens on the wire — layers, protocols, tools, and infrastructure a developer needs to understand.

The Model: What a Dev Needs to Know

The OSI model has 7 layers. In practice, the internet collapses into roughly 4 functional layers that matter to developers. You don't need the academic version — you need to know which layer a problem lives in so you can debug it.

What You Call ItWhat It DoesProtocolsWhat You Debug Here
ApplicationWhat your app sends/receivesHTTP, DNS, SSH, FTP, SMTPHTTP status codes, malformed requests, wrong data format
TransportReliable delivery between appsTCP, UDPConnection refused, timeouts, connection reset
Network / IPAddressing and routing between machinesIP, ICMP (ping), ARPHost unreachable, no route to host, wrong IP
Physical / LinkBits on the wire, frame delivery on a segmentEthernet, WiFi, MACCable unplugged, wrong VLAN, MAC filtering

The layer determines where to look. "Connection refused" is transport (TCP). "No route to host" is network (IP). "HTTP 400" is application (HTTP). Knowing the difference saves hours of debugging.

Physical vs Logical: The Key Distinction

Physical = the actual cables, radio waves, electrical signals. A wire is a physical thing. A WiFi signal is a physical thing.

Logical = the addressing and organization built on top of physical connections. IP addresses, subnets, VLANs, ports, protocols — these are all logical constructs that exist in software/config, not in the wire itself.

Physical layer: cable/wireless carries electrical signals Data link (logical): signals grouped into frames with MAC addresses (Ethernet) Network (logical): frames grouped into packets with IP addresses Transport (logical): packets broken into segments with ports (TCP/UDP) Application (logical): segments carry your actual data (HTTP body, SQL query, etc.)

A switch operates at the data link layer. It sees MAC addresses, not IP addresses. It delivers frames within a local network. A switch port is a physical jack. A VLAN is a logical partition of that switch — same physical cables, but logically isolated networks.

What Happens When You Type a URL (Simplified)

1. Browser parses "https://example.com/page" → extracts host, path 2. DNS lookup: example.com → 93.184.216.34 3. TCP handshake with 93.184.216.34:443 (3-way: SYN, SYN-ACK, ACK) 4. TLS handshake: negotiate encryption (if HTTPS) 5. HTTP request sent: GET /page HTTP/1.1 Host: example.com Connection: keep-alive 6. Server processes request, runs app code, generates HTML 7. HTTP response sent: HTTP/1.1 200 OK Content-Type: text/html Content-Length: 1234 <html>...</html> 8. Browser renders the page

Steps 2-4 happen in milliseconds. Step 5-7 is what your code controls. If something fails at step 2, it's a DNS problem. Step 3, it's network/firewall. Step 5-7, it's your app or the server.

Frames, Packets, Segments, Messages

These are all names for "data on the wire" at different layers:

NameLayerContainsAddressed By
FrameData link (Ethernet)Packet + MAC addresses + checksumMAC address (AA:BB:CC:DD:EE:FF)
PacketNetwork (IP)Segment + source/dest IP + TTLIP address (192.168.1.1)
SegmentTransport (TCP/UDP)Your data + source/dest port + sequence numbersPort (80, 443, 5432)
MessageApplication (HTTP)Headers + body (the actual content)URL path (/page)

Nesting: Frame wraps Packet wraps Segment wraps Message. When you send an HTTP request, it gets wrapped in a TCP segment, which gets wrapped in an IP packet, which gets wrapped in an Ethernet frame. Each layer adds its own header. This is called encapsulation.

MAC Address vs IP Address

MAC AddressIP Address
LayerData link (2)Network (3)
FormatAA:BB:CC:DD:EE:FF (48-bit, hex)192.168.1.1 (32-bit, decimal)
ScopeLocal network segment onlyGlobal (routable)
Assigned byHardware manufacturer (burned in)Network admin / DHCP
Changes?No (per interface)Yes (different network = different IP)
What a switch seesMAC addressesNothing (switches don't look at IP)
What a router seesNothing (routers strip MAC, add new one)IP addresses

MAC addresses don't cross routers. When a packet goes through a router, the router strips the source/dest MAC from the frame and adds new ones pointing to the next hop. MAC addresses are only meaningful on a local network segment. This is why you can't find someone's MAC address from across the internet.

TCP (Transmission Control Protocol)

TCP is reliable. It guarantees that data arrives intact, in order, and without duplication. It does this through:

The TCP Handshake (3-Way)

ClientSYN (I want to talk, my sequence starts at X) ServerSYN-ACK (OK, I acknowledge X, my sequence starts at Y) ClientACK (I acknowledge Y) === connection established === Data flows in both directions... Either sideFIN (I'm done sending) Other sideACK + FIN (OK, I'm done too) First sideACK (confirmed) === connection closed ===

"Connection refused" = the server sent a RST instead of SYN-ACK. Either nothing is listening on that port, or a firewall rejected it. "Connection timed out" = no response at all (firewall dropping packets, or host unreachable).

TCP State Machine (The Important Parts)

StateMeaning
LISTENServer is waiting for connections on a port
SYN_SENTClient sent SYN, waiting for SYN-ACK
ESTABLISHEDHandshake complete, data can flow
FIN_WAIT_1Initiated close, waiting for ACK
FIN_WAIT_2Sent FIN + got ACK, waiting for other side's FIN
CLOSE_WAITGot other side's FIN, haven't closed yet (app hasn't called close())
TIME_WAITClosed, waiting 2MSL (~60s) to ensure the other side got the ACK
CLOSEDConnection fully closed

TIME_WAIT is normal, not an error. After closing a connection, the side that initiated the close stays in TIME_WAIT for ~60 seconds. It prevents old packets from a previous connection from being confused with a new one. Thousands of TIME_WAIT connections can exhaust ports — this is why connection reuse and keep-alive matter.

UDP (User Datagram Protocol)

UDP is unreliable. No handshake, no acknowledgments, no ordering, no retransmission. It just sends data and hopes it arrives. So why use it?

ProtocolTransportWhy
HTTP/1.1TCPReliable page delivery matters
HTTP/2, HTTP/3TCP / UDP (QUIC)HTTP/3 uses UDP (QUIC) for faster connection setup
DNSUDPSmall queries, retry if no response is faster than TCP handshake
SSHTCPReliable shell session
Video streamingUDPDropped frames = momentary glitch, not a broken page
Databases (PostgreSQL, MySQL)TCPData integrity is non-negotiable
Logging/metricsUDPLosing a log line is acceptable
VPN (WireGuard)UDPImplements its own reliability on top of UDP

Ports

Ports are transport-layer addresses. They identify which application on a machine should receive the data. They range from 0 to 65535.

RangeNameExamples
0-1023Well-known (need root/admin)80 (HTTP), 443 (HTTPS), 22 (SSH), 53 (DNS), 25 (SMTP)
1024-49151Registered (assigned to specific apps)3306 (MySQL), 5432 (PostgreSQL), 6379 (Redis), 8080 (HTTP alt)
49152-65535Ephemeral (temporary, auto-assigned)Your outgoing connections use these

A port is just a number in a header field. It's not a physical thing. When you say "port 80 is open," it means "a program on this machine is listening for TCP segments addressed to port 80." Multiple programs can use port 80 on different IP addresses (different interfaces).

"Address already in use" means another process is already bound to that port. Use ss -tlnp | grep :8000 to find what's using it. kill <PID> to stop it. Or use a different port.

How Your OS Handles Connections

# A TCP connection is defined by 4 things (a socket tuple):
#   source IP + source port + dest IP + dest port
# Same dest IP+port, different source port = different connection
# This is why your browser can open 6 connections to the same server at once

# Only ONE process can bind to a specific local IP+port at a time.
# That's why you can't start two apps on port 8000.

What Your Code Actually Controls

# When you write this in Python:
socket.connect(("93.184.216.34", 443))

# Your OS:
# 1. Picks a random ephemeral source port (e.g., 51234)
# 2. Sends SYN to 93.184.216.34:443 with source 192.168.1.100:51234
# 3. Handles the TCP handshake, ACKs, retransmissions for you
# 4. Delivers the data you write() to the transport layer
# 5. Wraps it in IP packet with dest 93.184.216.34, source 192.168.1.100
# 6. Wraps that in an Ethernet frame with your router's MAC as dest

# When you write:
server = socket.socket()
server.bind(("0.0.0.0", 8000))

# Your OS:
# 1. Tells the network stack: "accept TCP segments for port 8000"
# 2. When a SYN arrives, responds with SYN-ACK (enters SYN_RCVD state)
# 3. When ACK comes back, moves to ESTABLISHED
# 4. Calls your accept() callback

HTTP Overview

HTTP is a text-based protocol that runs on top of TCP. It's just formatted text sent over a TCP connection. That's it. No magic. Every HTTP message is either a request (client to server) or a response (server to client), and both follow the same basic structure.

HTTP Request Structure

What actually gets sent on the wire
# Request line
GET /api/users?page=2&limit=50 HTTP/1.1
# ↑ method  ↑ path+query       ↑ version

# Headers (key: value, one per line)
Host: example.com
Accept: application/json
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6...
Content-Type: application/json
User-Agent: Mozilla/5.0
Accept-Encoding: gzip, deflate
Connection: keep-alive

# Empty line separates headers from body

# Body (optional - present in POST/PUT/PATCH, absent in GET/DELETE)
{"name": "Alice", "email": "alice@example.com"}

The empty line between headers and body is mandatory. Without it, the server can't tell where headers end and body begins. If you get "400 Bad Request" and can't figure out why, check for a missing blank line between headers and body.

HTTP Response Structure

What the server sends back
# Status line
HTTP/1.1 200 OK
# ↑ version  ↑ status code  ↑ reason phrase

# Headers
Content-Type: application/json
Content-Length: 47
Cache-Control: max-age=3600
Server: nginx/1.24
X-Request-Id: abc-123-def

# Empty line

# Body
{"id": 1, "name": "Alice"}

HTTP Methods (Verbs)

MethodHas Body?Idempotent?Safe?Purpose
GETNoYesYesRetrieve a resource
POSTYesNoNoCreate a resource (or trigger an action)
PUTYesYesNoReplace a resource entirely (full update)
PATCHYesNoNoPartial update (change only the fields you send)
DELETENoYesNoDelete a resource
HEADNoYesYesLike GET but only returns headers (check if exists, get Content-Length)
OPTIONSNoYesYesAsk server what methods are allowed (CORS preflight)

Idempotent = doing it once has the same effect as doing it 100 times. GET the same page 10 times = same result. POST create 10 orders = 10 different orders. This matters for retry logic: if a request fails mid-flight, safe to retry idempotent methods.

GET with a body is technically allowed by the spec but practically never used. Proxies and caches may strip it. Use POST if you need to send data, even if you're just retrieving.

Status Codes

RangeMeaningCommon Codes
1xxInformational100 Continue, 101 Switching Protocols
2xxSuccess200 OK, 201 Created, 204 No Content
3xxRedirection301 Moved Permanently, 302 Found, 304 Not Modified
4xxClient error (your fault)400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 405 Method Not Allowed, 413 Payload Too Large, 429 Too Many Requests, 422 Unprocessable Entity
5xxServer error (their fault)500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout

The Ones That Confuse Developers

CodeWhat It Actually Means
400Your request is malformed (bad JSON, missing required header, syntax error in URL). Read the response body for details.
401"I don't know who you are." Missing or invalid authentication (wrong token, expired session).
403"I know who you are, but you're not allowed." Authentication OK, authorization denied. Wrong role, wrong permissions.
404"This URL doesn't exist on this server." Route not found. Not "server not found" (that's DNS).
405"You used the wrong HTTP method." GET on an endpoint that only accepts POST. Or POST on one that only accepts GET.
413"Your request body is too big." Usually a server config limit (nginx default: 1MB).
422"I understood your request but the data is wrong." Validation error: missing required field, wrong data type, business rule violation.
429"You're sending too many requests." Rate limiting. Slow down.
500"Something broke on our end." Unhandled exception in the app code. Check server logs.
502"The server in front of me is broken." Nginx got an invalid response from the app. App is down or crashed.
503"The server is overloaded or down for maintenance." App isn't accepting connections yet.
504"The server in front of me timed out waiting for the app." App took too long to respond.
301 vs 302301: "Moved permanently, update your URL." 302: "Moved temporarily, keep using the old URL." Browsers cache 301s by default. Use 302 for POST redirects (to prevent method change to GET).
304"Nothing changed since last time you asked." Your cached version is still valid. No body returned.

Headers That Actually Matter

HeaderDirectionWhat It Does
HostRequestThe domain name. Mandatory in HTTP/1.1. Without it, the server can't route the request to the right virtual host. "400 Bad Request" if missing.
Content-TypeBothMIME type of the body. application/json, text/html, multipart/form-data. Mandatory when there's a body.
Content-LengthResponseBody size in bytes. Lets the client know when the response is complete. Replaced by Transfer-Encoding: chunked for streaming.
AuthorizationRequestCredentials. Usually Bearer <token> for JWT, or Basic base64(user:pass) for simple auth.
AcceptRequest"I can understand these formats." application/json, text/html. Server picks the best match.
Cache-ControlResponse"You can cache this for N seconds." max-age=3600, no-cache, no-store, private.
Set-CookieResponse onlyTell the browser to store a cookie. Cannot be set by JavaScript. This is why you can't read HttpOnly cookies from document.cookie.
Access-Control-Allow-OriginResponse onlyCORS: which origins can call this API. * = any origin. Never use * with credentials.
Access-Control-Allow-HeadersResponse onlyCORS: which request headers are allowed in preflight.
X-Forwarded-ForRequestOriginal client IP when behind a proxy. X-Forwarded-For: client_ip, proxy1_ip. Trust chain.
X-Request-IdResponseUnique ID for the request. Essential for tracing issues through logs across services.
User-AgentRequestBrowser/app identification string. Used by servers for analytics and compatibility.
ConnectionBothkeep-alive = reuse the TCP connection for multiple requests. close = close after one request.
Transfer-EncodingResponsechunked = send body in chunks (no need to know Content-Length upfront). Enables streaming.

HTTP/1.1 vs HTTP/2 vs HTTP/3

HTTP/1.1HTTP/2HTTP/3
TransportTCPTCPUDP (QUIC)
Connections1 request per TCP connection (or pipelined)Multiplexed: many requests over one TCP connectionMultiple streams over one UDP "connection"
HeadersFull headers repeated every requestHPACK compression: headers sent once, referenced afterQPACK: similar to HPACK
OrderingSequential (head-of-line blocking)Streams can interleave freelySame as HTTP/2
TLSSeparate handshake (2 RTTs added)Embedded (saves ~1 RTT)Embedded (same as HTTP/2)
Connection setupTCP handshake (1 RTT) + TLS (1-2 RTTs) = 2-3 RTTsTCP + TLS = 2-3 RTTsTLS (no TCP) = 1 RTT
Head-of-line blockingYes: browser waits for one response before sending next request on same connectionNo: multiplexed, no HOL blockingNo

HTTP/3 is not "HTTP over UDP" in the simple sense. It uses QUIC, which adds its own reliability, ordering, and congestion control on top of UDP. Applications still see an HTTP-like interface. The complexity is pushed into the library, not your code.

Keep-Alive and Connection Reuse

# HTTP/1.1 default: keep-alive is ON
# Without keep-alive (Connection: close):
#   Request 1: TCP handshake (1 RTT) + HTTP request/response (1 RTT) = 2 RTTs
#   Request 2: TCP handshake + HTTP = 2 RTTs
#   Request 3: same = 2 RTTs
#   Total: 6 RTTs for 3 requests

# With keep-alive (Connection: keep-alive):
#   Request 1: TCP handshake (1 RTT) + HTTP request/response = 2 RTTs
#   Request 2: just HTTP = 1 RTT
#   Request 3: just HTTP = 1 RTT
#   Total: 4 RTTs for 3 requests

# HTTP/2/3: all requests multiplexed over ONE connection:
#   Setup: 1 RTT (HTTP/3) or 2-3 RTTs (HTTP/2)
#   Requests 1-3 (and hundreds more): 1 RTT each (all in parallel if server allows)
#   Total: 2-4 RTTs for 3 requests

Form Data: Content-Type Matters

application/x-www-form-urlencoded (default for HTML forms)
# What your browser sends when you submit <form> without enctype
POST /submit HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 27

name=Alice&email=alice%40example.com&age=30
# URL-encoded: spaces become +, special chars become %XX
multipart/form-data (for file uploads)
# What your browser sends when <form enctype="multipart/form-data">
POST /upload HTTP/1.1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Length: 342

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="file"; filename="photo.jpg"
Content-Type: image/jpeg

[binary data]
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="description"

A photo of Alice
------WebKitFormBoundary7MA4YWxkTrZu0gW--
application/json (APIs)
POST /api/users HTTP/1.1
Content-Type: application/json
Content-Length: 47

{"name": "Alice", "email": "alice@example.com"}
# No URL encoding needed — JSON is the raw body, not form fields

Wrong Content-Type = wrong parsing. Sending JSON with application/x-www-form-urlencoded (or no Content-Type) will make the server try to parse JSON as form fields, which silently fails. Always set Content-Type when you have a body.

What DNS Does

DNS translates domain names (human-readable) into IP addresses (machine-readable). It's the phonebook of the internet. Without it, every URL would be http://93.184.216.34/.

The Resolution Process

1. Browser cache → "Have I looked up example.com recently?" (check TTL) If not found ↓ 2. OS cachecheck /etc/hosts, nscd, systemd-resolved If not found ↓ 3. Recursive resolver (usually your ISP's DNS, e.g., 8.8.8.8 or 1.1.1.1) ├─ Check its own cache ├─ If cached: return IP immediately └─ If not: continue ↓ 4. Root nameserver → "Who handles .com?" Returns the address of .com's authoritative servers 5. .com TLD server → "Who handles example.com?" Returns the address of example.com's nameserver (usually your DNS provider's) 6. Authoritative nameserver → "What is example.com?" Returns: 93.184.216.34 Result is cached at every level for the TTL duration

DNS lookups typically take 20-120ms. But they're cached aggressively. Once resolved, subsequent requests are instant. The TTL (Time To Live) in the DNS record controls cache duration. Low TTL = fresh data, more lookups. High TTL = stale data, fewer lookups.

Record Types

Reverse DNS (IP → name)
TypePoints ToExample
AIPv4 addressexample.com → 93.184.216.34
AAAAIPv6 addressexample.com → 2606:2800:220:1:248:1893:25c8:1946
CNAMEAlias (redirects one domain to another)www.example.com → example.com
MXMail serverexample.com → mail.example.com (with priority)
NSNameserver (who is authoritative for this domain)example.com → ns1.dnsprovider.com
TXTArbitrary text (SPF, verification, etc.)"v=spf1 include:_spf.google.com ~all"
SRVService (hostname + port)_sip._tcp.example.com → sip.example.com:5060
PTR93.184.216.34 → example.com
SOAStart of Authority (zone metadata)Primary NS, admin email, serial, refresh, retry, expire, minimum TTL

Common DNS Tools

# Query specific record type
dig example.com A                    # IPv4 address
dig example.com AAAA                 # IPv6 address
dig example.com MX                   # mail servers
# dig example.com CNAME               # alias
# dig example.com TXT                 # text records (SPF, verification)

# Trace the full resolution path (see every step)
dig +trace example.com

# Query a specific DNS server
# dig @8.8.8.8 example.com
dig @1.1.1.1 example.com A

# Short output
# dig +short example.com
dig +short example.com A

# Reverse DNS (IP → name)
# dig -x 93.184.216.34
dig -x 93.184.216.34 +short

# Check DNS propagation after a change
dig example.com @8.8.8.8 +short

# Alternative tools
nslookup example.com                # simpler output, fewer options
host -t A example.com               # another alternative

# Check your DNS resolver
# cat /etc/resolv.conf
# nmcli dev show | grep DNS       # NetworkManager DNS (Linux)

# Flush local DNS cache (macOS)
# sudo dscacheutil -flushcache
# sudo discoveryutil mdnsflushcache

# Flush local DNS cache (Linux with systemd-resolved)
sudo systemd-resolve --flush-caches

# Flush local DNS cache (Linux with nscd)
sudo nscd -i hosts

# Flush browser DNS: clear browsing data or use devtools > Network > Disable cache (temporarily)

DNS Propagation

When you change a DNS record, it doesn't take effect everywhere instantly. Propagation happens gradually:

Set low TTLs (300s) before making changes. If your TTL is 86400 (1 day), you'll wait up to a day for changes to propagate. Set it to 300 a day before, then make the change, then set it back to 86400 after confirming it works.

Common DNS Problems

SymptomLikely Cause
Works with IP, fails with domainDNS not resolving. Check with dig. Check propagation.
Works on one device, not anotherDNS cache on the failing device. Flush it.
"Server Not Found" (NXDOMAIN)Domain doesn't exist at all, or a typo
"Server Failure" (SERVFAIL)The authoritative server is misconfigured or down
"Refused" (REFUSED)The authoritative server refuses to answer (wrong name server, no access)
"No answer" (NOERROR, empty response)The domain exists but has no records of the queried type (e.g., no AAAA records)
Sporadic failuresTTL expiring on one resolver but not another. Inconsistent propagation.
Works in browser, fails in curlcurl doesn't use the system DNS. Use curl --resolve or check /etc/resolv.conf.
CNAME loopDomain A → Domain B → Domain A. Cannot resolve.

What TLS Does

TLS (Transport Layer Security) encrypts the data between your browser and the server so eavesdroppers on the network can't read it. That's it. It doesn't make your app secure — it protects data in transit.

HTTPS = HTTP over TLS over TCP. The data on the wire is encrypted, but the server and browser see plaintext. TLS is between transport endpoints, not end-to-end encryption.

The TLS Handshake (What Actually Happens)

Client HelloI want to talk HTTPS. Here are TLS version + cipher suites I support + random number (ClientHello) Server HelloOK, using TLS 1.3, this cipher suite, my certificate, and random number (ServerHello) Server CertificateHere's my certificate proving I am example.com (signed by a CA your browser trusts) Client validates"Is this cert valid? Not expired? Correct domain? Signed by a trusted CA?" Key ExchangeClient and server agree on a shared secret without sending it in the clear (using the server's public key in the certificate) === encryption established === Application data is now encrypted before being sent

Certificates &Chain of Trust
# Certificate contains:
# - Domain name (e.g., example.com)
# - Public key of the server
#> - Who issued it (the CA)
# - Validity period (not before / not after)
# - Signature from the CA (cryptographic proof the CA issued it)

# Chain of trust:
# Browser trusts Root CAs (built into OS/browser)
# Root CA signs Intermediate CA certificates
# Intermediate CA signs your certificate
# Browser validates: your cert → intermediate → root (must be unbroken chain)

# Self-signed cert: you sign it yourself. Browsers don't trust it by default.
# Fine for dev/testing. Never use in production.

Common TLS Issues

SymptomLikely Cause
"Your connection is not private" (NET::ERR_CERT_AUTHORITY_INVALID)Self-signed cert, expired cert, cert for wrong domain, or cert signed by untrusted CA
"Certificate has expired"Cert's not-after date has passed. Renew it.
"Hostname mismatch"Cert is for api.example.com but you're connecting to staging.example.com
"Mixed content" warningHTTPS page loading HTTP resources (images, scripts). Browser blocks or warns.
Cert works in browser, fails in curl/Pythoncurl/Python don't use the OS certificate store. Use curl -k (insecure, but confirms DNS/TCP works) or set REQUESTS_CA_BUNDLE in Python.

Certificate Types (Let's Encrypt vs Commercial)

Let's EncryptCommercial (DigiCert, etc.)
CostFree$50-$2000+/year
Validity90 days max (must renew frequently)1-3 years
AutomationDesigned for full automation (certbot)Often requires manual steps
Wildcard certsSupported (via DNS-01 challenge)Supported
EV (green bar in browser)Not offeredAvailable (requires business verification)
SupportCommunity (forums, docs)Dedicated support

Renewal and Expiry

# Check when a cert expires:
echo | openssl s_client -connect example.com:443 2>/dev/null | openssl x509 -noout -enddate -startdate

# Or with dig:
# dig +short -t cert example.com @8.8.8.8

# Let's Encrypt renewal (via certbot):
# certbot automatically handles this with a cron/systemd timer
sudo certbot renew --dry-run          # test renewal without making changes
sudo certbot renew                 # actually renew
# certbot installs a timer that auto-renews when cert is 30 days old

Mixed Content

# HTTPS page loading HTTP resource → browser blocks or warns
# Common causes:
# <img src="http://example.com/img.jpg">           ← blocks the image
# <script src="http://cdn.example.com/lib.js">   ← blocks the script
# <iframe src="http://...">                     ← blocks the iframe

# Fix: use https:// for ALL resources, or upgrade-insecure-requests in CSP
# Or for dev: chrome://flags/#allow-insecure-localhost

curl

# Basic GET (same as opening in a browser)
curl https://api.example.com/users

# Verbose (see the full request and response including headers)
curl -v https://api.example.com/users

# Show only response headers
# curl -I https://api.example.com/users

# Silent mode (no progress bar)
# curl -s https://api.example.com/users

# Follow redirects
# curl -L https://example.com/nonexistent-page

# Download a file
# curl -O https://example.com/file.zip
# curl -o myfile.zip https://example.com/file.zip  # custom filename

# POST with JSON
curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d '{"name":"Alice","email":"alice@example.com"}'

# POST with form data
# curl -X POST https://example.com/login -d "username=alice&password=secret"

# POST from a file
# curl -X POST -H "Content-Type: application/json" -d @data.json https://api.example.com/users

# Add auth header
# curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIs..." https://api.example.com/me

# Send a header multiple times
# curl -H "X-Custom: value1" -H "X-Custom: value2"

# Measure response time
# curl -w "\nTime: %{time_total}s\n" -o /dev/null -s https://example.com

# Show only status code
# curl -o /dev/null -s -w "%{http_code}\n" https://example.com

# Show response headers + body
# curl -i https://api.example.com/users

# Ignore SSL certificate errors (only for debugging!)
# curl -k https://self-signed.example.com

# Resolve using a specific DNS server
# curl --resolve "example.com:443:127.0.0.1" https://example.com

# Send a cookie
# curl -b "session=abc123" https://example.com/dashboard

# Save cookies to a file, then use them in next request
# curl -c cookies.txt -X POST https://example.com/login -d "user=alice"
# curl -b cookies.txt https://example.com/dashboard

# Upload a file
# curl -F "file=@localfile.jpg" https://example.com/upload
# curl -F "file=@localfile.jpg;type=image/jpeg" https://example.com/upload
# curl -F "file=@localfile.jpg;filename=photo.jpg" https://example.com/upload

# Verbose + show request headers only (no response body)
# curl -v --trace-ascii https://api.example.com/users 2>&1 | head -30

netcat (nc)

# Listen on a port and respond manually
nc -l -p 8080
# (type HTTP response manually, Ctrl+C to close)

# Connect to a port (test if something is listening)
# nc -v example.com 80
# If connected: something is listening. If refused: nothing there.

# Connect and send raw HTTP
# printf "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n" | nc example.com 80

# Port scan a single port range
# nc -zv example.com 80 443 8000-8010
# -z: scan without sending data (connect-only)
# -v: verbose (shows open/closed/filtered)

# Simple chat between two machines
# Machine A: nc -l 9999
# Machine B: nc machine-a-ip 9999
# Both can now type and see each other's text

# Test if a port is open locally
# nc -z localhost 8000

ss (Socket Statistics)

# All listening TCP ports
ss -tlnp

# All TCP connections (listening + established)
# ss -tamp

# Filter by port
# ss -tlnp | grep :5432

# Filter by process
# ss -tlnp | grep python

# All listening sockets (TCP + UDP)
# ss -tulnp

# Show process info
# ss -tulnp -p

tcpdump (Packet Capture)

# Capture all traffic on eth0
sudo tcpdump -i eth0

# Capture only port 80 (HTTP)
# sudo tcpdump -i eth0 port 80

# Capture HTTP requests (show readable text content)
# sudo tcpdump -i eth0 -A -s 0:0 -c "tcp port 80 and (((ip[2:2] != 10) or (ip[2:2] != 0))"

# Capture DNS queries
# sudo tcpdump -i any port 53

# Save capture to file
# sudo tcpdump -i eth0 -w capture.pcap

# Read a capture file
# tcpdump -r capture.pcap

# Human-readable output
# sudo tcpdump -i eth0 -A -nn

# Filter by host
# sudo tcpdump -i eth0 host example.com

# Filter by source
# sudo tcpdump -i eth0 src host 192.168.1.100

# Combination filter
# sudo tcpdump -i eth0 "host example.com and port 443"

# Capture only SYN packets (new connections)
# sudo tcpdump -i eth0 "tcp[tcpflags] & (tcp[tcpflags] & (tcp[13] != 0x10))"

# -i = interface
# -A = print ASCII text alongside hex
# -nn = no name resolution (faster)
# -w file = write to file
# -c count = stop after N packets
# -C snapshot length (default 262144 bytes)

traceroute / tracepath

# Trace the route packets take to reach a host
traceroute example.com      # Linux/macOS (uses UDP by default)
# tracepath example.com      # modern Linux (uses ICMP)

# Output: each hop shows the router/intermediate node and the round-trip time to reach it
# If you see * * * * at some hop, that hop blocks ICMP (common for cloud providers)

dig (DNS Lookup)

# Basic lookup (same as above, repeated here for completeness)
# dig example.com A
# dig example.com AAAA
# dig example.com MX
# dig example.com TXT
# dig example.com CNAME
# dig +short example.com A

# Full resolution trace
# dig +trace example.com

# Check all record types for a domain
# dig ANY example.com

# Query a specific server
# dig @8.8.8.8 example.com A
# dig @1.1.1.1 example.com A

# Check SOA (start of authority)
# dig SOA example.com

# Check DNSSEC (signed DNS)
# dig +dnssec example.com DNSKEY
# dig +dnssec example.com A +multiline

# Reverse DNS
# dig -x 93.184.216.34
# dig -x 93.184.216.104 +short

# Check specific nameserver's info
# dig @ns1.example.com example.com NS

wireshark

# Wireshark is a GUI packet analyzer. It does what tcpdump does but with a proper interface.

# Open it, select your network interface, start capture, do the action, stop capture, then inspect.

# Key features:
# - Color-coded protocol dissection (HTTP, TCP, DNS, TLS are highlighted differently)
# - "Follow TCP stream" to see the full conversation between two endpoints
# - Filters: "http" shows only HTTP, "tcp.port == 5432" shows only PostgreSQL traffic
# - "Follow TCP stream" → right-click a packet → Follow → see full conversation
# - Statistics → Conversations → shows all TCP streams
# - Analyze → Expert Info → automated problem detection

# Export → "Export specified packets" → save as .pcap for later analysis or sharing

Switch

A switch connects devices on a local network (LAN). It operates at the data link layer (layer 2) and uses MAC addresses to decide where to send frames. It builds a table mapping MAC addresses to ports, and learns which device is on which port by watching source MAC addresses in incoming frames.

# What a switch actually does with a frame:
# Frame arrives on port 1 from MAC AA:BB:CC:DD:EE:01, dest MAC FF:FF:FF:FF:FF:FF
# Switch looks up FF:FF:FF:FF:FF:FF → it's on port 5
# Switch forwards frame out port 5
# It does NOT look at IP addresses. The IP inside the frame is irrelevant to the switch.

# Broadcast frame (dest MAC FF:FF:FF:FF:FF:FF:FF):
# Switch forwards to ALL ports (except the one it arrived on)
# This is how ARP ("Who has 192.168.1.1? Tell 192.168.1.1 → replies to that port

A switch connects devices on the same network. Two switches can be connected to expand the network. Routers connect different networks.

Router

A router connects different networks (e.g., your LAN and the internet). It operates at the network layer (layer 3) and uses IP addresses to decide where to send packets.

# What a router does:
# Packet arrives from 192.168.1.100 destined for 93.184.216.34
# Router checks its routing table: "93.184.216.34 → send via WAN port"
# Before forwarding: strips the source MAC, adds its own MAC as source MAC
# (because the previous MAC is only valid on the local segment)
#>
# This is why your app sees the server's public IP as the source IP in logs
# not your machine's local IP

Your app usually sees the router's public IP, not your machine's local IP. If you need the real client IP behind a proxy/CDN, use X-Forwarded-For header parsing. The first IP in the chain is usually the real client IP.

NAT (Network Address Translation)

NAT maps private IP + port to public IP + port. This solves two problems:

# Problem 1: IPv4 exhaustion (not enough public IPs for every device)
# Problem 2: private IPs aren't routable on the internet

# What NAT does:
# Internal: 192.168.1.10:51234 → External: 93.184.216.34:51234
# Your app connects to example.com:443 from 192.168.1.10
# Router translates source IP: 192.168.1.10 → 93.184.216.34
# When response comes back, router translates dest IP back: 93.184.216.34 → 192.168.1.10
# Your app thinks it's talking to 192.168.1.10 but it's actually going through the router
# with the public IP

Port forwarding is manual NAT for incoming connections. Outgoing connections are NAT'd automatically. But if you run a web server on 192.168.1.10:8000, the internet can't reach it because no incoming NAT mapping exists yet. You must explicitly configure: "forward port 443 to 192.168.1.10:8000" on the router.

Firewall

A firewall controls which packets are allowed through based on rules: source/dest IP, port, protocol, connection state (new/established/related).

# Types:
# - Network firewall: on your router (controls traffic between LAN and internet)
# - Host firewall: on your machine (controls traffic to/from your machine)
# - Cloud security groups: on your cloud provider (controls traffic to/from VMs)

# How rules are evaluated (usually, in order):
# 1. Allow established connections (return traffic for existing connections)
# 2. Block specific things (drop packets matching deny rules)
# 3. Allow everything else (default allow or default deny)

# Common patterns:
# Allow all outgoing (default on most setups)
# Allow incoming: only specific ports (SSH=22, HTTP=80, HTTPS=443)
# Deny all incoming by default
# Allow ping (ICMP) for diagnostics
# Block specific IPs (e.g., known bad actors)

"Connection timed out" with nothing in logs = silently dropped by firewall. Firewalls often drop packets without logging. Check both the cloud security group AND the server's host firewall. Try curl -v on the server itself to see if the port is actually listening. If yes, it's the firewall. If "Connection refused," it's not.

VPN (Virtual Private Network)

A VPN creates an encrypted tunnel through the internet to a private network. What this actually means:

# Without VPN:
# Your laptop → open internet → raw packets → server on 203.0.113.1 → MySQL on 3306
# Anyone between you and the server can see you're connecting to 203.0.113.1 on port 3306
# They don't see what's inside the packets (TLS helps, but the metadata is visible)

# With VPN:
# Your laptop → VPN client → encrypted tunnel → VPN server → private network → MySQL on 10.0.0.5:3306
# Outsiders only see: you're sending encrypted data to 203.0.113.1
# They cannot see: the destination (10.0.0.5) or the port (3306)
# It looks like you're on the private network

A VPN does NOT encrypt your app's actual data. It encrypts the transport. The app inside the VPN tunnel is still HTTP/JSON/<whatever> — it's just delivered through an encrypted tunnel. You still need HTTPS for application-level encryption.

Subnets

# A subnet divides one IP range into smaller networks
# 192.168.1.0/24 means:
#   Network: 192.168.1.0
#   First usable: 192.168.1.1
#   Last usable: 192.168.1.254
#   Broadcast: 192.168.1.255
#   Total usable IPs: 254
# /24 = 255 addresses (256 total, 1 network, 254 usable)
# /25 = 128 addresses (126 usable)
# /26 = 64 addresses (62 usable)
# /27 = 32 addresses (30 usable)
# /28 = 16 addresses (14 usable)
# /32 = 1 address (1 usable, commonly used for single hosts)

# Subnet mask tells the OS which part of the IP is "network" and which is "host"
# 255.255.255.0 → /24 → last octet is host (254 usable hosts)
# 255.255.255.128 → /25 → last 7 bits are host (126 usable hosts)
# 255.255.255.192 → /26 → last 6 bits are host (62 usable hosts)

# Default gateway: the router's IP in the subnet (usually .1)
# DNS server: usually the router's IP (same as default gateway)
# Subnet vs broadcast: don't use .0 (network) or .255 (broadcast) as host addresses
# Those have special meaning in the protocol

Load Balancer

A load balancer distributes incoming requests across multiple backend servers. What it actually does:

# Client → load balancer → picks backend
# Load balancer → backend-1 (forwards request, remembers which client went where)
# Same client → same backend (session affinity / sticky sessions)
# Backend-1 goes down → health check fails → stop sending traffic to it
# Load balancer → routes around it to healthy backends

# Common algorithms:
# Round-robin: 1→1→1→1→1 (fair distribution)
# Least connections: to the backend with fewest active connections
# IP hash: same client → same backend (sticky sessions)
# Weighted round-robin: weighted 3:1 means backend-1 gets 3 requests for every 1 to backend-2
# Random: random backend each time (good for APIs with caching)

# Health checks (how the balancer knows if a backend is alive):
# GET /health → 200 OK → healthy
# GET /health → 500 → unhealthy
# Run on every 5-10 seconds per backend

Proxy

# Forward proxy: client → proxy → server (client doesn't know about the real server)
# Example: Nginx sitting in front of your app, forwarding to localhost:8000
# Client connects to proxy's port, proxy connects to app

# Reverse proxy: client → reverse proxy → backend (client doesn't know about backends)
# Example: Nginx receives all traffic for example.com and routes it to different backends based on path:
#   / → app-server:8000
#   /api → api-server:8001
#   /ws → websocket-server:8002

# A reverse proxy also handles:
# - SSL termination (proxy handles TLS, backend sees plain HTTP internally)
# - Buffering (proxy buffers responses, not the app)
# - Compression (proxy compresses responses)
# - Rate limiting (proxy rejects excessive requests)
# - Static file serving (proxy serves images/CSS/JS directly, never hits your app)

Nginx as reverse proxy is extremely common. Almost every production deployment has one. It handles SSL, static files, compression, and routing. Your app only ever receives requests that need dynamic content.

CDN (Content Delivery Network)

# What it does: cache your static files at edge locations worldwide
# User in Tokyo requests image.jpg → CDN edge server in Tokyo serves it locally (20ms instead of 200ms from your server)

# What it caches: static files (images, CSS, JS, fonts, videos, PDFs)
# What it doesn't cache: dynamic responses (API calls, personalized pages, authenticated pages)
# (unless configured otherwise)

# Common: Cloudflare, Fastly, AWS CloudFront, Vercel, Netlify

# CDN → checks cache → cache hit: served immediately
# CDN → cache miss → fetches from origin → stores and serves

# Cache invalidation:
# Manual: "Purge all" or "Purge by URL" in CDN dashboard
# Cache-Control header: "no-cache" or "max-age=0" tells CDN not to cache
# Versioned files: style.css?v=2 → changing the ?v=3 invalidates the old version
# (this is why you see ?v=hash on CSS/JS files in browsers)