Book 3 β€” Advanced OTP & Application Design

GenServer and Supervisor cover the fundamentals; production systems need the rest of the toolbox. This book covers concurrency helpers (Task, Agent), process discovery (Registry), runtime-spawned children (DynamicSupervisor), shared fast storage (ETS), and how it all assembles into a bootable, shippable OTP application.

πŸ““ Companion notebook: book3-advanced-otp.livemd β€” parallel HTTP-style fan-out with Tasks, a process-per-user session system with Registry + DynamicSupervisor, and an ETS cache that survives GenServer crashes.

1. Task & Agent β€” concurrency without ceremony

Task wraps a process around one function call. It's your Promise/asyncio.gather equivalent β€” for when you want parallelism, not a long-lived server:

# async/await, BEAM style β€” each task is a real process on its own core
task = Task.async(fn -> expensive_computation() end)
other_work()
result = Task.await(task, 5000)

# Parallel map over a collection, max 10 at a time, streaming results:
urls
|> Task.async_stream(&fetch/1, max_concurrency: 10, timeout: 10_000)
|> Enum.to_list()
Key differences from Promises: Task.async links to the caller β€” if the task crashes, you crash (fail-fast by default). Task.async_stream gives you bounded concurrency for free β€” no p-limit library needed. For fire-and-forget from a long-lived process, use Task.Supervisor.start_child/2 so the task lives under a supervisor rather than linked to (and dying with) whatever spawned it.

Agent is a GenServer reduced to "hold this value": Agent.get/2, Agent.update/2. Use it for simple shared state with zero protocol; graduate to GenServer the moment you need timers, handle_info, or real logic. (Many teams skip Agent entirely β€” it's fine.)

2. Registry β€” finding processes by name

Book 2 registered singletons with name: __MODULE__. But what about one process per entity β€” per user, per chat room, per game? You can't atom-name them (atoms are never garbage collected β€” dynamic atom creation is a leak). Enter Registry: a local, ETS-backed process directory supporting arbitrary terms as keys.

# 1. Start a registry (in your supervision tree):
{Registry, keys: :unique, name: MyApp.Registry}

# 2. Processes register via a "via tuple" in start_link:
def start_link(user_id) do
  name = {:via, Registry, {MyApp.Registry, {:session, user_id}}}
  GenServer.start_link(__MODULE__, user_id, name: name)
end

# 3. Callers address the process by key, no PID juggling:
GenServer.call({:via, Registry, {MyApp.Registry, {:session, 42}}}, :get_cart)

When the process dies, Registry automatically removes the entry (it monitors registrants). With keys: :duplicate, many processes can register under one key β€” a lightweight local pub/sub used to build things like Phoenix.PubSub.

3. DynamicSupervisor β€” children on demand

A plain Supervisor starts a fixed list of children at boot. A DynamicSupervisor starts empty and spawns children at runtime β€” one per incoming user, game, job:

# In the tree:
{DynamicSupervisor, name: MyApp.SessionSup, strategy: :one_for_one}

# At runtime, whenever a user shows up:
DynamicSupervisor.start_child(MyApp.SessionSup, {Session, user_id})

Combine the two patterns and you get the canonical Elixir architecture β€” process-per-entity:

# "Find or start the session for user 42, then call it"
def fetch_session(user_id) do
  case Registry.lookup(MyApp.Registry, {:session, user_id}) do
    [{pid, _}] -> {:ok, pid}
    [] -> DynamicSupervisor.start_child(MyApp.SessionSup, {Session, user_id})
  end
end
Why this beats an object or a DB row in memory: each session is isolated (one user's bug can't corrupt another's), concurrent (all sessions run in parallel), supervised (crashes restart cleanly), and addressable (Registry). This exact trio β€” Registry + DynamicSupervisor + GenServer β€” powers WhatsApp-scale chat, multiplayer games, IoT device shadows, LiveView… You'll build it in the notebook and again in the capstone.

QUIZ 3.1

Why not name per-user processes with dynamically created atoms like :"user_42"?
The atom table is finite (~1M by default) and atoms live forever. Creating atoms from user input is a denial-of-service vector. Registry exists precisely so any term (integers, strings, tuples) can name a process.

4. ETS β€” when a GenServer becomes the bottleneck

Everything so far routes reads through a single process, serially. For hot shared data (caches, counters, config) that's a bottleneck. ETS (Erlang Term Storage) is an in-memory table, owned by a process but readable/writable from any process concurrently, no message passing:

table = :ets.new(:my_cache, [:set, :public, :named_table, read_concurrency: true])
:ets.insert(:my_cache, {:user_42, %{name: "Ada"}})
[{_, user}] = :ets.lookup(:my_cache, :user_42)   # microseconds, from any process
GenServer stateETS table
Accessserialized through one mailboxconcurrent from all processes
Speedmessage round-trip (Β΅s–ms under load)direct memory read (sub-Β΅s)
Consistencyatomic per message β€” easy invariantsper-operation atomic only; races possible across ops
Lifetimedies with processdies with owner process (transferable via heir)
Use forstate with invariants & logichot read-mostly data, caches, counters

The classic pattern: a GenServer owns the table and handles writes/invariants; readers hit ETS directly. Because the table can outlive GenServer crashes (with a heir or a separate owner), this also solves Book 2's "restart loses state" problem. Look up :ets.update_counter/3 for atomic increments (rate limiters love it).

Which tool? Interactive picker

🧭 State & concurrency decision helper

What do you need?
Pick an option above…

5. Applications β€” how it all boots

An OTP application is the unit of packaging: your code + its supervision tree + its dependencies. Every Mix project has one; deps like Phoenix and Ecto are applications too, each booting its own tree. The entry point:

# mix.exs                                    # lib/my_app/application.ex
def application do                            defmodule MyApp.Application do
  [                                             use Application
    mod: {MyApp.Application, []},
    extra_applications: [:logger]               def start(_type, _args) do
  ]                                               children = [
end                                                 MyApp.Repo,
                                                    {Registry, keys: :unique, name: MyApp.Registry},
                                                    {DynamicSupervisor, name: MyApp.SessionSup},
                                                    MyAppWeb.Endpoint
                                                  ]
                                                  Supervisor.start_link(children,
                                                    strategy: :one_for_one, name: MyApp.Supervisor)
                                                end
                                              end

When you run iex -S mix or deploy, the VM starts each application, which starts its root supervisor, which starts its children in order. Your whole system's startup is a declarative tree, not a script. This is the file to read first in any unfamiliar Elixir codebase.

QUIZ 3.2

Your app's children: [Repo, Registry, SessionSup, Endpoint] with :one_for_one. The database goes down and Repo can't reconnect, crashing repeatedly. What ultimately happens?
Restart intensity applies at every level. Repo failing repeatedly kills the root supervisor, which stops the application β€” and typically the VM. That's by design: a node that can't reach its database is better restarted (or replaced by your orchestrator) than half-alive. (Ecto's pool does retry internally β€” but if it truly dies repeatedly, escalation is the behavior you want.)

6. Releases β€” shipping it

mix release builds a self-contained directory: the BEAM VM, Erlang/Elixir, your compiled apps, and boot scripts. No Elixir installation needed on the target machine β€” copy, run bin/my_app start. Configuration at boot comes from config/runtime.exs (reads env vars β€” your 12-factor entry point). Inspect a live node with bin/my_app remote β€” a REPL inside production, one of the BEAM's killer debugging features.

Deep dive: hot code upgrades β€” the famous feature you shouldn't use

The BEAM can swap a module's code while processes run β€” this is how telecom switches hit nine nines. But it requires careful state migration callbacks and is rarely worth it for web apps; nearly everyone does rolling restarts behind a load balancer instead. Know it exists; reach for it approximately never.

Recap

Task for parallelism, Agent for trivial state, Registry to name dynamic processes, DynamicSupervisor to spawn them, ETS when a single mailbox is too slow, Application to declare the boot tree, releases to ship it. You now have the complete OTP vocabulary β€” everything from here on (Phoenix included) is these tools arranged nicely.