Call topology cost

/tools/call-topology-cost/

Pick a call size and see where a voice call's cost lands for mesh, an SFU, an MCU and a server that is the other peer, with real packet sizes, and what would make it cheaper.

How it works

Call topology cost#

What it does#

Add people to a voice call and someone has to copy each voice for every listener. Where that copy is made decides what the call costs, and who pays:

Pattern	Who makes the copies	What grows with the number of people, N
Mesh	Every client, for every other client	Each client sends N-1 streams and receives N-1
SFU (selective forwarding)	A server that forwards packets without decoding them	The server sends N(N-1) streams, or K(N-1) if it forwards only the K loudest
MCU (mixer)	A server that decodes everyone, mixes, and encodes one mix per listener	N decodes, N mixes and N encodes on the server, every frame
Server is the peer	Nobody: one user talks to one service	One stream each way, per user

Pick the number of participants and the tool compares all four side by side: each client’s upload and download, the server’s traffic in and out, and the server’s operations every frame. The pattern you pick is drawn as a picture of its streams, and gets its own list of changes that would make it cheaper.

How to use it: an example#

Say you are planning a ten-person stand-up on an SFU, and want the server’s bandwidth with the packets a browser really sends, not the bare 120 B.

1. Enter it.

Input	You enter	Why
Participants	`10`	Everyone on the call
Draw the traffic for	SFU	The pattern you are planning
Codec	Opus at 32 kb/s, 20 ms frames	The default voice stream
Packet	SRTP and post 4’s two extensions, over IPv4	Every WebRTC packet is encrypted, and post 4’s offer asks for the audio level and the MID

2. Press Run, and read the comparison. All four patterns, in kb/s. The client columns are one device; the server columns are the whole call.

Pattern	Client up	Client down	Server in	Server out
Mesh	496.8	496.8	none	none
SFU (picked)	55.2	496.8	552	4968
MCU	55.2	55.2	552	552
Peer, per user	55.2	55.2	55.2	55.2

3. Read the picture. The SFU you picked, as streams: 10 clients --10 x 55.2 kb/s--> SFU --90 x 55.2 kb/s--> 10 clients, the SFU forwarding without decoding and each listener hearing 9 voices.

4. Read what would make it cheaper.

Change	What it saves	What it costs
Forward only the 3 loudest	The server’s output falls from 4968 to 1490.4 kb/s, each listener’s from 496.8 to 165.6.	Quieter voices are not heard while three others are louder. Senders must write the audio level.
Send 40 ms frames	Each stream falls from 55.2 to 43.6 kb/s, because the headers are sent half as often.	20 ms more delay, and each lost packet takes 40 ms of sound.

In plain words: the server’s 5 Mb/s comes from copying every voice to nine listeners. Forwarding only the three loudest cuts it by more than two thirds, and a speaker picker needs nothing but the one byte of loudness each sender already writes.

5. Read the server’s work and the packet. Server operations every 20 ms frame puts them in named columns: the SFU does 10 decrypts and 90 encrypts; the MCU does 10 of each. One stream adds that the SFU sends 4500 packets a second. One packet, at the bottom, itemises the 138 B: 80 B of sound, 12 B RTP, 8 B UDP, 20 B IPv4, 10 B SRTP tag, 8 B of header extensions. Headers are 58 B of every 138, which is why longer frames help. Pick AES-GCM and four extensions, over IPv6 and the same packet is 172 B, 92 of them headers, and the server sends 6192 kb/s.

How it counts#

One packet is the codec payload, bitrate x frame length / 8 (80 B at 32 kb/s in 20 ms frames), plus 12 B of RTP, 8 B of UDP and 20 B of IPv4 or 40 B of IPv6 (RFC 8200’s fixed header, with no extension headers). The SRTP tag is 10 B for AES_CM_128_HMAC_SHA1_80 (RFC 3711’s default 80-bit tag) and 16 B for AEAD_AES_128_GCM (RFC 7714).
Header extensions are in RFC 8285’s one-byte form: a 4-byte header (BE DE and a length), then one byte of ID and length per extension plus its data, padded to a multiple of 4. The data is 1 byte for the audio level (RFC 6464), 1 for a MID of 0 (RFC 8843), 2 for a transport-wide sequence number and 3 for abs-send-time. So the audio level alone is 8 B, adding the MID still 8, adding the sequence number 12, and adding abs-send-time 16.
The presets fill those in. Codec sets the bitrate and frame length, with G.711 at 64 kb/s (RFC 3551). Packet sets the IP version, the tag and the extensions. Change any of them and the preset shows Custom.
An SFU decrypts every incoming packet with its sender’s keys and encrypts each copy with its receiver’s, so it does one decrypt per participant and one encrypt per copy sent, and never decodes (RFC 7667 section 3.7). With the loudest K forwarded, each of the K speakers receives the other K-1, and everyone else receives K: K(N-1) in all.
An MCU decodes every stream and builds one mix per listener, leaving out the listener’s own voice, “as this would result in an echo” (RFC 7667 section 3.6.1). With two people a “mix” is one voice, so it counts no mixing.
The server that is the other peer decrypts and decodes what the user says, and encodes and encrypts its reply.
What would make it cheaper is computed from the same counts: the three loudest instead of everyone, or the next longer frame. Switching pattern is not listed, because the comparison already prices every switch. Sorted by what each takes off the busiest flow: the server’s output, or a mesh device’s upload.
The SRTP tag left out changes only the packet size, as post 1 counts it. The decrypt and encrypt counts stay, because WebRTC always encrypts.

What it assumes#

Everyone talks all the time. No silence suppression (DTX), so these are the busiest case.
Mesh runs one encoder per connection, as RFC 7667 section 3.4 draws it. A client that shares one encoder across its connections does one encode instead of N-1, but still encrypts N-1 times with N-1 keys.
Not modelled: video, simulcast, RTCP, link-layer headers such as Ethernet’s, the CPU each operation costs (that depends on the machine and the codec mode), and anything a server does besides media.

References#

RFC 7667: RTP topologies, including mesh (section 3.4), the mixer (3.6.1) and the selective forwarding middlebox (3.7).
RFC 6464: the audio level a client writes in its header.
RFC 8285: the one-byte header extension form.
RFC 3711 and RFC 7714: SRTP’s 80-bit HMAC tag and AES-GCM’s 16-byte tag.
RFC 8200: the 40-byte IPv6 header.
RFC 8843: the MID header extension.
Transport-wide congestion control: the 16-bit transport-wide sequence number.
abs-send-time: a 3-byte send timestamp.
RFC 3551: G.711 at 64 kb/s.

Source and references

index.ts305 lines, TypeScript

1
/**
2
 * Call topology cost: where the work of a voice call lands for mesh, an SFU, an MCU, and a server that is simply the
3
 * other end of the call.
4
 *
5
 * The arithmetic is post 6's demo, `topology_cost.py`, and the fixtures are its output. Every number is a count of
6
 * streams times one stream's rate, so nothing here is measured and nothing is estimated:
7
 *
8
 *  - one packet is the codec payload (bitrate x frame / 8), plus 12 B of RTP (RFC 3550), 8 of UDP and 20 of IPv4 or
9
 *    40 of IPv6 (RFC 8200's fixed header), plus the SRTP authentication tag (10 B for AES_CM_128_HMAC_SHA1_80, RFC
10
 *    3711; 16 B for AEAD_AES_128_GCM, RFC 7714) and any RTP header extensions, in RFC 8285's one-byte form;
11
 *  - mesh sends every voice N-1 times from the client; an SFU receives N and sends K(N-1), K being the streams it
12
 *    forwards; an MCU receives N and sends N mixes, each leaving out its listener's own voice (RFC 7667 section
13
 *    3.6.1); the server that is the other peer is one stream each way, per user.
14
 *
15
 * Assumptions, each a choice the README states: everyone talks all the time (no DTX), mesh runs one encoder per
16
 * connection as RFC 7667 section 3.4 draws it, and the peer server talks back while the user talks.
17
 */
18
import type { Cell, Output, Tool } from "@toolbench/sdk";
19

20
type Input = {
21
  participants: number;
22
  pattern: string;
23
  loudest: number;
24
  codec?: string;
25
  bitrate: number;
26
  frame: string;
27
  packet?: string;
28
  ip?: string;
29
  srtp: string;
30
  ext?: string;
31
};
32

33
class InputError extends Error {
34
  readonly input: string;
35
  constructor(message: string, input: string) {
36
    super(message);
37
    this.input = input;
38
  }
39
}
40

41
const RTP_B = 12;
42
const UDP_B = 8;
43
const IP_B: Record<string, number> = { "4": 20, "6": 40 };
44

45
/**
46
 * Header extensions, each as the bytes of data its element carries. In RFC 8285's one-byte form every element is one
47
 * byte of ID and length plus its data, after a 4-byte header (0xBEDE and a length), padded to a multiple of 4 bytes:
48
 * the audio level is 1 byte of data (RFC 6464), the MID "0" 1, a transport-wide sequence number 2, abs-send-time 3.
49
 */
50
const EXTENSIONS: Record<string, { label: string; data: number[] }> = {
51
  none: { label: "no header extensions", data: [] },
52
  level: { label: "the audio level", data: [1] },
53
  "level-mid": { label: "the audio level and the MID", data: [1, 1] },
54
  "level-mid-twcc": { label: "the audio level, the MID and a transport-wide sequence number", data: [1, 1, 2] },
55
  "level-mid-twcc-abs": { label: "the audio level, the MID, a transport-wide sequence number and abs-send-time", data: [1, 1, 2, 3] },
56
};
57

58
/** Bytes on the wire for a set of one-byte-form extensions: none at all, or the 4-byte header, the elements, and padding. */
59
export function extensionBytes(data: number[]): number {
60
  if (data.length === 0) return 0;
61
  const raw = 4 + data.reduce((sum, d) => sum + 1 + d, 0);
62
  return Math.ceil(raw / 4) * 4;
63
}
64

65
type Ops = { unprotect: number; decode: number; mix: number; encode: number; protect: number };
66
type Row = { key: string; name: string; up: number; down: number; serverIn: number | null; serverOut: number | null; server: Ops | null; grows: string };
67

68
/** kb/s with no trailing noise: 48, 55.2, 4320. */
69
const kb = (x: number) => String(Math.round(x * 100) / 100);
70
const plural = (n: number, one: string, many = `${one}s`) => `${n} ${n === 1 ? one : many}`;
71

72
export function rows(n: number, k: number): Row[] {
73
  const fwd = Math.min(k, n - 1);
74
  return [
75
    { key: "mesh", name: "Mesh", up: n - 1, down: n - 1, serverIn: null, serverOut: null, server: null, grows: "N-1 streams up from every client" },
76
    {
77
      key: "sfu",
78
      name: k < n ? `SFU, ${k} loudest` : "SFU",
79
      up: 1,
80
      down: fwd,
81
      serverIn: n,
82
      // Each of the K forwarded speakers gets the other K-1, everyone else gets K: K(K-1) + (N-K)K = K(N-1).
83
      serverOut: k * (n - 1),
84
      server: { unprotect: n, decode: 0, mix: 0, encode: 0, protect: k * (n - 1) },
85
      grows: k < n ? "K(N-1) streams out of the server" : "N(N-1) streams out of the server",
86
    },
87
    { key: "mcu", name: "MCU", up: 1, down: 1, serverIn: n, serverOut: n, server: { unprotect: n, decode: n, mix: n > 2 ? n : 0, encode: n, protect: n }, grows: "N decodes, mixes and encodes on the server" },
88
    { key: "peer", name: "Server is the peer", up: 1, down: 1, serverIn: 1, serverOut: 1, server: { unprotect: 1, decode: 1, mix: 0, encode: 1, protect: 1 }, grows: "one stream each way, per user" },
89
  ];
90
}
91

92
export default {
93
  run(input: Input) {
94
    try {
95
      const n = Number(input.participants);
96
      if (!Number.isInteger(n) || n < 2) throw new InputError("A call needs at least 2 participants.", "participants");
97
      if (n > 50) throw new InputError("At most 50 participants.", "participants");
98
      const bitrate = Number(input.bitrate);
99
      if (!Number.isInteger(bitrate) || bitrate < 6 || bitrate > 510)
100
        throw new InputError("The bitrate must be 6 to 510 kb/s, Opus's range (RFC 6716 section 2.1.1). G.711 is 64.", "bitrate");
101
      const frame = Number(input.frame);
102
      if (![10, 20, 40, 60].includes(frame)) throw new InputError("Frames are 10, 20, 40 or 60 ms.", "frame");
103
      if ((bitrate * frame) % 8 !== 0) throw new InputError("bitrate x frame length must be a whole number of bytes.", "bitrate");
104
      const loudest = Number(input.loudest ?? 0);
105
      if (!Number.isInteger(loudest) || loudest < 0) throw new InputError("The number of loudest streams is a whole number, 0 for all.", "loudest");
106
      if (loudest > n) throw new InputError("The loudest streams can be at most the number of participants.", "loudest");
107
      const ipB = IP_B[input.ip ?? "4"];
108
      if (ipB === undefined) throw new InputError("IP is version 4 or 6.", "ip");
109
      const ext = EXTENSIONS[input.ext ?? "none"];
110
      if (!ext) throw new InputError(`Header extensions must be one of ${Object.keys(EXTENSIONS).join(", ")}.`, "ext");
111
      const tag = Number(input.srtp);
112
      const extB = extensionBytes(ext.data);
113

114
      const payload = (bitrate * frame) / 8;
115
      const sizeAt = (ms: number) => (bitrate * ms) / 8 + RTP_B + UDP_B + ipB + tag + extB;
116
      const size = sizeAt(frame);
117
      const pps = 1000 / frame;
118
      const per = (size * 8 * pps) / 1000;
119
      const k = loudest === 0 ? n : loudest;
120
      const all = rows(n, k);
121
      const pick = all.find((r) => r.key === input.pattern) ?? all[1]!;
122
      const none: Cell = { text: "none", tone: "muted" };
123
      const rate = (streams: number | null): Cell => (streams === null ? none : { text: kb(streams * per), mono: true });
124

125
      /*
126
       * The answer is the comparison: the question is which pattern, so all four go first, in one frame. Client
127
       * columns are one device and server columns the whole call, which the caption says because the two scopes
128
       * sit side by side. The pattern the reader picked is named in words, not coloured: picking it is not a
129
       * judgement about it.
130
       */
131
      const comparison: Output = {
132
        kind: "table",
133
        role: "comparison",
134
        caption: `${plural(n, "person", "people")}, everyone talking${k < n ? `, the SFU forwarding the ${k} loudest` : ""}. In kb/s: the client columns are one device, the server columns the whole call`,
135
        columns: [
136
          { label: "Pattern", wrap: "prose", rowHeader: true },
137
          { label: "Client up", align: "end", mono: true, wrap: "none" },
138
          { label: "Client down", align: "end", mono: true, wrap: "none" },
139
          { label: "Server in", align: "end", mono: true, wrap: "none" },
140
          { label: "Server out", align: "end", mono: true, wrap: "none" },
141
        ],
142
        rows: all.map((r): Cell[] => [
143
          `${r.key === "peer" ? "Peer, per user" : r.name}${r.key === pick.key ? " (picked)" : ""}`,
144
          { text: kb(r.up * per), mono: true },
145
          { text: kb(r.down * per), mono: true },
146
          rate(r.serverIn),
147
          rate(r.serverOut),
148
        ]),
149
      };
150

151
      /*
152
       * The picked pattern's streams, as counts of one stream's rate. Nodes are aggregates (all the clients, not
153
       * each one), so the picture stays three boxes at 50 people; the per-device figures are in the table above.
154
       */
155
      const each = (count: number) => `${count} x ${kb(per)} kb/s`;
156
      const graph: Output = (() => {
157
        if (pick.key === "mesh") {
158
          return {
159
            kind: "graph",
160
            graph: {
161
              label: `Mesh, ${n} people: every device sends to every other`,
162
              spacing: "compact",
163
              nodes: [
164
                { id: "talk", label: plural(n, "client"), note: `each sends ${plural(n - 1, "stream")}` },
165
                { id: "hear", label: plural(n, "client"), note: `each receives ${n - 1}` },
166
              ],
167
              edges: [{ from: "talk", to: "hear", label: each(n * (n - 1)) }],
168
            },
169
          };
170
        }
171
        if (pick.key === "peer") {
172
          return {
173
            kind: "graph",
174
            graph: {
175
              label: "The server is the other end: one stream each way, per user",
176
              spacing: "compact",
177
              nodes: [
178
                { id: "user", label: "1 user" },
179
                { id: "server", label: "Server", note: "decodes, then encodes its reply" },
180
              ],
181
              edges: [
182
                { from: "user", to: "server", label: each(1) },
183
                { from: "server", to: "user", label: each(1) },
184
              ],
185
            },
186
          };
187
        }
188
        const server = pick.key === "sfu" ? "SFU" : "MCU";
189
        return {
190
          kind: "graph",
191
          graph: {
192
            label: `${pick.name}, ${n} people: every stream in and out of the server`,
193
            spacing: "compact",
194
            nodes: [
195
              { id: "talk", label: plural(n, "client"), note: "talking" },
196
              { id: "server", label: server, note: pick.key === "sfu" ? "forwards without decoding" : n > 2 ? "one mix per listener" : "decodes and re-encodes" },
197
              { id: "hear", label: plural(n, "client"), note: pick.key === "mcu" ? (n > 2 ? "each hears one mix" : "each hears 1 voice") : `each hears ${plural(pick.down, "voice")}` },
198
            ],
199
            edges: [
200
              { from: "talk", to: "server", label: each(pick.serverIn!) },
201
              { from: "server", to: "hear", label: pick.key === "mcu" && pick.server!.mix > 0 ? `${n} mixes x ${kb(per)} kb/s` : each(pick.serverOut!) },
202
            ],
203
          },
204
        };
205
      })();
206

207
      /*
208
       * What would make the picked pattern cheaper, each computed rather than asserted. Switching pattern is not
209
       * here: the comparison above already prices every switch. Sorted by the kb/s each takes off the busiest
210
       * flow, the server's output or a mesh device's upload, largest first.
211
       */
212
      type Advice = { change: string; saves: number; gain: string; cost: string };
213
      const advice: Advice[] = [];
214
      if (pick.key === "sfu" && k === n && n > 3) {
215
        const top = rows(n, 3)[1]!;
216
        advice.push({
217
          change: "Forward only the 3 loudest",
218
          saves: (pick.serverOut! - top.serverOut!) * per,
219
          gain: `The server's output falls from ${kb(pick.serverOut! * per)} to ${kb(top.serverOut! * per)} kb/s, each listener's from ${kb(pick.down * per)} to ${kb(top.down * per)}.`,
220
          cost: "Quieter voices are not heard while three others are louder. Senders must write the audio level.",
221
        });
222
      }
223
      if (frame < 60) {
224
        const longer = frame === 10 ? 20 : frame === 20 ? 40 : 60;
225
        const perLonger = (sizeAt(longer) * 8 * (1000 / longer)) / 1000;
226
        const streams = pick.serverOut ?? pick.up;
227
        advice.push({
228
          change: `Send ${longer} ms frames`,
229
          saves: (per - perLonger) * streams,
230
          gain: `Each stream falls from ${kb(per)} to ${kb(perLonger)} kb/s, because the headers are sent ${longer === 2 * frame ? "half" : "two thirds"} as often.`,
231
          cost: `${longer - frame} ms more delay, and each lost packet takes ${longer} ms of sound.`,
232
        });
233
      }
234
      advice.sort((x, y) => y.saves - x.saves);
235

236
      /*
237
       * Server work, with the five operations as named columns rather than "3 / 0 / 0 / 0 / 6". Mesh has no
238
       * server, so it has no row. With the tag left out of the size, the crypto counts still stand: WebRTC always
239
       * encrypts, and the choice only changes what one packet weighs.
240
       */
241
      const servers = all.filter((r) => r.server !== null);
242
      const work: Output = {
243
        kind: "table",
244
        role: "comparison",
245
        caption: `Server operations every ${frame} ms frame${tag === 0 ? ". Decrypts and encrypts assume SRTP, which WebRTC always uses, though the packet size here leaves out its tag" : ""}`,
246
        columns: [
247
          { label: "Pattern", wrap: "prose", rowHeader: true },
248
          { label: "Decrypt", align: "end", mono: true, wrap: "none" },
249
          { label: "Decode", align: "end", mono: true, wrap: "none" },
250
          { label: "Mix", align: "end", mono: true, wrap: "none" },
251
          { label: "Encode", align: "end", mono: true, wrap: "none" },
252
          { label: "Encrypt", align: "end", mono: true, wrap: "none" },
253
        ],
254
        rows: servers.map((r): Cell[] => [
255
          r.key === "peer" ? "Peer, per user" : r.name,
256
          { text: String(r.server!.unprotect), mono: true },
257
          { text: String(r.server!.decode), mono: true },
258
          { text: String(r.server!.mix), mono: true },
259
          { text: String(r.server!.encode), mono: true },
260
          { text: String(r.server!.protect), mono: true },
261
        ]),
262
      };
263

264
      const breakdown = [
265
        `${payload} B of sound`,
266
        `${RTP_B} B RTP`,
267
        `${UDP_B} B UDP`,
268
        `${ipB} B IPv${input.ip ?? "4"}`,
269
        ...(tag ? [`${tag} B SRTP tag`] : []),
270
        ...(extB ? [`${extB} B of header extensions, for ${ext.label}`] : []),
271
      ];
272
      const method: Output = {
273
        kind: "fields",
274
        role: "method",
275
        fields: [
276
          { label: "One packet", value: `${size} B`, note: `${breakdown.join(", ")}.` },
277
          { label: "One stream", value: `${kb(per)} kb/s`, note: `${pps} packets a second, so the server sends ${pick.serverOut === null ? "none" : kb(pick.serverOut * pps)} a second for the ${pick.key === "peer" ? "peer server, per user" : pick.name}.` },
278
          { label: "Assumes", value: "Everyone talks all the time, with no silence suppression, so these are the busiest case.", mono: false },
279
        ],
280
      };
281

282
      const parts: Output[] = [
283
        comparison,
284
        graph,
285
        ...(advice.length > 0
286
          ? [
287
              {
288
                kind: "table" as const,
289
                role: "actions",
290
                caption: `What would make the ${pick.key === "peer" ? "peer server" : pick.name} cheaper, largest saving first`,
291
                columns: [{ label: "Change" }, { label: "What it saves" }, { label: "What it costs" }],
292
                rows: advice.map((a): Cell[] => [a.change, a.gain, a.cost]),
293
              },
294
            ]
295
          : []),
296
        work,
297
        method,
298
      ];
299
      return { kind: "group", parts };
300
    } catch (error) {
301
      if (error instanceof InputError) return { kind: "error", message: error.message, input: error.input };
302
      throw error;
303
    }
304
  },
305
} satisfies Tool<Input>;

2 items

Oct 15, 2026 27 min read Networking & media

Where the server sits

Add Bob and Carol to Alice's call and someone has to copy her voice twice. Mesh, an SFU and an MCU put that copy in three different places, and the cost moves with it: upload, server bandwidth, or a decode and an encode per person. Then back to Alice's assistant, where the server is simply the other end of the call.

Real-time voice, from the packet upwebrtcarchitecturenetworking
6 posts Series

Real-time voice, from the packet up

How a voice call actually works, one layer at a time.

Networking & media

Where the server sits

Add Bob and Carol to Alice's call and someone has to copy her voice twice. Mesh, an SFU and an MCU put that copy in three different places, and the cost moves with it: upload, server bandwidth, or a decode and an encode per person. Then back to Alice's assistant, where the server is simply the other end of the call.

webrtcarchitecture
Oct 2026
Series

Real-time voice, from the packet up

How a voice call actually works, one layer at a time.

6 posts