Every framework is sent the same 65 tests in 19 families. A test is one request and the checks its answer has to pass. A framework is measured only after it passes all 62 performance tests.
Most tests are read against another test, their base, and differ from it in one thing. Open a family for each test's request, the answer it expects, and what the difference from its base measures.
Each performance test has a heft from 1 to 5, for how much work it asks of a framework. The load sends a heavier test less often, in one fixed order for every framework. The tests README gives the scale and the order.
2 tests
The framework's own authorization mechanism, with the crypto left out.
allowed · denied
A bearer token the framework's own authorization mechanism has to check before the handler runs. Read against json.small, the difference is the check and the plumbing that carries it, not the crypto, which this family leaves out.
GET /authorized/small
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { TOKEN } from "#models/configuration";
import { items } from "#payloads";
const path = "/authorized/small";
export default performanceTest({
id: { family: "authorized", name: "allowed" },
path,
base: "json.small",
varies: "authorization",
heft: 1,
about:
"A bearer token the framework's own authorization mechanism has to check " +
"before the handler runs. Read against json.small, the difference is the " +
"check and the plumbing that carries it, not the crypto, which this " +
"family leaves out.",
request: (c) => c.get(path).header("authorization", `Bearer ${TOKEN}`).okWith(items.small),
});
The same endpoint refusing. The token differs from the accepted one by its last character, so the comparison walks the whole string and this row measures the refusal path rather than a length check.
GET /authorized/small
No body.
HTTP 403
The body is not checked.
import { performanceTest } from "#kit";
import { TOKEN } from "#models/configuration";
const path = "/authorized/small";
/** The token with its last character changed, so a refusal compares the whole string. */
const wrong = `${TOKEN.slice(0, -1)}0`;
export default performanceTest({
id: { family: "authorized", name: "denied" },
path,
base: "authorized.allowed",
varies: "outcome",
heft: 2,
about:
"The same endpoint refusing. The token differs from the accepted one by " +
"its last character, so the comparison walks the whole string and this " +
"row measures the refusal path rather than a length check.",
request: (c) => c.get(path).header("authorization", `Bearer ${wrong}`).status(403),
});
1 test
The dispatch floor, with nothing serialised.
plaintext
The dispatch floor. A fixed string out, with no serialiser in the way, so what is left is the framework accepting a connection, matching a route and writing a response. Every other row in the corpus is read against this one.
GET /plaintext
No body.
HTTP 200
/^text\/plain/payload Hello, World!
Hello, World!
Compared byte for byte.
import { performanceTest, text } from "#kit";
const path = "/plaintext";
/** Written by the handler as a literal, so it is the one answer with no file in tests/payloads. */
const HELLO = text("Hello, World!", "Hello, World!");
export default performanceTest({
id: { family: "baseline", name: "plaintext" },
path,
heft: 1,
about:
"The dispatch floor. A fixed string out, with no serialiser in the way, " +
"so what is left is the framework accepting a connection, matching a " +
"route and writing a response. Every other row in the corpus is read " +
"against this one.",
request: (c) => c.get(path).okWith(HELLO).hasHeader("content-type", /^text\/plain/),
});
8 tests
The parser and the validator, with size crossed against validation.
bind_small · bind_medium · bind_large · validate_large · validate_medium · validate_small · rejected_all · rejected_first
A refusal's status is written 4XX. Frameworks refuse with different statuses, so each declares its own in its client-exception, and the test reads it from there.
A body parsed and bound without being validated. The answer carries a count of the leaves it found, which is what says the request was parsed rather than piped to the response. This is what the validate row of the same size is subtracted from.
POST /body/bind/small
payload order.small
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1}]}HTTP 200
payload bind.small
{"fields":4,"bytes":66,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1}]}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/bind/small";
export default performanceTest({
id: { family: "body", name: "bind_small" },
path,
heft: 2,
about:
"A body parsed and bound without being validated. The answer carries a " +
"count of the leaves it found, which is what says the request was parsed " +
"rather than piped to the response. This is what the validate row of the " +
"same size is subtracted from.",
request: (c) => c.post(path, order.small.value).okWith(bind.small),
});
A body parsed and bound without being validated. The answer carries a count of the leaves it found, which is what says the request was parsed rather than piped to the response. This is what the validate row of the same size is subtracted from.
POST /body/bind/medium
payload order.medium
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"productId":26,"qty":1},{"productId":27,"q
… 8,379 bytes totalHTTP 200
payload bind.medium
{"fields":674,"bytes":8379,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"product
… 8,414 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/bind/medium";
export default performanceTest({
id: { family: "body", name: "bind_medium" },
path,
base: "body.bind_small",
varies: "size",
heft: 3,
about:
"A body parsed and bound without being validated. The answer carries a " +
"count of the leaves it found, which is what says the request was parsed " +
"rather than piped to the response. This is what the validate row of the " +
"same size is subtracted from.",
request: (c) => c.post(path, order.medium.value).okWith(bind.medium),
});
A body parsed and bound without being validated, at about 40 KB. The answer carries a count of the leaves it found, which is what says the request was parsed rather than piped to the response. Read against body.bind_medium, the difference is the parser and the binder at scale.
POST /body/bind/large
payload order.large
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"productId":26,"qty":1},{"productId":27,"q
… 39,754 bytes totalHTTP 200
payload bind.large
{"fields":3202,"bytes":39754,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"produ
… 39,791 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/bind/large";
export default performanceTest({
id: { family: "body", name: "bind_large" },
path,
base: "body.bind_medium",
varies: "size",
heft: 4,
about:
"A body parsed and bound without being validated, at about 40 KB. The " +
"answer carries a count of the leaves it found, which is what says the " +
"request was parsed rather than piped to the response. Read against " +
"body.bind_medium, the difference is the parser and the binder at scale.",
request: (c) => c.post(path, order.large.value).okWith(bind.large),
});
The same body checked against a schema before the handler sees it, and answered exactly as the bind row answers it. Read against body.bind_large, the difference is the validator alone rather than the validator plus the parse.
POST /body/validate/large
payload order.large
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"productId":26,"qty":1},{"productId":27,"q
… 39,754 bytes totalHTTP 200
payload bind.large
{"fields":3202,"bytes":39754,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"produ
… 39,791 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/validate/large";
export default performanceTest({
id: { family: "body", name: "validate_large" },
path,
base: "body.bind_large",
varies: "validation",
heft: 5,
about:
"The same body checked against a schema before the handler sees it, and " +
"answered exactly as the bind row answers it. Read against " +
"body.bind_large, the difference is the validator alone rather than the " +
"validator plus the parse.",
request: (c) => c.post(path, order.large.value).okWith(bind.large),
});
The same body checked against a schema before the handler sees it, and answered exactly as the bind row answers it. Read against body.bind_medium, the difference is the validator alone rather than the validator plus the parse.
POST /body/validate/medium
payload order.medium
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"productId":26,"qty":1},{"productId":27,"q
… 8,379 bytes totalHTTP 200
payload bind.medium
{"fields":674,"bytes":8379,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1},{"productId":2,"qty":2},{"productId":3,"qty":3},{"productId":4,"qty":4},{"productId":5,"qty":5},{"productId":6,"qty":1},{"productId":7,"qty":2},{"productId":8,"qty":3},{"productId":9,"qty":4},{"productId":10,"qty":5},{"productId":11,"qty":1},{"productId":12,"qty":2},{"productId":13,"qty":3},{"productId":14,"qty":4},{"productId":15,"qty":5},{"productId":16,"qty":1},{"productId":17,"qty":2},{"productId":18,"qty":3},{"productId":19,"qty":4},{"productId":20,"qty":5},{"productId":21,"qty":1},{"productId":22,"qty":2},{"productId":23,"qty":3},{"productId":24,"qty":4},{"productId":25,"qty":5},{"product
… 8,414 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/validate/medium";
export default performanceTest({
id: { family: "body", name: "validate_medium" },
path,
base: "body.bind_medium",
varies: "validation",
heft: 3,
about:
"The same body checked against a schema before the handler sees it, and " +
"answered exactly as the bind row answers it. Read against " +
"body.bind_medium, the difference is the validator alone rather than the " +
"validator plus the parse.",
request: (c) => c.post(path, order.medium.value).okWith(bind.medium),
});
The same body checked against a schema before the handler sees it, and answered exactly as the bind row answers it. Read against body.bind_small, the difference is the validator alone rather than the validator plus the parse.
POST /body/validate/small
payload order.small
{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1}]}HTTP 200
payload bind.small
{"fields":4,"bytes":66,"echo":{"customerId":1,"status":"open","lines":[{"productId":1,"qty":1}]}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { bind, order } from "#payloads";
const path = "/body/validate/small";
export default performanceTest({
id: { family: "body", name: "validate_small" },
path,
base: "body.bind_small",
varies: "validation",
heft: 2,
about:
"The same body checked against a schema before the handler sees it, and " +
"answered exactly as the bind row answers it. Read against " +
"body.bind_small, the difference is the validator alone rather than the " +
"validator plus the parse.",
request: (c) => c.post(path, order.small.value).okWith(bind.small),
});
Three fields wrong in one body. What a rejection looks like is the framework's own contract, so the status and the field paths are read through its exceptions declaration and this test never sees an error body.
POST /body/validate/small
payload order.invalid
{"customerId":0,"status":"","lines":[]}HTTP 4XX
The status the framework declares as rejected in its client-exception.
The framework's own error body, read through its client-exception. It has to name customerId, status and lines, or exactly one of them where the framework declares that it reports only the first error.
import { performanceTest } from "#kit";
import { order } from "#payloads";
const path = "/body/validate/small";
export default performanceTest({
id: { family: "body", name: "rejected_all" },
path,
base: "body.validate_small",
varies: "outcome",
heft: 2,
about:
"Three fields wrong in one body. What a rejection looks like is the " +
"framework's own contract, so the status and the field paths are read " +
"through its exceptions declaration and this test never sees an error " +
"body.",
request: (c) => c.post(path, order.invalid.value).rejected("customerId", "status", "lines"),
});
The same body against a route that stops at the first bad field. Read against body.rejected_all, the difference is the two error contracts: one walks the whole object and one gives up, and the second is doing less work.
POST /body/validate/first-error
payload order.invalid
{"customerId":0,"status":"","lines":[]}HTTP 4XX
The status the framework declares as rejected in its client-exception.
The framework's own error body, read through its client-exception. It has to name customerId.
import { performanceTest } from "#kit";
import { order } from "#payloads";
const path = "/body/validate/first-error";
export default performanceTest({
id: { family: "body", name: "rejected_first" },
path,
base: "body.rejected_all",
varies: "error_contract",
heft: 2,
about:
"The same body against a route that stops at the first bad field. Read " +
"against body.rejected_all, the difference is the two error contracts: " +
"one walks the whole object and one gives up, and the second is doing " +
"less work.",
request: (c) => c.post(path, order.invalid.value).rejected("customerId"),
});
5 tests
The framework's response cache: the handler skipped and a stored answer replayed, keyed by a path segment and by request header, and kept for 30 seconds.
large · medium · small · vary_one · vary_many
The handler skipped and a stored answer written back, under one of four keys the path names. Read against json.large, the difference is the store answering instead of the framework, which is why this row asserts the serial repeated rather than changed.
GET /cache/large/{draw.key}No body.
HTTP 200
payload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/cache/large/{draw.key}";
export default performanceTest({
id: { family: "cache", name: "large" },
path,
base: "json.large",
varies: "response_cache",
heft: 3,
about:
"The handler skipped and a stored answer written back, under one of four " +
"keys the path names. Read against json.large, the difference is the store " +
"answering instead of the framework, which is why this row asserts the " +
"serial repeated rather than changed.",
request: (c) => c.get(`/cache/large/${c.draw.key()}`).okWith(items.large).replayed(),
});
The handler skipped and a stored answer written back, under one of four keys the path names. Read against json.medium, the difference is the store answering instead of the framework, which is why this row asserts the serial repeated rather than changed.
GET /cache/medium/{draw.key}No body.
HTTP 200
payload items.medium
{"size":"medium","count":89,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kitc
… 7,953 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/cache/medium/{draw.key}";
export default performanceTest({
id: { family: "cache", name: "medium" },
path,
base: "json.medium",
varies: "response_cache",
heft: 1,
about:
"The handler skipped and a stored answer written back, under one of four " +
"keys the path names. Read against json.medium, the difference is the store " +
"answering instead of the framework, which is why this row asserts the " +
"serial repeated rather than changed.",
request: (c) => c.get(`/cache/medium/${c.draw.key()}`).okWith(items.medium).replayed(),
});
The handler skipped and a stored answer written back, under one of four keys the path names. Read against json.small, the difference is the store answering instead of the framework, which is why this row asserts the serial repeated rather than changed.
GET /cache/small/{draw.key}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/cache/small/{draw.key}";
export default performanceTest({
id: { family: "cache", name: "small" },
path,
base: "json.small",
varies: "response_cache",
heft: 1,
about:
"The handler skipped and a stored answer written back, under one of four " +
"keys the path names. Read against json.small, the difference is the store " +
"answering instead of the framework, which is why this row asserts the " +
"serial repeated rather than changed.",
request: (c) => c.get(`/cache/small/${c.draw.key()}`).okWith(items.small).replayed(),
});
The stored answer keyed by a request header as well as by the path. One header with two values, picked per instance, so a store that ignores the vary header holds four entries where the plan sent eight keys.
GET /cache/vary/one/{draw.key}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/cache/vary/one/{draw.key}";
/** Two tenants, so a store that ignores the header holds one entry where two keys were sent. */
const tenants = ["alpha", "beta"];
export default performanceTest({
id: { family: "cache", name: "vary_one" },
path,
base: "cache.small",
varies: "cache_key",
heft: 1,
about:
"The stored answer keyed by a request header as well as by the path. One " +
"header with two values, picked per instance, so a store that ignores the " +
"vary header holds four entries where the plan sent eight keys.",
request: (c) =>
c
.get(`/cache/vary/one/${c.draw.key()}`)
.header("x-rb-tenant", c.draw.choice(tenants))
.okWith(items.small)
.replayed(),
});
The same thing keyed on three headers instead of one. Read against cache.vary_one, the difference is thirty-two distinct keys where there were eight, which is what a store has to be sized against rather than what it costs to read.
GET /cache/vary/many/{draw.key}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/cache/vary/many/{draw.key}";
/** Two values of each header, so a store that ignores one holds half the entries it was sent. */
const channels = ["web", "app"];
const regions = ["eu", "us"];
const tenants = ["alpha", "beta"];
export default performanceTest({
id: { family: "cache", name: "vary_many" },
path,
base: "cache.vary_one",
varies: "cache_key",
heft: 1,
about:
"The same thing keyed on three headers instead of one. Read against " +
"cache.vary_one, the difference is thirty-two distinct keys where there were " +
"eight, which is what a store has to be sized against rather than what it " +
"costs to read.",
request: (c) =>
c
.get(`/cache/vary/many/${c.draw.key()}`)
.header("x-rb-channel", c.draw.choice(channels))
.header("x-rb-region", c.draw.choice(regions))
.header("x-rb-tenant", c.draw.choice(tenants))
.okWith(items.small)
.replayed(),
});
4 tests
Outbound gzip: the wiring declining, the wiring working, and the small-body threshold.
identity_large · gzip_large · identity_small · gzip_small
The compression middleware installed and declining. The client asks for identity, so nothing is compressed and what this row carries is the cost of the wiring being in the path at all. It is what the gzip arm is subtracted from.
GET /compressed/large
No body.
HTTP 200
identitypayload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/compressed/large";
export default performanceTest({
id: { family: "compressed", name: "identity_large" },
path,
base: "json.large",
varies: "compression_wiring",
heft: 4,
about:
"The compression middleware installed and declining. The client asks for " +
"identity, so nothing is compressed and what this row carries is the cost " +
"of the wiring being in the path at all. It is what the gzip arm is " +
"subtracted from.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "identity")
.header("cache-control", "no-cache")
.okWith(items.large, { compressed: false })
.fresh(),
});
The middleware compressing a body large enough to be worth it. Read against compressed.identity_large, the difference is the compression and nothing else, because both rows carry the same wiring.
GET /compressed/large
No body.
HTTP 200
gzippayload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/compressed/large";
export default performanceTest({
id: { family: "compressed", name: "gzip_large" },
path,
base: "compressed.identity_large",
varies: "compression",
heft: 5,
about:
"The middleware compressing a body large enough to be worth it. Read " +
"against compressed.identity_large, the difference is the compression and " +
"nothing else, because both rows carry the same wiring.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "gzip")
.header("cache-control", "no-cache")
.okWith(items.large, { compressed: true })
.fresh(),
});
The compression middleware installed and declining. The client asks for identity, so nothing is compressed and what this row carries is the cost of the wiring being in the path at all. It is what the gzip arm is subtracted from.
GET /compressed/small
No body.
HTTP 200
identitypayload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/compressed/small";
export default performanceTest({
id: { family: "compressed", name: "identity_small" },
path,
base: "json.small",
varies: "compression_wiring",
heft: 1,
about:
"The compression middleware installed and declining. The client asks for " +
"identity, so nothing is compressed and what this row carries is the cost " +
"of the wiring being in the path at all. It is what the gzip arm is " +
"subtracted from.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "identity")
.header("cache-control", "no-cache")
.okWith(items.small, { compressed: false })
.fresh(),
});
The middleware actually asked to compress, on a body too small to benefit. Whether a framework bothers is the point, so this row does not assert that the answer came back compressed; it asserts the answer is right either way.
GET /compressed/small
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/compressed/small";
export default performanceTest({
id: { family: "compressed", name: "gzip_small" },
path,
base: "compressed.identity_small",
varies: "compression",
heft: 2,
about:
"The middleware actually asked to compress, on a body too small to " +
"benefit. Whether a framework bothers is the point, so this row does not " +
"assert that the answer came back compressed; it asserts the answer is " +
"right either way.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "gzip")
.header("cache-control", "no-cache")
.okWith(items.small)
.fresh(),
});
5 tests, 3 not measured
The framework's own CORS feature, attached to /cors with the one policy every framework configures itself: a preflight the feature answers alone, and the real request it lets through.
disallowed · preflight · request · scoped · vary
A value written {run.name} is drawn once per run and never given to the framework, so it cannot answer from a table.
A preflight from an origin the policy does not name gets no access-control-allow-origin, which is what stops the browser. The status is not checked, because frameworks differ on it.
OPTIONS /cors/small
No body.
Any status
The body is not checked.
import { validationTest } from "#kit";
import { CORS } from "#models/configuration";
const path = "/cors/small";
const cors = CORS;
export default validationTest({
id: { family: "cors", name: "disallowed" },
path,
about:
"A preflight from an origin the policy does not name gets no " +
"access-control-allow-origin, which is what stops the browser. The status " +
"is not checked, because frameworks differ on it.",
request: (c) =>
c
.options(path)
.header("origin", "https://elsewhere.example.net")
.header("access-control-request-method", cors.method)
.header("access-control-request-headers", cors.header)
.noHeader("access-control-allow-origin"),
});
The question a browser asks before a cross-origin request with a custom header, answered by the CORS feature before any handler runs. The handler on this route writes x-rb-serial, so its absence shows the feature answered alone. 200 and 204 are both accepted, because the Fetch standard takes any 2xx and frameworks split. Read against baseline.plaintext, the difference is the policy being matched.
OPTIONS /cors/small
No body.
HTTP 200 or 204
https://shop.example.com/(^|,)\s*x-rb-tenant\s*(,|$)/i600The body is not checked.
import { performanceTest } from "#kit";
import { CORS } from "#models/configuration";
const path = "/cors/small";
const cors = CORS;
/** The header among any others the framework lists, in any case. */
const LISTS = new RegExp(`(^|,)\\s*${cors.header}\\s*(,|$)`, "i");
export default performanceTest({
id: { family: "cors", name: "preflight" },
path,
base: "baseline.plaintext",
varies: "preflight",
heft: 1,
about:
"The question a browser asks before a cross-origin request with a custom " +
"header, answered by the CORS feature before any handler runs. The " +
"handler on this route writes x-rb-serial, so its absence shows the " +
"feature answered alone. 200 and 204 are both accepted, because the Fetch " +
"standard takes any 2xx and frameworks split. Read against " +
"baseline.plaintext, the difference is the policy being matched.",
request: (c) =>
c
.options(path)
.header("origin", cors.origin)
.header("access-control-request-method", cors.method)
.header("access-control-request-headers", cors.header)
.status(200, 204)
.hasHeader("access-control-allow-origin", cors.origin)
.hasHeader("access-control-allow-headers", LISTS)
.hasHeader("access-control-max-age", String(cors.maxAgeSeconds))
.noHeader("x-rb-serial"),
});
The cross-origin request itself, with the origin and the custom header the preflight asked about. The feature adds its header and lets the request through to the handler. Read against json.small, the difference is the policy checked on a request that passes it.
GET /cors/small
No body.
HTTP 200
https://shop.example.compayload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { CORS } from "#models/configuration";
import { items } from "#payloads";
const path = "/cors/small";
const cors = CORS;
export default performanceTest({
id: { family: "cors", name: "request" },
path,
base: "json.small",
varies: "cors",
heft: 1,
about:
"The cross-origin request itself, with the origin and the custom header " +
"the preflight asked about. The feature adds its header and lets the " +
"request through to the handler. Read against json.small, the difference " +
"is the policy checked on a request that passes it.",
request: (c) =>
c
.get(path)
.header("origin", cors.origin)
.header(cors.header, c.run.tenant)
.okWith(items.small)
.hasHeader("access-control-allow-origin", cors.origin)
.fresh(),
});
The allowed origin asking a route outside /cors gets no access-control-allow-origin, because the policy is attached to /cors and nowhere else. A framework whose CORS feature can only cover the whole application cannot pass this, and says why in the skips of its rb.json.
GET /json/small
No body.
HTTP 200
The body is not checked.
import { validationTest } from "#kit";
import { CORS } from "#models/configuration";
const path = "/json/small";
const cors = CORS;
export default validationTest({
id: { family: "cors", name: "scoped" },
path,
about:
"The allowed origin asking a route outside /cors gets no " +
"access-control-allow-origin, because the policy is attached to /cors and " +
"nowhere else. A framework whose CORS feature can only cover the whole " +
"application cannot pass this, and says why in the skips of its rb.json.",
request: (c) => c.get(path).header("origin", cors.origin).ok().noHeader("access-control-allow-origin"),
});
The real response carries Vary: Origin. A policy that names its origin answers differently per origin, so a cache in front of the framework has to key on it.
GET /cors/small
No body.
HTTP 200
/(^|,)\s*origin\s*(,|$)/iThe body is not checked.
import { validationTest } from "#kit";
import { CORS } from "#models/configuration";
const path = "/cors/small";
const cors = CORS;
export default validationTest({
id: { family: "cors", name: "vary" },
path,
about:
"The real response carries Vary: Origin. A policy that names its origin " +
"answers differently per origin, so a cache in front of the framework has " +
"to key on it.",
request: (c) =>
c
.get(path)
.header("origin", cors.origin)
.header(cors.header, c.run.tenant)
.ok()
.hasHeader("vary", /(^|,)\s*origin\s*(,|$)/i),
});
4 tests
A router miss, a handler's miss, a method the route does not have, and a parser failure.
malformed · not_found · unmatched · wrong_method
{draw.item} is a row of items.large, picked per request. A refusal's status is written 4XX. Frameworks refuse with different statuses, so each declares its own in its client-exception, and the test reads it from there.
A body that is not JSON at all. Read against body.rejected_all, the difference is the parser failing rather than the validator refusing, which is often not even the same status inside one framework.
POST /body/validate/small
{"customerId": 1, "lines": [HTTP 4XX
The status the framework declares as malformed in its client-exception.
The framework's own error body, which is not checked.
import { performanceTest } from "#kit";
const path = "/body/validate/small";
/** Truncated mid-array, so the parser fails before any field is ever looked at. */
const MALFORMED = '{"customerId": 1, "lines": [';
export default performanceTest({
id: { family: "errors", name: "malformed" },
path,
base: "body.rejected_all",
varies: "parse_failure",
heft: 2,
about:
"A body that is not JSON at all. Read against body.rejected_all, the " +
"difference is the parser failing rather than the validator refusing, " +
"which is often not even the same status inside one framework.",
request: (c) => c.post(path).raw(MALFORMED).unparseable(),
});
A lookup the router matches and the handler refuses, because no row has that id. Read against items.read, the difference is the refusal in place of a row, and against errors.unmatched it is the handler deciding rather than the router missing.
GET /items/999999
No body.
HTTP 4XX
The status the framework declares as notFound in its client-exception.
The framework's own error body, which is not checked.
import { performanceTest } from "#kit";
const path = "/items/999999";
export default performanceTest({
id: { family: "errors", name: "not_found" },
path,
base: "items.read",
varies: "outcome",
heft: 1,
about:
"A lookup the router matches and the handler refuses, because no row has " +
"that id. Read against items.read, the difference is the refusal in place " +
"of a row, and against errors.unmatched it is the handler deciding rather " +
"than the router missing.",
request: (c) => c.get(path).notFound(),
});
A path no route matches, which is the router's own miss rather than a handler's decision. A framework that walks its whole route table before giving up pays for it here and nowhere else.
GET /errors/unmatched
No body.
HTTP 4XX
The status the framework declares as notFound in its client-exception.
The framework's own error body, which is not checked.
import { performanceTest } from "#kit";
const path = "/errors/unmatched";
export default performanceTest({
id: { family: "errors", name: "unmatched" },
path,
heft: 1,
about:
"A path no route matches, which is the router's own miss rather than a " +
"handler's decision. A framework that walks its whole route table before " +
"giving up pays for it here and nowhere else.",
request: (c) => c.get(path).notFound(),
});
A path with routes, asked with a method none of them has. A router that matches the path first answers 405, and one that matches the method and the path together answers 404, so the status is read from the framework's declaration. Read against errors.unmatched, the difference is how far the router got before it gave up.
POST /items/{draw.item}No body.
HTTP 4XX
The status the framework declares as wrongMethod in its client-exception.
The framework's own error body, which is not checked.
import { performanceTest } from "#kit";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "errors", name: "wrong_method" },
path,
base: "errors.unmatched",
varies: "known_path",
heft: 1,
about:
"A path with routes, asked with a method none of them has. A router that " +
"matches the path first answers 405, and one that matches the method and " +
"the path together answers 404, so the status is read from the " +
"framework's declaration. Read against errors.unmatched, the difference " +
"is how far the router got before it gave up.",
request: (c) => c.post(`/items/${c.draw.item()}`).wrongMethod(),
});
4 tests
The framework's own conditional-request machinery: the validator it computes over the body, and the 304 it answers.
large · match_large · stale_large · small
The framework hashing the body it is about to send and writing the validator it computed onto the response. Read against json.large, the difference is the hash and the header. The tag is the framework's own, so no two of them agree on it.
GET /etag/large
No body.
HTTP 200
payload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/etag/large";
export default performanceTest({
id: { family: "etag", name: "large" },
path,
base: "json.large",
varies: "validators",
heft: 4,
about:
"The framework hashing the body it is about to send and writing the " +
"validator it computed onto the response. Read against json.large, the " +
"difference is the hash and the header. The tag is the framework's own, " +
"so no two of them agree on it.",
request: (c) => c.get(path).okWith(items.large).hasHeader("etag").fresh(),
});
The only row that cannot be sent until the framework has answered a different one: the validator is the framework's to produce. A 304 saves the write and nothing else, because the body is built and hashed before anything is compared.
Before this request, the test sends GET /etag/large once and reads its etag. That request is not measured.
GET /etag/large
No body.
HTTP 304
No body.
import { performanceTest } from "#kit";
const path = "/etag/large";
export default performanceTest({
id: { family: "etag", name: "match_large" },
path,
base: "etag.large",
varies: "conditional",
heft: 4,
about:
"The only row that cannot be sent until the framework has answered a " +
"different one: the validator is the framework's to produce. A 304 saves " +
"the write and nothing else, because the body is built and hashed before " +
"anything is compared.",
request: async (c) => {
const tag = await c.once(path, () => c.get(path).etag());
return c.get(path).header("if-none-match", tag).notModified().emptyBody();
},
});
A conditional request whose validator does not match, answered in full. Read against etag.match_large, the difference is the write the 304 saved, and read against etag.large it is the comparison that failed.
GET /etag/large
No body.
HTTP 200
payload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/etag/large";
/** A validator no framework computes. */
const stale = '"0000000000000000"';
export default performanceTest({
id: { family: "etag", name: "stale_large" },
path,
base: "etag.large",
varies: "stale_validator",
heft: 4,
about:
"A conditional request whose validator does not match, answered in full. " +
"Read against etag.match_large, the difference is the write the 304 " +
"saved, and read against etag.large it is the comparison that failed.",
request: (c) => c.get(path).header("if-none-match", stale).okWith(items.large).fresh(),
});
The framework hashing the body it is about to send and writing the validator it computed onto the response. Read against json.small, the difference is the hash and the header. The tag is the framework's own, so no two of them agree on it.
GET /etag/small
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/etag/small";
export default performanceTest({
id: { family: "etag", name: "small" },
path,
base: "json.small",
varies: "validators",
heft: 1,
about:
"The framework hashing the body it is about to send and writing the " +
"validator it computed onto the response. Read against json.small, the " +
"difference is the hash and the header. The tag is the framework's own, " +
"so no two of them agree on it.",
request: (c) => c.get(path).okWith(items.small).hasHeader("etag").fresh(),
});
2 tests
Request bodies that are not JSON: a urlencoded form and a multipart upload, each bound through the framework's own form support.
urlencoded · multipart
A value written {run.name} is drawn once per run and never given to the framework, so it cannot answer from a table.
query.many's eight fields posted as an application/x-www-form-urlencoded body instead of a query string, bound and echoed. The answer is exactly what query.many answers, so the difference between the two is the form parser against the query parser.
POST /forms/urlencoded
page={run.page}&size={run.size}&status={run.status}&category={run.category}&sort={run.sort}&q={run.q}&minPrice={run.minPrice}&maxPrice={run.maxPrice}HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"page":{run.page},"size":{run.size},"status":"{run.status}","category":"{run.category}","sort":"{run.sort}","q":"{run.q}","minPrice":{run.minPrice},"maxPrice":{run.maxPrice}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/forms/urlencoded";
const FIELDS = ["page", "size", "status", "category", "sort", "q", "minPrice", "maxPrice"] as const;
export default performanceTest({
id: { family: "forms", name: "urlencoded" },
path,
base: "query.many",
varies: "form",
heft: 2,
about:
"query.many's eight fields posted as an application/x-www-form-urlencoded " +
"body instead of a query string, bound and echoed. The answer is exactly " +
"what query.many answers, so the difference between the two is the form " +
"parser against the query parser.",
request: (c) => {
const form = new URLSearchParams(FIELDS.map((name) => [name, String(c.run[name])])).toString();
return c.post(path).raw(form, "application/x-www-form-urlencoded").okWith(items.small, { echo: FIELDS });
},
});
A multipart/form-data upload of two fields and a 32 KB text file. The handler echoes the fields and answers the file's name and byte count, so it has to have read the whole part. Read against forms.urlencoded, the difference is the multipart parser and a body of 32 KB instead of a line.
POST /forms/multipart
--rb-7c4f1e0a9d
Content-Disposition: form-data; name="tenant"
{run.tenant}
--rb-7c4f1e0a9d
Content-Disposition: form-data; name="requestId"
{run.requestId}
--rb-7c4f1e0a9d
Content-Disposition: form-data; name="file"; filename="forms.file.txt"
Content-Type: text/plain
id,name,category,priceCents,inStock
1,slate-lamp-6647,tools,18928,true
2,brass-hinge-5716,garden,17577,true
3,linen-ring-6971,kitchen,5540,true
4,copper-bolt-9217,outdoor,11562,true
5,linen-lamp-8288,office,1413,true
6,copper-pan-9308,tools,10795,true
7,slate-seed-3051,garden,9667,false
8,oak-lamp-8662,kitchen,18032,false
9,amber-hinge-8405,outdoor,9566,true
10,linen-lamp-9676,office,4288,true
11,brass-trowel-2042
… 33,066 bytes totalHTTP 200
payload forms.file.txt as received
{"file":{"name":"forms.file.txt","bytes":32762},"echo":{"tenant":"{run.tenant}","requestId":"{run.requestId}"}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import type { RunValues } from "#kit";
import { forms, uploaded } from "#payloads";
const path = "/forms/multipart";
const BOUNDARY = "rb-7c4f1e0a9d";
function part(disposition: string, body: string, type?: string): string {
const typed = type === undefined ? "" : `Content-Type: ${type}\r\n`;
return `--${BOUNDARY}\r\nContent-Disposition: form-data; ${disposition}\r\n${typed}\r\n${body}\r\n`;
}
/** Built once per run's values, so the 32 KB concatenation is not inside every instance's timed window. */
const bodies = new WeakMap<RunValues, string>();
function multipart(run: RunValues): string {
let body = bodies.get(run);
if (body === undefined) {
body =
part('name="tenant"', run.tenant) +
part('name="requestId"', run.requestId) +
part(`name="file"; filename="${forms.file.name}"`, forms.file.value, "text/plain") +
`--${BOUNDARY}--\r\n`;
bodies.set(run, body);
}
return body;
}
export default performanceTest({
id: { family: "forms", name: "multipart" },
path,
base: "forms.urlencoded",
varies: "multipart",
heft: 3,
about:
"A multipart/form-data upload of two fields and a 32 KB text file. The " +
"handler echoes the fields and answers the file's name and byte count, so " +
"it has to have read the whole part. Read against forms.urlencoded, the " +
"difference is the multipart parser and a body of 32 KB instead of a " +
"line.",
request: (c) =>
c
.post(path)
.raw(multipart(c.run), `multipart/form-data; boundary=${BOUNDARY}`)
.okWith(uploaded, { echo: ["tenant", "requestId"] }),
});
4 tests
The request header map at five headers and at thirty, left unread and with three of them bound and echoed.
few · bind_few · bind_many · many
A value written {run.name} is drawn once per run and never given to the framework, so it cannot answer from a table.
Five request headers, none of which the handler reads. Two of the five are what an HTTP client adds itself, and the other three are the ones the binding rows bind. This is what headers.many is subtracted from.
GET /headers
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/headers";
export default performanceTest({
id: { family: "headers", name: "few" },
path,
heft: 1,
about:
"Five request headers, none of which the handler reads. Two of the five " +
"are what an HTTP client adds itself, and the other three are the ones " +
"the binding rows bind. This is what headers.many is subtracted from.",
request: (c) =>
c
.get(path)
.header("x-rb-tenant", c.run.tenant)
.header("x-rb-request-id", c.run.requestId)
.header("x-rb-account", String(c.run.account))
.okWith(items.small),
});
The same five headers with three of them bound through the framework and written back, one as an integer. Read against headers.few, the difference is the binding rather than the materialising.
GET /headers/bind
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"tenant":"{run.tenant}","requestId":"{run.requestId}","account":{run.account}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/headers/bind";
export default performanceTest({
id: { family: "headers", name: "bind_few" },
path,
base: "headers.few",
varies: "header_binding",
heft: 1,
about:
"The same five headers with three of them bound through the framework and " +
"written back, one as an integer. Read against headers.few, the " +
"difference is the binding rather than the materialising.",
request: (c) =>
c
.get(path)
.header("x-rb-tenant", c.run.tenant)
.header("x-rb-request-id", c.run.requestId)
.header("x-rb-account", String(c.run.account))
.okWith(items.small, { echo: ["tenant", "requestId", "account"] }),
});
Three headers bound out of thirty instead of out of five. Read against headers.bind_few, the difference is whether a framework's binder pays for the headers it was not asked about.
GET /headers/bind
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"tenant":"{run.tenant}","requestId":"{run.requestId}","account":{run.account}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
import { BROWSER_AND_PROXY } from "#models/fixture";
const path = "/headers/bind";
export default performanceTest({
id: { family: "headers", name: "bind_many" },
path,
base: "headers.bind_few",
varies: "header_count",
heft: 2,
about:
"Three headers bound out of thirty instead of out of five. Read against " +
"headers.bind_few, the difference is whether a framework's binder pays " +
"for the headers it was not asked about.",
request: (c) => {
let call = c
.get(path)
.header("x-rb-tenant", c.run.tenant)
.header("x-rb-request-id", c.run.requestId)
.header("x-rb-account", String(c.run.account));
for (const [name, value] of BROWSER_AND_PROXY) call = call.header(name, value);
return call.okWith(items.small, { echo: ["tenant", "requestId", "account"] });
},
});
Thirty request headers, still read by nothing. Read against headers.few, the difference is the cost of materialising twenty-five more that nobody asked for, which is about a kilobyte and close to what a real request carries.
GET /headers
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
import { BROWSER_AND_PROXY } from "#models/fixture";
const path = "/headers";
export default performanceTest({
id: { family: "headers", name: "many" },
path,
base: "headers.few",
varies: "header_count",
heft: 2,
about:
"Thirty request headers, still read by nothing. Read against headers.few, " +
"the difference is the cost of materialising twenty-five more that nobody " +
"asked for, which is about a kilobyte and close to what a real request " +
"carries.",
request: (c) => {
let call = c
.get(path)
.header("x-rb-tenant", c.run.tenant)
.header("x-rb-request-id", c.run.requestId)
.header("x-rb-account", String(c.run.account));
for (const [name, value] of BROWSER_AND_PROXY) call = call.header(name, value);
return call.okWith(items.small);
},
});
6 tests
Every method on one resource, over the rows of the large item payload: a row read, its headers alone, and four writes that answer as if they had written.
create · replace · update · read · delete · head
{draw.item} is a row of items.large, picked per request.
A new item posted as JSON and answered 201, with where it would live and what it would hold. Nothing is stored, so the answer is always the id after the last row. Read against body.bind_small, the difference is a body bound to a model and answered as a created resource rather than echoed with counts.
POST /items
payload items.new
{"name":"amber-trowel-4410","category":"garden","priceCents":2499,"inStock":true}HTTP 201
/^(https?:\/\/[^/]+)?\/items\/1426$/payload items.new as row 1426
{"id":1426,"name":"amber-trowel-4410","category":"garden","priceCents":2499,"inStock":true}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { CREATED, created, items } from "#payloads";
const path = "/items";
/** Relative, or absolute on whatever host the framework thinks it is. */
const LOCATION = new RegExp(`^(https?://[^/]+)?/items/${CREATED}$`);
export default performanceTest({
id: { family: "items", name: "create" },
path,
base: "body.bind_small",
varies: "creation",
heft: 2,
about:
"A new item posted as JSON and answered 201, with where it would live and " +
"what it would hold. Nothing is stored, so the answer is always the id " +
"after the last row. Read against body.bind_small, the difference is a " +
"body bound to a model and answered as a created resource rather than " +
"echoed with counts.",
request: (c) => c.post(path, items.new.value).status(201).bodyIs(created).hasHeader("location", LOCATION),
});
A whole item put at an id and answered with the item under that id. Read against items.create, the difference is the id coming from the path rather than from the server.
PUT /items/{draw.item}payload items.new
{"name":"amber-trowel-4410","category":"garden","priceCents":2499,"inStock":true}HTTP 200
payload items.new as row {draw.item}
{"id":1417,"name":"amber-trowel-4410","category":"garden","priceCents":2499,"inStock":true}Compared as parsed JSON, so key order and how a number is written do not matter. Shown with 1417 for {draw.item}.
import { performanceTest } from "#kit";
import { items, replaced } from "#payloads";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "items", name: "replace" },
path,
base: "items.create",
varies: "method",
heft: 2,
about:
"A whole item put at an id and answered with the item under that id. Read " +
"against items.create, the difference is the id coming from the path " +
"rather than from the server.",
request: (c) => {
const id = c.draw.item();
return c.put(`/items/${id}`, items.new.value).okWith(replaced(id));
},
});
Two fields patched onto a row and answered with the row as it would be. The handler has to read the row, merge the body into it and serialise the result. Read against items.replace, the difference is the merge.
PATCH /items/{draw.item}payload items.patch
{"priceCents":1999,"inStock":false}HTTP 200
payload row {draw.item} of items.large with items.patch applied
{"id":1417,"name":"slate-seed-6074","category":"garden","priceCents":1999,"inStock":false}Compared as parsed JSON, so key order and how a number is written do not matter. Shown with 1417 for {draw.item}.
import { performanceTest } from "#kit";
import { items, patched } from "#payloads";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "items", name: "update" },
path,
base: "items.replace",
varies: "method",
heft: 2,
about:
"Two fields patched onto a row and answered with the row as it would be. " +
"The handler has to read the row, merge the body into it and serialise " +
"the result. Read against items.replace, the difference is the merge.",
request: (c) => {
const id = c.draw.item();
return c.patch(`/items/${id}`, items.patch.value).okWith(patched(id));
},
});
One row of the large payload, looked up by the id in the path. The id is drawn per request, so each instance reads a different row. Read against parameters.one, the difference is a lookup and one row serialised in place of an echo beside the small payload.
GET /items/{draw.item}No body.
HTTP 200
payload row {draw.item} of items.large
{"id":1417,"name":"slate-seed-6074","category":"garden","priceCents":9466,"inStock":true}Compared as parsed JSON, so key order and how a number is written do not matter. Shown with 1417 for {draw.item}.
import { performanceTest } from "#kit";
import { row } from "#payloads";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "items", name: "read" },
path,
base: "parameters.one",
varies: "lookup",
heft: 1,
about:
"One row of the large payload, looked up by the id in the path. The id is " +
"drawn per request, so each instance reads a different row. Read against " +
"parameters.one, the difference is a lookup and one row serialised in " +
"place of an echo beside the small payload.",
request: (c) => {
const id = c.draw.item();
return c.get(`/items/${id}`).okWith(row(id));
},
});
A row deleted and answered 204 with no body. Nothing is removed, so every instance finds the row it names. Read against items.read, the difference is that nothing is serialised at all.
DELETE /items/{draw.item}No body.
HTTP 204
No body.
import { performanceTest } from "#kit";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "items", name: "delete" },
path,
base: "items.read",
varies: "method",
heft: 1,
about:
"A row deleted and answered 204 with no body. Nothing is removed, so " +
"every instance finds the row it names. Read against items.read, the " +
"difference is that nothing is serialised at all.",
request: (c) => c.delete(`/items/${c.draw.item()}`).status(204).emptyBody(),
});
The same lookup asked with HEAD, which the framework answers from its GET route with no body. Read against items.read, the difference is the body left unwritten. Content-Length is not checked, because a framework that streams its JSON sends none on the GET either.
HEAD /items/{draw.item}No body.
HTTP 200
/^application\/json/No body.
import { performanceTest } from "#kit";
const path = "/items/{draw.item}";
export default performanceTest({
id: { family: "items", name: "head" },
path,
base: "items.read",
varies: "method",
heft: 1,
about:
"The same lookup asked with HEAD, which the framework answers from its GET " +
"route with no body. Read against items.read, the difference is the body " +
"left unwritten. Content-Length is not checked, because a framework that " +
"streams its JSON sends none on the GET either.",
request: (c) => c.head(`/items/${c.draw.item()}`).ok().hasHeader("content-type", /^application\/json/).emptyBody(),
});
3 tests
Serialising a body the framework already holds, across three size regimes.
small · large · medium
One row out, serialised from a body the framework already holds. The plainest question in the corpus once something has to be serialised. Read against baseline.plaintext, the difference is the codec.
GET /json/small
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/json/small";
export default performanceTest({
id: { family: "json", name: "small" },
path,
heft: 1,
about:
"One row out, serialised from a body the framework already holds. The " +
"plainest question in the corpus once something has to be serialised. " +
"Read against baseline.plaintext, the difference is the codec.",
request: (c) => c.get(path).okWith(items.small),
});
Fourteen hundred rows out, serialised from a body the framework already holds. The size at which the writer stops being free. Read against json.small, the pair is what separates a framework with a fast codec from one with a fast request path.
GET /json/large
No body.
HTTP 200
payload items.large
{"size":"large","count":1425,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kit
… 128,510 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/json/large";
export default performanceTest({
id: { family: "json", name: "large" },
path,
base: "json.small",
varies: "size",
heft: 4,
about:
"Fourteen hundred rows out, serialised from a body the framework already " +
"holds. The size at which the writer stops being free. Read against " +
"json.small, the pair is what separates a framework with a fast codec " +
"from one with a fast request path.",
request: (c) => c.get(path).okWith(items.large),
});
Eighty-nine rows out, serialised from a body the framework already holds. The middle size, where the codec is doing real work and the response still fits a single write. Read against json.small rather than on its own.
GET /json/medium
No body.
HTTP 200
payload items.medium
{"size":"medium","count":89,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},{"id":8,"name":"oak-lamp-8662","category":"kitc
… 7,953 bytes totalCompared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/json/medium";
export default performanceTest({
id: { family: "json", name: "medium" },
path,
base: "json.small",
varies: "size",
heft: 2,
about:
"Eighty-nine rows out, serialised from a body the framework already " +
"holds. The middle size, where the codec is doing real work and the " +
"response still fits a single write. Read against json.small rather than " +
"on its own.",
request: (c) => c.get(path).okWith(items.medium),
});
3 tests
Per-layer dispatch cost at zero, four and sixteen no-op layers.
none · four · sixteen
No layers in front of a handler that serialises the small payload. A different route carrying the same handler, which should cost nothing. It is the zero point the other two are read against, and a framework where this differs from json.small is paying for the route rather than the layers.
GET /middleware/none
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/middleware/none";
export default performanceTest({
id: { family: "middleware", name: "none" },
path,
base: "json.small",
varies: "route",
heft: 1,
about:
"No layers in front of a handler that serialises the small payload. A " +
"different route carrying the same handler, which should cost nothing. It " +
"is the zero point the other two are read against, and a framework where " +
"this differs from json.small is paying for the route rather than the " +
"layers.",
request: (c) => c.get(path).okWith(items.small),
});
Four no-op layers in front of a handler that serialises the small payload. Four layers in front of the handler, each calling the next and doing nothing else. Read against middleware.none, the difference divided by four is the framework's per-layer cost.
GET /middleware/four
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/middleware/four";
export default performanceTest({
id: { family: "middleware", name: "four" },
path,
base: "middleware.none",
varies: "layers",
heft: 1,
about:
"Four no-op layers in front of a handler that serialises the small " +
"payload. Four layers in front of the handler, each calling the next and " +
"doing nothing else. Read against middleware.none, the difference divided " +
"by four is the framework's per-layer cost.",
request: (c) => c.get(path).okWith(items.small),
});
Sixteen no-op layers in front of a handler that serialises the small payload. Sixteen layers, which is where a per-layer cost that looked like noise at four becomes readable. Read against middleware.none, and against middleware.four to see whether the cost is linear.
GET /middleware/sixteen
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/middleware/sixteen";
export default performanceTest({
id: { family: "middleware", name: "sixteen" },
path,
base: "middleware.none",
varies: "layers",
heft: 1,
about:
"Sixteen no-op layers in front of a handler that serialises the small " +
"payload. Sixteen layers, which is where a per-layer cost that looked " +
"like noise at four becomes readable. Read against middleware.none, and " +
"against middleware.four to see whether the cost is linear.",
request: (c) => c.get(path).okWith(items.small),
});
4 tests
Router captures, with segment depth held constant, each bound as an integer and echoed.
static · one · two · three
A value written {run.name} is drawn once per run and never given to the framework, so it cannot answer from a table.
Four static segments and no captures. This holds the route depth constant for the two rows that capture, so subtracting it leaves the router capturing rather than the router matching a longer path.
GET /parameters/static/segment/literal
No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}]}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/parameters/static/segment/literal";
export default performanceTest({
id: { family: "parameters", name: "static" },
path,
base: "json.small",
varies: "depth",
heft: 1,
about:
"Four static segments and no captures. This holds the route depth " +
"constant for the two rows that capture, so subtracting it leaves the " +
"router capturing rather than the router matching a longer path.",
request: (c) => c.get(path).okWith(items.small),
});
One segment captured, bound as an integer and written back. The value is drawn per run, so a framework that answered from a table would have had to know it in advance, and the echo is checked against what was sent.
GET /parameters/{run.one}/segment/literalNo body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"one":{run.one}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/parameters/{run.one}/segment/literal";
export default performanceTest({
id: { family: "parameters", name: "one" },
path,
base: "parameters.static",
varies: "captures",
heft: 1,
about:
"One segment captured, bound as an integer and written back. The value is " +
"drawn per run, so a framework that answered from a table would have had " +
"to know it in advance, and the echo is checked against what was sent.",
request: (c) => c.get(`/parameters/${c.run.one}/segment/literal`).okWith(items.small, { echo: ["one"] }),
});
The same depth with a second capture in it. Read against parameters.one, the difference is one more segment the router has to capture and one more value the framework has to convert.
GET /parameters/{run.one}/with-second/{run.two}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"one":{run.one},"two":{run.two}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/parameters/{run.one}/with-second/{run.two}";
export default performanceTest({
id: { family: "parameters", name: "two" },
path,
base: "parameters.one",
varies: "captures",
heft: 1,
about:
"The same depth with a second capture in it. Read against parameters.one, " +
"the difference is one more segment the router has to capture and one " +
"more value the framework has to convert.",
request: (c) =>
c.get(`/parameters/${c.run.one}/with-second/${c.run.two}`).okWith(items.small, { echo: ["one", "two"] }),
});
The same depth with three of its four segments captured, which leaves parameters as the only static one. Read against parameters.two, the difference is a third segment the router has to capture and a third value the framework has to convert.
GET /parameters/{run.one}/{run.two}/{run.three}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"one":{run.one},"two":{run.two},"three":{run.three}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/parameters/{run.one}/{run.two}/{run.three}";
export default performanceTest({
id: { family: "parameters", name: "three" },
path,
base: "parameters.two",
varies: "captures",
heft: 1,
about:
"The same depth with three of its four segments captured, which leaves " +
"parameters as the only static one. Read against parameters.two, the " +
"difference is a third segment the router has to capture and a third " +
"value the framework has to convert.",
request: (c) =>
c.get(`/parameters/${c.run.one}/${c.run.two}/${c.run.three}`).okWith(items.small, { echo: ["one", "two", "three"] }),
});
2 tests
Query string parsing, percent-decoding and coercion, with the values echoed and put to no other use.
one · many
A value written {run.name} is drawn once per run and never given to the framework, so it cannot answer from a table.
One query parameter parsed, coerced to an integer and written back. Read against json.small, the difference is the query string being parsed at all.
GET /query/one?page={run.page}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"page":{run.page}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/query/one?page={run.page}";
export default performanceTest({
id: { family: "query", name: "one" },
path,
base: "json.small",
varies: "query_params",
heft: 1,
about:
"One query parameter parsed, coerced to an integer and written back. Read " +
"against json.small, the difference is the query string being parsed at " +
"all.",
request: (c) => c.get("/query/one").query("page", String(c.run.page)).okWith(items.small, { echo: ["page"] }),
});
Eight parameters, which is what a real search endpoint carries: a page and a size, a sort, a text term and four filters. Read against query.one, the difference is seven more keys parsed, coerced and echoed.
GET /query/many?page={run.page}&size={run.size}&status={run.status}&category={run.category}&sort={run.sort}&q={run.q}&minPrice={run.minPrice}&maxPrice={run.maxPrice}No body.
HTTP 200
payload items.small
{"size":"small","count":1,"items":[{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}],"echo":{"page":{run.page},"size":{run.size},"status":"{run.status}","category":"{run.category}","sort":"{run.sort}","q":"{run.q}","minPrice":{run.minPrice},"maxPrice":{run.maxPrice}}}Compared as parsed JSON, so key order and how a number is written do not matter.
import { performanceTest } from "#kit";
import { items } from "#payloads";
const path = "/query/many?page={run.page}&size={run.size}&status={run.status}&category={run.category}&sort={run.sort}&q={run.q}&minPrice={run.minPrice}&maxPrice={run.maxPrice}";
export default performanceTest({
id: { family: "query", name: "many" },
path,
base: "query.one",
varies: "query_params",
heft: 2,
about:
"Eight parameters, which is what a real search endpoint carries: a page " +
"and a size, a sort, a text term and four filters. Read against " +
"query.one, the difference is seven more keys parsed, coerced and echoed.",
request: (c) =>
c
.get("/query/many")
.query("page", String(c.run.page))
.query("size", String(c.run.size))
.query("status", c.run.status)
.query("category", c.run.category)
.query("sort", c.run.sort)
.query("q", c.run.q)
.query("minPrice", String(c.run.minPrice))
.query("maxPrice", String(c.run.maxPrice))
.okWith(items.small, { echo: ["page", "size", "status", "category", "sort", "q", "minPrice", "maxPrice"] }),
});
1 test
Server-sent events: a text/event-stream response written event by event through the framework's own support for it.
medium
items.medium's 89 rows, each the data of one server-sent event. The request carries the Accept header an EventSource sends. Every event is of type message with no id, and there is no Content-Length. Read against stream.ndjson, the difference is the framework's event framing against a line per row.
GET /sse/medium
No body.
HTTP 200
/^text\/event-stream/payload items.medium, one row per event
data: {"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}
data: {"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true}
data: {"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true}
data: {"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true}
data: {"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true}
data: {"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true}
data: {"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false}
data: {"id":8,"name":"oak-lamp-8662
… 8,538 bytes totalCompared event by event as an EventSource dispatches them: each of type message with no id, and its data as parsed JSON.
import { performanceTest } from "#kit";
import { sse } from "#payloads";
const path = "/sse/medium";
export default performanceTest({
id: { family: "sse", name: "medium" },
path,
base: "stream.ndjson",
varies: "event_stream",
heft: 3,
about:
"items.medium's 89 rows, each the data of one server-sent event. The request " +
"carries the Accept header an EventSource sends. Every event is of type " +
"message with no id, and there is no Content-Length. Read against " +
"stream.ndjson, the difference is the framework's event framing against a " +
"line per row.",
request: (c) =>
c
.get(path)
.header("accept", "text/event-stream")
.okWith(sse)
.hasHeader("content-type", /^text\/event-stream/)
.noHeader("content-length"),
});
3 tests
The framework's static-file feature, serving committed files from the payload directory.
large · medium · small
items.large.json sent by the framework's static-file feature from the payload directory, with its modification time. The request accepts gzip, as a browser's does, and the body is compared byte for byte after decoding, so the file may go out as it is or compressed. This is the one family where serving the file's bytes is the point. Last-Modified is checked rather than an ETag, because Go's file server sends no ETag.
GET /static/items.large.json
No body.
HTTP 200
/^application\/json/payload items.large.json
{"size":"large","count":1425,"items":[
{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},
{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},
{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},
{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},
{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},
{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},
{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},
{"id":8,"name":"oak-lamp-8662","catego
… 129,937 bytes totalCompared byte for byte.
import { performanceTest } from "#kit";
import { files } from "#payloads";
const path = "/static/items.large.json";
export default performanceTest({
id: { family: "static", name: "large" },
path,
base: "json.large",
varies: "static_file",
heft: 4,
about:
"items.large.json sent by the framework's static-file feature from the " +
"payload directory, with its modification time. The request accepts gzip, " +
"as a browser's does, and the body is compared byte for byte after " +
"decoding, so the file may go out as it is or compressed. This is the one " +
"family where serving the file's bytes is the point. Last-Modified is " +
"checked rather than an ETag, because Go's file server sends no ETag.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "gzip")
.okWith(files.large)
.hasHeader("content-type", /^application\/json/)
.hasHeader("last-modified"),
});
items.medium.json sent by the same static-file feature, with its modification time, to a request that accepts gzip. Read against json.medium, the difference is the file served against the same 89 rows serialised, and against static.small it is the feature's cost per byte.
GET /static/items.medium.json
No body.
HTTP 200
/^application\/json/payload items.medium.json
{"size":"medium","count":89,"items":[
{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true},
{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true},
{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true},
{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true},
{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true},
{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true},
{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false},
{"id":8,"name":"oak-lamp-8662","categor
… 8,044 bytes totalCompared byte for byte.
import { performanceTest } from "#kit";
import { files } from "#payloads";
const path = "/static/items.medium.json";
export default performanceTest({
id: { family: "static", name: "medium" },
path,
base: "json.medium",
varies: "static_file",
heft: 2,
about:
"items.medium.json sent by the same static-file feature, with its " +
"modification time, to a request that accepts gzip. Read against " +
"json.medium, the difference is the file served against the same 89 rows " +
"serialised, and against static.small it is the feature's cost per byte.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "gzip")
.okWith(files.medium)
.hasHeader("content-type", /^application\/json/)
.hasHeader("last-modified"),
});
items.small.json sent by the same static-file feature, with its modification time, to a request that accepts gzip. The file is one row, so what is left is the feature's own cost per request: finding the file, reading its metadata and writing its headers.
GET /static/items.small.json
No body.
HTTP 200
/^application\/json/payload items.small.json
{"size":"small","count":1,"items":[
{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}
]}
Compared byte for byte.
import { performanceTest } from "#kit";
import { files } from "#payloads";
const path = "/static/items.small.json";
export default performanceTest({
id: { family: "static", name: "small" },
path,
base: "json.small",
varies: "static_file",
heft: 1,
about:
"items.small.json sent by the same static-file feature, with its " +
"modification time, to a request that accepts gzip. The file is one row, " +
"so what is left is the feature's own cost per request: finding the file, " +
"reading its metadata and writing its headers.",
request: (c) =>
c
.get(path)
.header("accept-encoding", "gzip")
.okWith(files.small)
.hasHeader("content-type", /^application\/json/)
.hasHeader("last-modified"),
});
1 test
A response written in parts as it is produced, rather than serialised whole and sent with its length.
ndjson
items.medium's 89 rows written one per line as application/x-ndjson. There is no Content-Length, which is what shows the body left in parts rather than buffered. Read against json.medium, the difference is the framework's streaming path and a write per row.
GET /stream/items
No body.
HTTP 200
/^application\/x-ndjson/payload items.medium, one row per line
{"id":1,"name":"slate-lamp-6647","category":"tools","priceCents":18928,"inStock":true}
{"id":2,"name":"brass-hinge-5716","category":"garden","priceCents":17577,"inStock":true}
{"id":3,"name":"linen-ring-6971","category":"kitchen","priceCents":5540,"inStock":true}
{"id":4,"name":"copper-bolt-9217","category":"outdoor","priceCents":11562,"inStock":true}
{"id":5,"name":"linen-lamp-8288","category":"office","priceCents":1413,"inStock":true}
{"id":6,"name":"copper-pan-9308","category":"tools","priceCents":10795,"inStock":true}
{"id":7,"name":"slate-seed-3051","category":"garden","priceCents":9667,"inStock":false}
{"id":8,"name":"oak-lamp-8662","category":"kitchen","priceCents":18032,"inStock":fal
… 7,915 bytes totalCompared line by line, each line as parsed JSON.
import { performanceTest } from "#kit";
import { stream } from "#payloads";
const path = "/stream/items";
export default performanceTest({
id: { family: "stream", name: "ndjson" },
path,
base: "json.medium",
varies: "streaming",
heft: 3,
about:
"items.medium's 89 rows written one per line as application/x-ndjson. " +
"There is no Content-Length, which is what shows the body left in parts " +
"rather than buffered. Read against json.medium, the difference is the " +
"framework's streaming path and a write per row.",
request: (c) => c.get(path).okWith(stream).hasHeader("content-type", /^application\/x-ndjson/).noHeader("content-length"),
});
3 tests
Server-side rendering of the same model the json family serialises.
large · medium · small
All 1,425 rows of items.large through the same template. Read against template.medium, the difference is the engine's per-row cost at scale, and against json.large it is rendering against serialising at one size.
GET /template/large
No body.
HTTP 200
/html/payload items.large as a page
<!doctype html><html><head><title>items</title></head><body><h1>large</h1><table><thead><tr><th>id</th><th>name</th><th>category</th><th>price</th><th>stock</th></tr></thead><tbody><tr><td>1</td><td>slate-lamp-6647</td><td>tools</td><td>18928</td><td>yes</td></tr><tr><td>2</td><td>brass-hinge-5716</td><td>garden</td><td>17577</td><td>yes</td></tr><tr><td>3</td><td>linen-ring-6971</td><td>kitchen</td><td>5540</td><td>yes</td></tr><tr><td>4</td><td>copper-bolt-9217</td><td>outdoor</td><td>11562</td><td>yes</td></tr><tr><td>5</td><td>linen-lamp-8288</td><td>office</td><td>1413</td><td>yes</td></tr><tr><td>6</td><td>copper-pan-9308</td><td>tools</td><td>10795</td><td>yes</td></tr><tr><td>7</td>< … 122,576 bytes total
Compared with whitespace at element boundaries removed and every other run of whitespace collapsed to one space.
import { performanceTest } from "#kit";
import { pages } from "#payloads";
const path = "/template/large";
export default performanceTest({
id: { family: "template", name: "large" },
path,
base: "json.large",
varies: "renderer",
heft: 5,
about:
"All 1,425 rows of items.large through the same template. Read against " +
"template.medium, the difference is the engine's per-row cost at scale, " +
"and against json.large it is rendering against serialising at one size.",
request: (c) => c.get(path).okWith(pages.large).hasHeader("content-type", /html/),
});
Eighty-nine rows through the same template. Read against template.small, the difference is the engine's per-row cost, and against json.medium it is rendering against serialising at one size.
GET /template/medium
No body.
HTTP 200
/html/payload items.medium as a page
<!doctype html><html><head><title>items</title></head><body><h1>medium</h1><table><thead><tr><th>id</th><th>name</th><th>category</th><th>price</th><th>stock</th></tr></thead><tbody><tr><td>1</td><td>slate-lamp-6647</td><td>tools</td><td>18928</td><td>yes</td></tr><tr><td>2</td><td>brass-hinge-5716</td><td>garden</td><td>17577</td><td>yes</td></tr><tr><td>3</td><td>linen-ring-6971</td><td>kitchen</td><td>5540</td><td>yes</td></tr><tr><td>4</td><td>copper-bolt-9217</td><td>outdoor</td><td>11562</td><td>yes</td></tr><tr><td>5</td><td>linen-lamp-8288</td><td>office</td><td>1413</td><td>yes</td></tr><tr><td>6</td><td>copper-pan-9308</td><td>tools</td><td>10795</td><td>yes</td></tr><tr><td>7</td> … 7,755 bytes total
Compared with whitespace at element boundaries removed and every other run of whitespace collapsed to one space.
import { performanceTest } from "#kit";
import { pages } from "#payloads";
const path = "/template/medium";
export default performanceTest({
id: { family: "template", name: "medium" },
path,
base: "json.medium",
varies: "renderer",
heft: 3,
about:
"Eighty-nine rows through the same template. Read against template.small, " +
"the difference is the engine's per-row cost, and against json.medium it " +
"is rendering against serialising at one size.",
request: (c) => c.get(path).okWith(pages.medium).hasHeader("content-type", /html/),
});
One row rendered through the framework's own view layer instead of serialised. Read against json.small, the difference is the engine, which is why this family is comparable across engines and not across frameworks.
GET /template/small
No body.
HTTP 200
/html/payload items.small as a page
<!doctype html><html><head><title>items</title></head><body><h1>small</h1><table><thead><tr><th>id</th><th>name</th><th>category</th><th>price</th><th>stock</th></tr></thead><tbody><tr><td>1</td><td>slate-lamp-6647</td><td>tools</td><td>18928</td><td>yes</td></tr></tbody></table><p>1 rows</p></body></html>
Compared with whitespace at element boundaries removed and every other run of whitespace collapsed to one space.
import { performanceTest } from "#kit";
import { pages } from "#payloads";
const path = "/template/small";
export default performanceTest({
id: { family: "template", name: "small" },
path,
base: "json.small",
varies: "renderer",
heft: 2,
about:
"One row rendered through the framework's own view layer instead of " +
"serialised. Read against json.small, the difference is the engine, which " +
"is why this family is comparable across engines and not across " +
"frameworks.",
request: (c) => c.get(path).okWith(pages.small).hasHeader("content-type", /html/),
});