Skip to content

561 Group · Operations

How to prove a service reached someone: the receiving ladder

A green build proves a build. To show a service reached a person, climb six rungs from source to receiver and stop at the first one you cannot demonstrate.

Operations

How to prove a service reached someone: the receiving ladder

You prove a service reached someone by climbing a ladder of observations from the source change to the person's actual experience, and treating the first rung you cannot demonstrate as the location of the fault. A successful build, a healthy server and an HTTP 200 are each true statements about a different rung; none of them says the right thing arrived. Coesite Operations exists for this problem: the interval between "our system says it is done" and "somebody outside it received the result."

What does "running" versus "received" mean?

A website can have a successful build, a healthy server and a broken public experience at the same time. That is not a paradox; the three measurements concern different parts of the service. Running is a claim about the supplier's machinery. Received is a claim about a counterparty's experience. The Group states the invariant in four words: supplier completion is not customer acceptance.

The practical consequence is that the finish line for operations work is not a closed ticket. It is a result somebody outside the machine room can actually receive, together with enough evidence to improve the next attempt.

What are the six rungs?

RungQuestionIf it is missing or disagrees
1. SourceWhich revision carries the intended change?The change may be a draft, on the wrong branch, or an unintegrated dependency
2. ArtifactWhich build artifact included that revision?Investigate source composition, omitted files, failed builds, stale dependency pins
3. ProviderWhich provider instance accepted that artifact?Investigate release selection, provider refusal, an unreceived deployment
4. RouteWhich origin and edge routes point to that instance?Investigate DNS, route claims, reverse proxies, stale worker bindings, cache rules
5. ResponseWhat did an independent client receive at the intended address?Compare status, body, headers and revision with expected behavior
6. ReceiverDid the actual receiver obtain the promised outcome?Repair the user-facing effect, not the dashboard label

Each rung narrows the intervention. A source check does not prove a release. A provider receipt does not prove that a hostname routes to it. An HTTP 200 does not prove the right article or function arrived.

How do you take the reader's reading?

Record the exact requested address, the observation time with a time zone, the request method, the response status, any redirect destination and the content that actually arrived. Reproduce from outside the serving system, not only from the machine running it. Keep ordinary browser caching, logged-in visibility and CDN behavior inside the investigation.

A first anonymous reading of a public route needs only a few lines:

sh
curl --silent --show-error --max-time 10 \
  --dump-header response.headers \
  --output response.body \
  https://example.org/
sha256sum response.body

Replace the example domain with the real canonical address. Two cautions apply. First, a digest establishes the received bytes, not whether they are semantically correct. Second, do not automatically follow redirects on the first reading; the redirect is often the result you need to inspect.

For a static page, compare the expected artifact against the retrieved bytes. For a dynamic page, compare stable identity, canonical links and the specific required behavior; a request-time clock or other changing fields makes whole-body byte equality the wrong test. Do not declare a mismatch from an unrelated variable.

What if the monitor cannot reach the server?

Then the observation failed, which is a fact about the monitor. It does not by itself prove the server is down. And if a public reader cannot reach a route, that experience is nevertheless a meaningful service failure. Preserve both truths. This is why the Group reads its estate in four values instead of two; see four truth values for service status.

How do you write the incident record?

A useful incident record answers six questions in plain language.

  1. Who was affected and what failed? The user-visible symptom and the first verified observation.
  2. What should have happened? The accepted content, service contract or customer result.
  3. Which boundary broke? Cite evidence at the source, build, deployment, routing, response or receiver rung.
  4. What changed? The exact intervention, the owning system and its version.
  5. What happened afterward? Repeat the same external test and preserve its response.
  6. What remains unknown? Missing provider access, unmeasured audience effects and unresolved edge conditions stay explicit.

Append later observations and corrections to the same accountable incident. Do not write four independent summaries, each confidently paraphrasing the previous one; the next operator needs to reproduce the check and see what is still open without reconstructing a conversation. In MAPE-K terms (how to run a company as a MAPE-K loop) this is the closure that makes a pipeline into a loop: the effect of Execute is re-observed by Monitor.

What belongs in a public account?

An operations article explains the problem, date, observed impact, repair, independently checked outcome and the lesson other operators can reuse. Internal host paths, credentials, financial accounts and customer records do not belong in it. Publishing the lesson is a separate disclosure decision from recording the full incident. When the Group later finds that something it published was wrong, the default is to correct the public surface quietly unless people were materially affected or a duty requires notice; see first contact is one-shot, and public errors are quietly corrected.

How do you verify a payment-gated or login-gated service?

Preserve the meaningful refusal. A machine interface that answers 402 has told you something true: it is alive and it wants payment. Classifying it as "down" is a monitoring error and classifying it as "up" is another one. The Group tells it as both. Use the declared method, media type and input contract of the interface, and keep stable identity and terms separate from observation times, security nonces and other intentional variation.

A worked example, in outline

Suppose an editor revises an article, the release job reports success, and a reader still sees the old text. The ladder gives an order of questions, not a guess.

  1. Is the revision on the main branch? (rung 1)
  2. Does the built artifact contain the new text? Compare the artifact's page to the source. (rung 2)
  3. Which provider version is live, and is it the one built from that artifact? (rung 3)
  4. Does the hostname route to that version, or to a stale binding? (rung 4)
  5. What bytes does an anonymous outside client receive, and what are the caching headers? (rung 5)
  6. Does the reader's experience match, including any cached copy on their side? (rung 6)

This is hypothetical; it is an illustration of the method, not a report of a particular incident. Locate the first unsupported link and repair that, not the whole chain.

Is this implemented?

The ladder is an operating procedure written by the operations company and used in its editorial and incident practice. The Group's public site-building tooling separately tracks source, build, provider and public receiving as distinct states, and its own documentation refuses to call an address live until independently observed. No quantitative claim about reduction in incidents is made here.

Where to go next

Four truth values for service status · Deterministic tools before model tokens · Market observations, where each reading carries its publisher, source URL, time, digest and unit · Services · the 561 Group

Contents

Explore

Tagged

  • service-delivery
  • evidence
  • verification
  • operations
  • receiving