Measurement
Part of Newsletter measurement that keeps permission, attention, customers and cash apart
Treat newsletter opens as a qualified proxy and define every metric by its event
Define newsletter metrics for England with precise events, denominators, periods, sources and limits across delivery, access and money records.
Choose measures from the decision, then define each one before viewing results. For an England operation, record the geography rule rather than treating a UK total as proof of residence.
What to take away
- Define each metric by its event, population, numerator, denominator, unit and version before viewing results.
- Treat opens as a qualified proxy for a remote-content load, not as proof of readership.
- Keep legal eligibility and suppression logic outside the performance score to avoid unlawful sends.
- Report money using the exact records of the contracted provider, not another provider's labels.
A compact metric register
This set covers different stages without supplying target values.
| Measure | Exact specification | Version field |
|---|---|---|
| Edition release completion | Event: an approved edition enters its intended delivery route. Population: editions scheduled in the period. Numerator: released editions; denominator: scheduled editions. Unit: percentage and counts. Record timezone, reschedules, test sends, publishing system, query version and editorial owner. | Query version for the publishing-system extract; definition version for the release rule. |
| Eligible transport acceptance | Event: the supplier records its documented accepted or delivered state. Population: unique intended recipients after current suppressions. Numerator: recipients in that supplier state; denominator: eligible recipients submitted. State period, timezone, bounces, duplicates, internal tests, supplier/export version and technical uncertainty. | Supplier/export version plus query version for the recipient extract. |
| Suppression exception count | Event: a currently suppressed address reaches the send population. Population: selected recipients. Numerator: verified exceptions; denominator: selected recipients; unit: rate and count. Period/timezone: edition window. Exclude synthetic tests. Source/version: audience and suppression exports plus query version. Permission owner investigates every exception. | Query version for audience and suppression exports; definition version for the exception rule. |
| Unique click proxy | Event: at least one eligible tracked-link event after bot and test filters. Population: recipients accepted for that edition. Numerator: deduplicated recipients with an eligible click; denominator: accepted recipients. Preserve filter, lookback, timezone, redirect source and definition version. | Definition version for the filter and lookback rule; query version for the event extract. |
| Paid-access fulfilment | Event: approved entitlement becomes available after required customer and payment states. Population: eligible completed orders. Numerator: orders with verified access; denominator: eligible orders; unit: rate and counts. State period, timezone, source, definition version and exclusions. | Definition version for the entitlement rule; query version for the order extract. |
| Cash reconciliation | Event: an eligible transaction matches processor, bank and accounting records. Population: period transactions at the accounting cut-off. Numerator: matched items; denominator: eligible items; unit: rate and counts. Report timezone, currency, exclusions, ledger sources, query version, unmatched items and uncertainty. | Query version for each ledger extract; definition version for the match rule. |
| Illustrative England example: eligible transport acceptance | Period: 1 to 31 March 2025, Europe/London. Population: 12,000 eligible recipients for an England-focused edition after suppression. Numerator: 11,820 recipients in the supplier accepted state. Denominator: 12,000 eligible recipients submitted. Unit: 98.5% and 11,820/12,000. Version: supplier export v4.1 and query v7.2. | Illustrative only. Supplier/export version plus query version. |
For a UK-wide total, report the England, Scotland, Wales and Northern Ireland split only where residence is known. When residence is unknown, show the total and the unknown share separately. Do not treat a UK figure as proof of England residence. Record the geography rule and the source of residence data.
SMTP concerns mail transport, not human reading, as the global RFC 5321 standard makes clear. Keep a supplier's status name and documentation date instead of translating it into attention.
Treat opens as a qualified proxy
If opens are retained, define the event as a qualifying remote-content load under the chosen supplier rule. Do not label it "readership". Apple's UK Mail Privacy Protection documentation says remote content may be downloaded privately in the background when a message is received rather than viewed. State this technical limitation beside the result and avoid person-level interpretation.
Click data also has uncertainty. Document scanner and bot filters, repeated-action handling and links excluded from measurement. A click may show an observable interaction, not comprehension, satisfaction, purchase or causation.
Keep legal eligibility outside the performance score
The send denominator should reflect the approved permission and suppression logic. ICO electronic-mail guidance explains that rules differ with subscriber type and route. Qualified privacy and PECR review must set the actual eligibility rule. Performance cannot compensate for an unlawful or inappropriate send.
For an England operation, check the ICO registration self-assessment and record the registration number or exemption basis. PECR 2003 as amended is the governing instrument. For the ICO electronic-mail guidance, cite the page title and the last-updated date shown on the page.
Minimise the underlying data. The ICO's data-minimisation principle requires personal information to be relevant and limited to what is necessary. Prefer aggregated reporting where it answers the decision, control access and define deletion.
Interpret money with the right owner
Gross receipts record billed amounts, settled cash records cleared funds, and refunds reverse both, so they are distinct events; processor deductions and taxable profit follow different rules again. Stripe's current global records describe refund states separately from formal card disputes. Use the exact records of the contracted provider, not Stripe's labels if another service is used.
HMRC requires self-employed businesses to retain sales, income and expense records for at least 5 years after the 31 January following the relevant tax year, as set out in its business-record guidance. Accounting and tax reviewers must decide recognition and VAT treatment. Analysts should report the event ledger and reconciliation gap without making those conclusions.
Approve a metric only when another analyst can reproduce it from the registry. If its event, denominator, source or version is unknown, show the raw observation with a warning or leave the field blank.
Before you act
- Choose measures from the decision first.
- Define each metric before viewing results.
- Record timezone, source and version for every metric.
- Treat opens as a qualified proxy, not readership.
- Keep legal eligibility outside the performance score.
- Use the contracted provider's exact records for money.
Common questions
How should opens be defined if they are retained?
See "Treat opens as a qualified proxy". Define the event as a qualifying remote-content load under the chosen supplier rule, not readership. Record the limitation beside the result.
What should the send denominator reflect?
See "Keep legal eligibility outside the performance score". The denominator follows approved permission and suppression logic. Qualified privacy and PECR review set eligibility.
Why should analysts avoid making accounting conclusions?
See "Interpret money with the right owner". Report the event ledger and reconciliation gap. Accounting and tax reviewers decide recognition and VAT treatment.
