GeezerVison

1. Introduction

This document provides three pieces of infrastructure for generic reporting, which may be used or extended by other specifications:

1. A generic framework for defining report types and reporting endpoints, and a document format for sending reports to endpoints over HTTP.

2. A specific mechanism for configuring reporting endpoints in a document or worker, and for delivering reports whose lifetime is tied to that document or worker.

3. A JavaScript interface for observing reports generated within a document or worker.

Other specifications may extend or make use of these pieces, for instance by defining concrete report types, or alternative configuration or delivery mechanisms for non-document-based reports.

1.1. Guarantees

This specification aims to provide a best-effort report delivery system that executes out-of-band with website activity. The user agent will be able to do a better job prioritizing and scheduling delivery of reports, as it has an overview of cross-origin activity that individual websites do not, and can deliver reports based on error conditions that would prevent a website from loading in the first place.

The delivery is not, however, guaranteed in any way, and reporting is not intended to be used as a reliable communications channel. Network conditions may prevent reports from reaching their destination at all, and user agents are permitted to reject and not deliver a report for any reason.

1.2. Examples

Reporting-Endpoints: endpoint-1="https://example.com/reports"

Content-Security-Policy: ...; report-to endpoint-1
Public-Key-Pins: ...; report-to=endpoint-1

Reporting-Endpoints: csp-endpoint="https://example.com/csp-reports",
                     hpkp-endpoint="https://example.com/hpkp-reports"

Content-Security-Policy: ...; report-to csp-endpoint
Public-Key-Pins: ...; report-to=hpkp-endpoint

2. Generic Reporting Framework

This section defines the generic concepts of reports and endpoints, and how reports are serialized into the application/reports+json format.

2.1. Concepts

2.1.1. Endpoints

An endpoint is location to which reports for a particular origin may be sent.

Each endpoint has a name, which is an ASCII string.

Each endpoint has a url, which is a URL.

Each endpoint has a failures, which is a non-negative integer representing the number of consecutive times this endpoint has failed to respond to a request.

2.1.2. Report Type

A report type is a non-empty string that specifies the set of data that is contained in the body of a report.

When a report type is defined (in this spec or others), it can be specified to be visible to ReportingObservers, meaning that reports of that type can be observed by a reporting observer. By default, report types are not visible to ReportingObservers.

2.1.3. Reports

A report is a collection of arbitrary data which the user agent is expected to deliver to a specified endpoint.

Each report has a body, which is either null or an object which can be serialized into a JSON text. The fields contained in a report’s body are determined by the report’s type.

Each report has a url, which is typically the address of the Document or Worker from which the report was generated.

Note: We strip the username, password, and fragment from this serialized URL. See § 8.1 Capability URLs.

Each report has a user agent, which is the value of the User-Agent header of the request from which the report was generated.

Note: The user agent of a report represents the User-Agent sent by the browser for the page which generated the report. This is potentially distinct from the User-Agent sent in the HTTP headers when uploading the report to a collector — for instance, where the browser has chosen to use a non-default User-Agent string such as the "request desktop site" feature.

Each report has a destination, which is a string representing the name of the endpoint that the report will be sent to.

Each report has a type, which is a report type.

Each report has a timestamp, which records the time at which the report was generated, in milliseconds since the unix epoch.

Each report has an attempts counter, which is a non-negative integer representing the number of times the user agent attempted to deliver the report.

2.2. Media Type

The media type used when POSTing reports to a specified endpoint is application/reports+json.

2.3. Queue data as type for destination

To generate a report given a serializable object (data), a string (type), another string (destination), and an environment settings object (settings):

1. Let report be a new report object with its values initialized as follows:

   body

   data

   user agent

   The current value of navigator.userAgent

   destination

   destination

   type

   type

   timestamp

   The current timestamp.

   attempts

   0

2. Let url be settings’s creation URL.

3. Set url’s username to the empty string, and its password to null.

4. Set report’s url to the result of stripping URL for use in reports, given url.

5. Return report.

Note: reporting observers can only observe reports from the same environment settings object.

Note: We strip the username, password, and fragment from the serialized URL in the report. See § 8.1 Capability URLs.

Note: The user agent MAY reject reports for any reason. This API does not guarantee delivery of arbitrary amounts of data, for instance.

Note: Non user agent clients (with no JavaScript engine) should not interact with reporting observers, and thus should return in step 6.

2.4. Serialize Reports

To serialize a list of reports to JSON,

1. Let collection be an empty list.

2. For each report in reports:

   1. Let data be a map with the following key/value pairs:

      age

      The number of milliseconds between report’s timestamp and the current time.

      type

      report’s type

      url

      report’s url

      user_agent

      report’s user agent

      body

      report’s body

      Note: Client clocks are unreliable and subject to skew. We therefore deliver an age attribute rather than an absolute timestamp. See also § 9.2 Clock Skew

   2. Increment report’s attempts.

   3. Append data to collection.

3. Return the byte sequence resulting from executing serialize an Infra value to JSON bytes on collection.

3. Document Centered Reporting

This section defines the mechanism for configuring reporting endpoints for reports generated by actions in a document (or in a worker script). Such reports have a lifetime which is tied to that of the document or worker where they were generated.

3.1. Document configuration

Each object implementing WindowOrWorkerGlobalScope has an endpoints list, which is a list of endpoints, each of which MUST have a distinct name. (Uniqueness is guaranteed by the algorithm in § 3.3 Process reporting endpoints for response.)

Each object implementing WindowOrWorkerGlobalScope has an reports list, which is a list of reports.

To initialize a global’s endpoint list, given a WindowOrWorkerGlobalScope (scope) and a response (response), set scope’s endpoints to the result of executing § 3.3 Process reporting endpoints for response given response.

3.2. The Reporting-Endpoints HTTP Response Header Field

A server MAY define a set of reporting endpoints for a document or a worker script resource it returns, via the Reporting-Endpoints HTTP response header field. This mechanism is defined in § 3.2 The Reporting-Endpoints HTTP Response Header Field, and its processing in § 3.3 Process reporting endpoints for response.

The value of the Reporting-Endpoints HTTP response header field is used to construct the reporting configuration for a resource.

Reporting-Endpoints is a Dictionary Structured Field [STRUCTURED-FIELDS]. Each entry in the dictionary defines an endpoint to which reports may be delivered. The entry value MUST be a string.

Each endpoint is defined by a String Item, which is interpreted as a URI-reference. If its value is not a valid URI-reference, that endpoint member MUST be ignored.

Moreover, the URL that the member’s value represents MUST be potentially trustworthy [SECURE-CONTEXTS]. Non-secure endpoints will be ignored.

No parameters are defined for endpoints, and any parameters which are specified will be silently ignored.

The header is represented by the following ABNF grammar [RFC5234]:

Reporting-Endpoints = sf-dictionary

Specifications that define Structured Fields or parameters (such as report-to) referencing an endpoint by its name SHOULD specify that the value is an sf-token [STRUCTURED-FIELDS].

3.3. Process reporting endpoints for response

Given a response (response), this algorithm extracts and returns a list of endpoints.

1. Abort these steps if response’s HTTPS state is not "modern", and the origin of response’s url is not potentially trustworthy.

2. Let parsed header be the result of executing get a structured field value given "Reporting-Endpoints" and "dictionary" from response’s header list.

3. If parsed header is null, abort these steps.

4. Let endpoints be an empty list.

5. For each name → value_and_parameters of parsed header:

   1. Let endpoint url string be the first element of the tuple value_and_parameters. If endpoint url string is not a string, then continue.

   2. Let endpoint url be the result of executing the URL parser on endpoint url string, with base URL set to response’s url. If endpoint url is failure, then continue.

   3. If endpoint url’s origin is not potentially trustworthy, then continue.

   4. Let endpoint be a new endpoint whose properties are set as follows:

      name

      name

      url

      endpoint url

      failures

      0

   5. Add endpoint to endpoints.

6. Return endpoints.

3.4. Report Generation

3.4.1. Generate report of type with data

When the user agent is to generate and queue a report for a Document or WorkerGlobalScope object (context), given a string (type), a string (destination), and a serializable object (data), it must run the following steps:

1. Let settings be context’s relevant settings object.

2. Let report be the result of running generate a report with data, type, destination and settings.

3. If settings is given, then

   1. Let scope be settings’s global object.

   2. If scope is an object implementing WindowOrWorkerGlobalScope, then execute § 4.2 Notify reporting observers on scope with report with scope and report.

4. Append report to context’s reports.

3.5. Report Delivery

Over time, various features will queue up a list of reports in documents and workers. The user agent will periodically grab the list of currently queued reports, and deliver them to the associated endpoints. This document does not define a schedule for the user agent to follow, and assumes that the user agent will have enough contextual information to deliver reports in a timely manner, balanced against impacting a user’s experience.

That said, a user agent SHOULD make an effort to deliver reports as soon as possible after queuing, as a report’s data might be significantly more useful in the period directly after its generation than it would be a day or a week later.

3.5.1. Send reports

A user agent sends a list of reports (reports) for WindowOrWorkerGlobalScope object (context) by executing the following steps:

1. Let endpoint map be an empty map of endpoint objects to lists of report objects.

2. For each report in reports:

   1. If there exists an endpoint (endpoint) in context’s endpoints list whose name is report’s destination:

      1. Append report to endpoint map’s list of reports for endpoint.

      2. Otherwise, remove report from reports.

3. For each (endpoint, report list) pair in endpoint map:

   1. Let origin map be an empty map of origins to lists of report objects.

   2. For each report in report list:

      1. Let origin be the origin of report’s url.

      2. Append report to origin map’s list of reports for origin.

   3. For each (origin, per-origin reports) pair in origin map, execute the following steps asynchronously:

      1. Let result be the result of executing § 3.5.2 Attempt to deliver reports to endpoint on endpoint, origin, and per-origin reports.

      2. If result is "Failure":

         1. Increment endpoint’s failures.

      3. If result is "Remove Endpoint":

         1. Remove endpoint from context’s endpoints list.

      4. Remove each report from reports.

      We don’t specify any retry mechanism here for failed reports. We may want to add one here, or provide some indication that the delivery failed.

Note: User agents MAY decide to attempt delivery for only a subset of the collected reports or endpoints (because, for example, sending all the reports at once would consume an unreasonable amount of bandwidth, etc). As reports are only removed from the cache after delivery has been attempted, skipped reports will simply be delivered later.

3.5.2. Attempt to deliver reports to endpoint

Given an endpoint (endpoint), an origin (origin), and a list of reports (reports), this algorithm will construct a request, and attempt to deliver it to endpoint. It returns "Success" if that delivery succeeds, "Remove Endpoint" if the endpoint explicitly removes itself as a reporting endpoint by sending a 410 response, and "Failure" otherwise.

1. Let body be the result of executing serialize a list of reports to JSON on reports.

2. Let request be a new request with the following properties [FETCH]:

   method

   "POST"

   url

   endpoint’s url

   origin

   origin

   header list

   A new header list containing a header named `Content-Type` whose value is `application/reports+json`

   client

   null

   window

   "no-window"

   service-workers mode

   "none"

   initiator

   ""

   destination

   "report"

   mode

   "cors"

   unsafe-request flag

   set

   credentials

   "same-origin"

   body

   A body whose source is body.

   Note: Reports are sent with credentials set to same-origin. This allows reporting endpoints which are same-origin with the reporting page to get extra context about the nature of the report: for example, to understand whether a given user’s account is triggering errors consistently, or if a certain sequence of actions taken on other pages is triggering a report on this page. This does not leak any new information to the reporting endpoint that it could not obtain in other ways. That is not the case for cross-origin reporting endpoints, so they do not receive credentials.

3. Queue a task to fetch request.

4. Wait for a response (response).

5. If response’s status is an OK status (200-299), return "Success".

6. If response’s status is 410 Gone [RFC9110], return "Remove Endpoint".

7. Return "Failure".

3.6. Strip URL for use in reports

1. If url’s scheme is not an HTTP(S) scheme, then return url’s scheme.

2. Set url’s fragment to the empty string.

3. Set url’s username to the empty string.

4. Set url’s password to the empty string.

5. Return the result of executing the URL serializer on url.

4. Reporting Observers

A reporting observer observes some types of reports from JavaScript, and is represented in JavaScript by the ReportingObserver object.

Each object implementing WindowOrWorkerGlobalScope has a registered reporting observer list, which is an ordered set of reporting observers.

Any reporting observer that is in a registered reporting observer list is considered registered.

Each object implementing WindowOrWorkerGlobalScope has a report buffer, which is a list of reports that have been generated in that WindowOrWorkerGlobalScope. This list is initially empty, and the reports are stored in the same order in which they are generated.

Note: The purpose of the report buffer is to allow reporting observers to observe reports that were generated earlier than that observer could be created (via the buffered option). For example, some reports might be generated during an earlier stage of page loading than when an observer could first be created, or before a JavaScript library is loaded that wishes to observe these reports.

Note: Reporting observers are only relevant for user agents with JavaScript engines.

4.1. Interface ReportingObserver

dictionary ReportBody {
};

dictionary Report {
  DOMString type;
  DOMString url;
  ReportBody? body;
};

[Exposed=(Window,Worker)]
interface ReportingObserver {
  constructor(ReportingObserverCallback callback, optional ReportingObserverOptions options = {});
  undefined observe();
  undefined disconnect();
  ReportList takeRecords();
};

callback ReportingObserverCallback = undefined (sequence<Report> reports, ReportingObserver observer);

dictionary ReportingObserverOptions {
  sequence<DOMString> types;
  boolean buffered = false;
};

typedef sequence<Report> ReportList;

A Report is the application-exposed representation of a report.

ReportBody is an abstract dictionary type from which specific report types should inherit.

Each ReportingObserver object has these associated concepts:

• A callback function set on creation.

• A ReportingObserverOptions dictionary called options.

• A list of Report objects called the report queue, which is initially empty.

A ReportList represents a sequence of Reports, providing developers with all the convenience methods found on JavaScript arrays.

The ReportingObserver(callback, options) constructor, when invoked, must run these steps:

1. Create a new ReportingObserver object observer.

2. Set observer’s callback to callback.

3. Set observer’s options to options.

4. Return observer.

The observe() method, when invoked, must run these steps:

1. Let global be the be the relevant global object of this.

2. Append this to the global’s registered reporting observer list.

3. If this’s buffered option is false, return.

4. Set this’s buffered option to false.

5. For each report in global’s report buffer, queue a task to execute § 4.3 Add report to observer with report and this.

The disconnect() method, when invoked, must run these steps:

1. If this is not registered, return.

2. Let global be the relevant global object of this.

3. Remove this from global’s registered reporting observer list.

The takeRecords() method, when invoked, must run these steps:

1. Let reports be a copy of this’s report queue.

2. Empty this’s report queue.

3. Return reports.

4.2. Notify reporting observers on scope with report

This algorithm makes report’s contents available to any registered reporting observers on the provided WindowOrWorkerGlobalScope.

1. For each ReportingObserver observer registered with scope, execute § 4.3 Add report to observer on report and observer.

2. Append report to scope’s report buffer.

3. Let type be report’s type.

4. If scope’s report buffer now contains more than 100 reports with type equal to type, remove the earliest item with type equal to type in the report buffer.

4.3. Add report to observer

Given a report report and a ReportingObserver observer, this algorithm adds report to observer’s report queue, so long as report’s type is observable by observer.

1. If report’s type is not visible to ReportingObservers, return.

2. If observer’s options has a non-empty types member which does not contain report’s type, return.

3. Create a new Report r with type initialized to report’s type, url initialized to report’s url, and body initialized to report’s body.

how to polymorphically initialize body?

3. Append r to observer’s report queue.

4. If the size of observer’s report queue is 1:

   1. Let global be observer’s relevant global object.

   2. Queue a task to § 4.4 Invoke reporting observers with notify list with a copy of global’s registered reporting observer list.

4.4. Invoke reporting observers with notify list

This algorithm invokes observer callback functions for reports of previously observed behavior.

1. For each ReportingObserver observer in notify list:

   1. If observer’s report queue is empty, then continue.

   2. Let reports be a copy of observer’s report queue

   3. Empty observer’s report queue

   4. Invoke observer’s callback with « reports, observer » and "report", and with observer as the callback this value.

5. Implementation Considerations

5.1. Delivery

The user agent SHOULD attempt to deliver reports as soon as possible to provide feedback to developers as quickly as possible. However, when this desire is balanced against the impact on the user, the user wins. With that in mind, the user agent MAY delay delivery of reports based on its knowledge of the user’s activities and context.

For instance, the user agent SHOULD prioritize the transmission of reporting data lower than other network traffic. The user’s explicit activities on a website should preempt reporting traffic.

The user agent MAY choose to withhold report delivery entirely until the user is on a fast, cheap network in order to prevent unnecessary data cost.

The user agent MAY choose to prioritize reports from particular origins over others (perhaps those that the user visits most often?)

5.2. Garbage Collection

Periodically, the user agent SHOULD walk through the cached reports and endpoints, and discard those that are no longer relevant. These include:

• endpoints whose failures exceed some user-agent-defined threshold (~5 seems reasonable)

• reports which have not been delivered in some arbitrary period of time (perhaps ~2 days?)

For any reports that are discarded, these reports should also be removed from the report buffer of any reporting observer.

6. Sample Reports

This section is non-normative.

This example shows the format in which reports are sent by the user agent to the reporting endpoint. The sample submission contains three reports which have been bundled together and sent in a single HTTP request. (The report types and bodies themselves are not intended to be representative of any actual feature, as those are outside of the scope of this specification).

POST / HTTP/1.1
Host: example.com
...
Content-Type: application/reports+json

[{
  "type": "security-violation",
  "age": 10,
  "url": "https://example.com/vulnerable-page/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "blocked": "https://evil.com/evil.js",
    "policy": "bad-behavior 'none'",
    "status": 200,
    "referrer": "https://evil.com/"
  }
}, {
  "type": "certificate-issue",
  "age": 32,
  "url": "https://www.example.com/",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "date-time": "2014-04-06T13:00:50Z",
    "hostname": "www.example.com",
    "port": 443,
    "effective-expiration-date": "2014-05-01T12:40:50Z",
    "served-certificate-chain": [
      "-----BEGIN CERTIFICATE-----\n
      MIIEBDCCAuygAwIBAgIDAjppMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT\n
      ...
      HFa9llF7b1cq26KqltyMdMKVvvBulRP/F/A8rLIQjcxz++iPAsbw+zOzlTvjwsto\n
      WHPbqCRiOwY1nQ2pM714A5AuTHhdUDqB1O6gyHA43LL5Z/qHQF1hwFGPa4NrzQU6\n
      yuGnBXj8ytqU0CwIPX4WecigUCAkVDNx\n
      -----END CERTIFICATE-----",
      ...
    ]
  }
}, {
  "type": "cpu-on-fire",
  "age": 29,
  "url": "https://example.com/thing.js",
  "user_agent": "Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0",
  "body": {
    "temperature": 614.0
  }
}]

7. Automation

For the purposes of user-agent automation and application testing, this document defines a number of extension commands for the [WebDriver] specification.

7.1. Generate Test Report

The Generate Test Report extension command simulates the generation of a report for the purposes of testing. This report will be observed by any registered reporting observers.

The extension command is defined as follows:

dictionary GenerateTestReportParameters {
  required DOMString message;
  DOMString group = "default";
};

The remote end steps are:

1. If parameters is not a JSON Object, return a WebDriver error with WebDriver error code invalid argument.

2. Let message be the result of trying to get parameters’s message property.

3. If message is not present, return a WebDriver error with WebDriver error code invalid argument.

4. If the current browsing context is no longer open, return a WebDriver error with WebDriver error code no such window.

5. Handle any user prompts and return its value if it is a WebDriver error.

6. Let group be parameters’s group property.

7. Let body be a new object that can be serialized into a JSON text, containing a single string field, body_message.

8. Set body_message to message.

9. Let settings be the environment settings object of the current browsing context’s active document.

10. Execute generate and queue a report with body, "test", group, and settings.

11. Return success with data null.

8. Security Considerations

8.1. Capability URLs

Some URLs are valuable in and of themselves. They may contain explicit credentials in the username and password portion of the URL, or may grant access to some resource to anyone with knowledge of the URL path. Additionally, they may contain information which was never intended leave the user’s browser in the URL fragment. See [CAPABILITY-URLS] for more information.

To mitigate the possibility that such URLs will be leaked via this reporting mechanism, the algorithms here strip out credential information and fragment data from the URL sent as a report’s originator. It is still possible, however, for sensitive information in the URL’s path to be leaked this way. Sites which use such URLs may need to operate their own reporting endpoints.

Additionally, such URLs may be present in a report’s body. Specifications which extend this API and which include any URLs in a report’s body SHOULD require that they be similarly stripped.

9. Privacy Considerations

9.1. Network Leakage

Because there is a delay between a page being loaded and a report being generated and sent, it’s entirely possible for a report generated while a user is on one network to be sent while the user is on another network.

This behaviour is limited to the lifetime of the document which generated the reports, though, and such a document could be generating traffic on the new network through other means in any case, even after the document is closed, through mechanisms such as navigator.sendBeacon.

Consider mitigations. For example, we could drop reports if we change from one network to another. [WICG/background-sync Issue #107]

9.2. Clock Skew

Each report is delivered along with an age property, rather than the timestamp at which it was generated. We do this because each user’s local clock will be skewed from the clock on the server by an arbitrary amount. The difference between the time the report was generated and the time it was sent will be stable, regardless of clock skew, and we can avoid the fingerprinting risk of exposing the clock skew via this API.

9.3. Cross-origin correlation

If multiple origins all use the same reporting endpoint, that endpoint may learn that a particular user has interacted with a certain set of websites, as it will receive origin-tagged reports from each. This doesn’t seem worse than the status quo ability to track the same information from cooperative origins, and doesn’t grant any new tracking ability above and beyond what’s possible with <img> today.

9.4. Disabling Reporting

Reporting is, to some extent, a question of commons. In the aggregate, it seems useful for everyone for reports to be delivered. There is direct benefit to developers, as they can fix bugs, which means there’s indirect benefit to users, as the sites they enjoy will be more stable and enjoyable. As a concrete example, Content Security Policy grants something like herd immunity to cross-site scripting attacks by alerting developers about potential holes in their sites' defenses. Fixing those bugs helps every user, even those whose user agents don’t support Content Security Policy.

The calculus, of course, depends on the nature of data that’s being delivered, and the relative maliciousness of the reporting endpoints, but that’s the value proposition in broad strokes.

That said, it can’t be the case that this general benefit be allowed to take priority over the ability of a user to individually opt-out of such a system. Sending reports costs bandwidth, and potentially could reveal some small amount of additional information above and beyond what a website can obtain in-band ([NETWORK-ERROR-LOGGING], for instance). User agents MUST allow users to disable reporting with some reasonable amount of granularity in order to maintain the priority of constituencies espoused in [HTML-DESIGN-PRINCIPLES].

10. IANA Considerations

10.1. The Reporting-Endpoints Header

The permanent message header field registry should be updated with the following registration: [RFC3864]

Header field name

Reporting-Endpoints

Applicable protocol

http

Status

standard

Author/Change controller

W3C

Specification document

This specification (see § 3.2 The Reporting-Endpoints HTTP Response Header Field)

10.2. The application/reports+json Media Type

Type name

application

Subtype name

reports+json

Required parameters

N/A

Optional parameters

N/A

Encoding considerations

Encoding considerations are identical to those specified for the "application/json" media type. See [RFC8259].

Security considerations

See § 8 Security Considerations.

Interoperability considerations

This document specifies the format of conforming messages and the interpretation thereof.

Published specification

§ 2.2 Media Type

Applications that use this media type

Fragment identifier considerations

Additional information

N/A

Person and email address to contact for further information

This document’s editors.

Intended usage:

COMMON

Restrictions on usage:

N/A

Author

This document’s editors.

Change controller

W3C

Provisional registration?

Yes.