Privacy-respecting usage metrics

Will Thompson

3rd February 2024

Who am I?

Engineer at Endless OS Foundation

GNOME contributor

“Metrics”?

“Privacy-respecting”?

Why?

Ask the users?

“If all your friends jumped into a lake, would you do it too?”

We can do better

No personal data

DNF Count Me

GET https://mirrors.fedoraproject.org/metalink
        ?repo=fedora-39
        &arch=x86_64
        &countme=2

User-Agent: libdnf (Fedora 39; workstation; Linux.x86_64)

User-Agent: rpm-ostree (Fedora 39; silverblue; Linux.aarch64)

hits version os_variant os_arch sys_age
6771 36 generic x86_64 > 6mo
21906 38 workstation x86_64 2–6mo
20138 38 cloud aarch64 1w
6710 38 server x86_64 > 6mo
6393 39 container x86_64 2-6mo
1013 39 silverblue x86_64 2-6mo
︙

3 main ideas

Fixed event frequencies

Static segments

Client-side aggregation

Endless OS metrics system

Local event recording

On end-user device

Timestamp 2024-01-12 21:43:39Z
OS Version 5.1.0
Event UUID 927d0f61-4890-4912-a513-b2cb0205908f (“Updater Failure”)
Payload "Error fetching update: Delta requires 1,9 GB free space, but only 492,7 MB available"

Date 2024-01-30
OS Version 5.2.0
Event UUID 49d0451a-f706-4f50-81d2-70cc0ec923a4 (“Daily App Usage”)
Payload "org.gnome.Terminal.desktop"
Counter 14657 (i.e. 4h 4m 17s)

Submission

Events submitted to server

Channel

Roughly: how the user got their system.

  • Original installation image identifier
  • Standalone install / Dual-boot install / Live system

Image identifier

product download, OEM, impact partnership, custom-built, …
branch eos3.9, eos5.1, …
arch amd64, arm64, …
platform amd64, rpi4, pinebookpro, …
timestamp 240103-034925, 220825-180956, …
personality es_MX, base, $partnername, …

Downloadable French version

product eos
branch eos5.1
arch amd64
platform amd64
timestamp 240103-025438
personality fr

My laptop

product eosnonfree
branch eos3.6
arch amd64
platform amd64
timestamp 190614-110534
personality en

Server-side processing

Ingestion

One table per event

Linked to channel, not device

On by default

What have we learned?

Parental Controls

Tour

const TOUR_RESPONSE_EVENT='140643be-fe47-4b4b-985b-d16f8f3973a9'
// ...
_sendResponse(response) {
  let payload = new GLib.Variant('b',
    response == DialogResponse.TAKE_TOUR);
  EosMetrics.EventRecorder.get_default()
    .record_event(TOUR_RESPONSE_EVENT, payload);

  if (response === DialogResponse.TAKE_TOUR) {
    this._tourAppInfo.launch(0, -1, Shell.AppLaunchGpu.APP_PREF);
    Main.overview.hide();

class TourResponse(SingularEvent):
    __tablename__ = 'tour_response_v3'
    __event_uuid__ = '140643be-fe47-4b4b-985b-d16f8f3973a9'
    __payload_type__ = 'b'

    took_tour = Column(Boolean, nullable=False)

    @staticmethod
    def _get_fields_from_payload(payload):
        return {
            'took_tour': payload.get_boolean(),
        }

Shortcomings & Future Ideas

Quite annoying to not be able to correlate separate events!

Indeterminate upload latency

How do you know when you have as much data about last month as you are going to get?

Trim excess precision

  • Timestamps
  • Durations

Randomised responses

More questions

  • Are most desktop systems single-user?
  • Common monitor configurations
  • Workspace usage patterns
  • GNOME Shell extensions in use
  • …

Data from other GNOME distros?

Public aggregate data & dashboards

Prior art:

Year In Review?

You were in the top 5% of GNOME Builder users this year!

In summary

Telemetry doesn’t have to be creepy

Lead by example

Data + vision

Telemetry BOF

Sunday, 11:00–12:00

AW1.121 (BOF – Track C)

Questions?

@wjt@mastodon.me.uk

blogs.gnome.org/wjjt

azafea · eos-event-recorder-daemon

endlessos.org