Trust and Operating Systems

Trust and Operating Systems

Trust in General

What is "trust"?

  • When you stop questioning the dependability of a thing

Why trust?

  • Trust "extends our agency"
  • See slide for what is "agency"
  • Trust is empowering, improves efficiency: I can do more with trust than without
  • Trust is fundamental to all social systems
  • Example: blood glucose monitor with Bluetooth connection to phone
  • Trust is essential for mental health

Trust can be risky:

  • Dependence
  • Vulnerability: violation of trust can be deeply upsetting, even dangerous
  • Agential gullibility: trusting more than warranted
  • How to tell how much to trust?

How is trust achieved?

  1. Assumption: trust without any clues to warrant it
  2. Inference: various indicators that suggest trust is warranted
  • Some indicators are weaker, others stronger
  1. Substitution: structural arrangements that will compensate for misplaced trust
  • Backup plan

Trust and Software

Trust is essential for software, because we now depend on software for virtually every aspect of our lives:

  • Business
  • Transportation
  • Utilities: water, electricity, telecommunications, etc.
  • Science
  • Education
  • News, social interactions

Establishing trust in software:

  • Assumption: ineffective, not used
  • Inference: the path to trust is through distrust:
    • Code reviews, etc.
  • Subsitution: detect errors when they occur, correct if possible
    • Logging
    • Timeouts, consistency checks
    • Redundancy

Challenge: confirmation bias

  • Failure to scrutinize system when it seems to be behaving properly

Trust and Operating Systems

Operating systems provide the root of trust for software:

  • All software runs on top of OSes, so applications are only as trustworthy as the OS they run on
  • Operating systems implement mechanisms used by applications to ensure trust, such as security and protection

Example: the Linux kernel

  • Around since 1991
  • > 8M lines of code
  • > 15000 develepors
  • > 1.2M commits
  • Widely used in devices (e.g. Android smartphones), datacenters

How does trust emerge for:

  • Users trusting Linux:
    • Assumption
      • "Never thought about it"
      • "No other option"
    • Inference
      • General trust in open source software
        • Many eyes to detect and fix problems
      • Have used it before without problems
    • Substitution
      • 3rd party antivirus software
      • Replicate/encrypt important files
  • Application developers trusting Linux:
    • Assumption: rare
    • Inference:
      • Used by other app developers
      • GitHub stars
      • Trust Linus Torvalds
    • Substitution:
      • Code is open source
        • Read it
        • Clone the repo to fix bugs
  • Linux developer community trusting each other
    • Assumption: none (risks of bugs)
    • Inference:
      • Known in community
      • Previous patches were high quality
    • Substitution
      • Changes must be reviewed, accepted in layered process
      • Linus has final authority

Recent Trojan Horse discovered in the Linux ssh program:

  • Would have enabled attackers to gain access to any Linux system
  • Discovered before widely deployed, but only by chance
  • Agential gullibility enabled the attackers to insert the Trojan Horse:
    • Attack wasn't in ssh itself, but in a package xz used for encryption; ssh developers trusted xz.
    • Someone posed as a legitimate open source developer named Jia Tan for multiple years (since 2021).
    • Tan began to express impatience with the xz lead maintainer (not incorporating patches fast enough).
    • Multiple other developers (probably also phonies) pressured the lead maintainer to accept help from Tan.
    • Tan eventually was given permission to merge changes into xz.
    • He also submitted a pull request for OSS-Fuzz, which scans open-source package for malicious code; the patch disabled a check that would have exposed the Trojan Horse.

Recap

What is trust?

  • An unquestioning attitude
  • Beneficial because it extends agency

Trust is essential but risky

Trust emerges through:

  • Assumption
  • Inference (most powerful)
  • Substitution

Because of software's ubiquity and high impact, it is important for software to be trustworthy

This is even more true for operating systems