Large-Scale Web Applications

Lecture Notes for CS 142
Fall 2010
John Ousterhout

  • Readings for this topic: none.
  • Scale of Web applications: 1000x anything previously built.
  • Load-balancing routers: replicate Web front ends.
    • DNS (Domain Name System) load balancing:
      • Specify multiple targets for a given name
      • DNS servers rotate among those targets
    • HTTP redirection (HotMail, now LiveMail):
      • Front-end machine accepts initial connections
      • Redirects them among an array of back-end machines.
    • Load-balancing switch:
      • All incoming packets pass through one switch, which dispatches them to one of many servers; once TCP connection established, load balancer will send all packets for that connection to the same server.
      • In some cases the switches are smart enough to inspect session cookies, so that the same session always goes to the same server.
  • How to handle session data?
    • Different requests may go to different servers
    • Individual servers may crash
    • Need for session data to move from server to server as necessary.
    • Solution #1: keep all session data in shared storage:
      • File system
      • Database
      • May be expensive to retrieve for each request
    • Solution #2: keep session data in cookies
      • No server storage required
      • Cookies limit the amount of data that can be stored
    • Solution #3: cache session data in last server that used it
      • Store server map in shared storage
      • If future request goes to different server, use map to find server holding session data, retrieve data from previous server.
  • Scaling the storage system:
    • Almost all Web applications start off using relational databases.
    • A single database instance doesn't scale very far.
    • Applications must partition data among multiple independent databases, which adds complexity.
    • Memcache: main-memory caching system
      • Key-value store (both keys and values are arbitrary blobs)
      • Used to cache results of recent database queries
      • Much faster than databases: 500-microsecond access time, vs. 10's of milliseconds
      • Problems:
        • Writes must still go to the DBMS, so no performance improvement for them
        • Must manage consistency in software (e.g., flush relevant memcache data when database gets modified)
      • Example: Facebook has 2000 memcache servers
  • Because of scalability problems, we are seeing many new approaches to storage:
    • RAMCloud: new storage system under development in a research project here at Stanford:
      • Store all data in DRAM permanently
      • Aggregate thousands of servers in a datacenter
      • Use disk to backup data for high durability and availability
      • 32-64 GB per server
      • 100-500 TB per system
      • Potentially very high performance:
        • 5-10 microsecond access time
        • 1 million operations/second/server
  • Scaling issues make it difficult to create new Web applications:
    • Initially, can't afford expensive systems for managing large scale.
    • But, application can suddenly become very popular ("flash crowd"); can be disastrous if scalability mechanisms are not in place.
    • Can take weeks or months to buy and install new servers.
    • Must become expert in datacenter management.
    • Each 10x growth in application scale typically requires new application-specific techniques.
  • Cloud computing:
    • Separate scalability issues from application development.
    • Specialized providers offer scalable infrastructure.
    • Just pay for what you need.
  • Example #1: Amazon Web Services
    • Elastic Compute Cloud (EC2): rent CPUs in an Amazon datacenter for $0.10/CPU/hour
    • Scale up and down by hundreds of CPUs almost instantly
    • Simple Storage Service (S3): stores blobs of data inexpensively ($0.12-$0.15/GB/month).
    • AWS provides low-level facilities; users still have to worry about various management issues ("how do I know it's time to allocate more CPUs?")
  • Example #2: Google AppEngine
    • Much higher level interface:
      • You provide pieces of Python or Java code, URLs associated with each piece of code.
      • Google does the rest:
        • Allocate machines to run your code
        • Arrange for name mappings so that HTTP requests find their way to your code
        • Scale machine allocations up and down automatically as load changes
      • AppEngine also includes a scalable storage system
    • More constrained environment
      • Must use Python or Java
      • Must use specialized Google storage system
  • In the future we are going to see more systems like AWS and AppEngine, with more and more convenient high-level interfaces.