CSL862: Spl. Topics in Computer Systems: Virtualization and Cloud Computing

Sem I, 2010-11

Programming Assignments

1. Binary Translation inside Qemu
2. Write an application designed to scale over cloud
3. Map-Reduce

Lecture Slides

• Introduction [PDF]
• Virtual Machines [PDF]
• Amazon EC2 [PPTX]
• Google AppEngine [PPTX]
• Microsoft Azure [PPTX]
• Security [PPT]
• GFS [PPTX]
• MapReduce [PPTX]
• Hadoop [PDF]
• BigTable [PPTX]
• Locality Sensitive Hashing [PDF, PDF (internal only)]
• Application Virtualization [PPTX]

Lecture Notes

Exams

• Minor 1
• Minor 2

Readings

Virtual Machines

• Compiling Java Just in Time, Cramer et al., Sun Microsystems 1997
• QEMU, a Fast and Portable Dynamic Translator, Fabrice Bellard, USENIX Annual Technical Conference 2005.
• Xen and the Art of Virtualization, Pratt et al. SOSP 2003.
• Scale and Performance in the Denali Isolation Kernel, A. Whitaker et al. OSDI 2002.
• LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation, Lattner and Adve, CGO 2004.
• Disco: Running Commodity Operating Systems on Scalable Multiprocessors, E. Bugnion, S. Devine, M. Roseblum. SOSP 97.

I/O Virtualization

• Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor, Sugerman et al. USENIX 2001.
• virtio: Towards a De-Facto Standard for Virtual I/O Devices, R. Russell. SIGOPS Op. Sys. Review. 2008
• Unmodified Device Driver Reuse and Improved System Dependability via Virtual Machines, J. LeVasseur et al. OSDI 2004
• Live Migration for Passthrough Devices, Kadav and Swift. WIOV 2008
• Performance Comparison of Virtual Network Devices, VMware ESX Server 3.5.

Memory Virtualization

• Memory Resource management in VMware ESX Server, Carl Waldspurger. OSDI 2002.
• Accelerating Two-Dimensional Page Walks for Virtualized Systems, R. Bhargava, B. Serebrin, F. Spadini, S. Manne. ASPLOS 08.
• AMD-V Nested Paging, AMD White Paper, July 2008
• Peformance Evaluation of AMD RVI Hardware Assist, VMware ESX 3.5.
• Performance Evaluation of Intel EPT Hardware Assist, VMware ESX 3.5.

Processor Virtualization

• A Comparison of Software and Hardware Techniques for x86 Virtualization, K. Adams, O. Agesen. ASPLOS 2006.
• Performance Aspects of x86 Virtualization, Ole Agesen. VMworld 2007.
• Analysis of the Intel Pentium's Ability to Support a Secure Virtual Machine Monitor, J.S. Robin and C.E. Irvine. USENIX Security Symposium 2000.
• Pre-Virtualization: Soft Layering for Virtual Machines, Levasseur et al. ACSAC 2008.
• Software and Hardware Techniques for x86 Virtualization, VMware ESX.

Applications of Virtual Machines

• ReVirt: Enabling Intrusion Detection through Virtual-Maching Logging and Replay, Dunlap et al. OSDI 2002.
• SubVirt: Implementing Malware with Virtual Machines, S. King et al. S&P 2006.
• Desktop Virtualization (MokaFive, XenDesktop, VMware ACE)
• Replay Debugging, Replay Solutions
• VMware High Availability

Nested Virtualization

• The Turtles Project: Design and Implementation of Nested Virtualization, Ben-Yehuda et al. OSDI 2010.

Recording Non-Determinism vs Enforcing Determinism

• ReTrace: Collecting Execution Trace with Virtual Machine Deterministic Replay, M. Xu et al., VMware Inc. Workshop on Modeling, Benchmarking and Simulation (MoBS), June 2007.
• Debugging Operating Systems with Time-Traveling Virtual Machines, S. King et al., U. Michigan. USENIX Technical Conference 2005.
• Execution Replay for Multiprocessor Virtual Machines, G. Dunlap et al., U. Michigan, VEE 2008.
• Efficient System-Enforced Deterministic Parallelism, B. Ford et al. Yale University. OSDI 2010.
• CoreDet: A Compiler and Runtime System for Deterministic Multithreaded Execution, T. Bergan et al., U. Washington. ASPLOS '10.
• Capo: A Software-Hardware Interface for Practical Deterministic Multiprocessor Replay, P. Montesinos et al. UIUC. ASPLOS '09.
• Kendo: Efficient Deterministic Multithreading in Software, S. Amarasinghe et al. MIT. ASPLOS 09.
• DMP: Deterministic Shared Memory Multiprocessing, J. Devietti et al. U. Washington. ASPLOS 2009.
• Deterministic Consistency: A Programming Model for Shared Memory Parallelism, A. Aviram, B. Ford. Technical Report '10.
• VMware KB: Processors and guest operating systems that support VMware Fault Tolerance

Timekeeping in Virtual Machines

• Timekeeping in VMware Virtual Machines, VMware Workstation 6.5
• Virtualize Everything but Time, Broomhead et al., OSDI 2010

Translation Across ISAs

• Binary Translation Using Peephole Superoptimizers, S. Bansal and A. Aiken. OSDI 2008

Parallel Programming Models

• On the Duality of Operating System Structures, H. Lauer, R. Needham. Operating System Review 1979.
• Cilk 5.4.6 Reference Manual, MIT.
• Trade-Offs in Transactional Memory Virtualization, J. Chung et al. Stanford University. ASPLOS '06.
• X10: An Object-Oriented Approach to Non-Uniform Cluster Computing, P. Charles et al. IBM Research. OOPSLA '05.

Managing "Cloud-Scale" Data

• The Google File System, S. Ghemawat et al. Google Inc. SOSP 2003.
• Bigtable: A Distributed Storage System for Structured Data, Fay Chang et al. Google Inc. OSDI 2006
• MapReduce: Simplified Data Processing on Large Clusters, J. Dean et al. Google Inc. OSDI 2004.
• Apache Hadoop

Cloud Computing

• BOOK: Enterprise Cloud Computing: Technology, Architecture, and Applications. Gautam Shroff (draft copy, will be put offline soon, once published).
  • Table of Contents
  • Enterprise Computing: A Retrospective
  • The Internet as a Platform
  • Software as a Service and Cloud Computing
  • Enterprise Architecture: Role and Evolution
  • Cloud Computing Platforms
  • Cloud Computing Economics
  • Web Services, AJAX, and Mashups
  • Virtualization Technology
  • Multi-Tenant Software
  • Data in the Cloud
  • MapReduce and Extensions
  • Dev 2.0 Platforms
  • Enterprise Software: ERP, SCM, CRM
  • Custom Enterprise Applications and Dev 2.0
  • Workflow and Business Processes
  • Enterprise Analytics and Search
  • Enterprise Cloud Computing Ecosystem
  • Roadmap for Enterprise Cloud Computing

Present-day Clouds

• Amazon EC2/S3
• Google AppEngine
• Windows Azure

Cloud Management Tools

• VMware vSphere
• Eucalyptus Systems
• Private Clouds