Spring 1999 CS594

Spring 1998

Spring 1998 CS594

CS 594 Understanding Parallel Architectures:
From Theory To Practice
Spring 1998 - 3 credits - Room 655 Buehler

Jack Dongarra, Professor; Shirley Browne, Adjunct Professor; and Erich Strohmaier, Research Assistant Professor

Email: and
Phone: 423-974-8295
Fax: 423-974-8296
Office hours: Wednesday 11:00 - 1:00, or by appointment

TA: Song Jin,, Rm 217 Ayres Hall

Phone: 423-974-4247

To find out more about this Course click here.

For the Course handout click here.
For the Course survey click here.
Lecture Notes: (Tentative outline of the class)
  • Jan 14
    Course Introduction
    Basics of performance evaluation of parallel systems
  • Jan 21
  • Jan 28
    Overview of High-Performance Computing
    Reading on Overview of Scientific Computing
  • Feb 4
  • Feb 11
    Memory Hierarchy
  • Feb 18
    Blocked linear algebra
    IBM RS6000 Algorithms and Architecture
    IBM RS6000 590
  • Feb 25
    Linear Algebra Algorithms (part a)
  • Mar 4
    Performance Modeling of parallel applications
  • Mar 11
    Scalability Analysis
  • Mar 18
    Linear Algebra Algorithms (part b)
    Architectures for High-Peformance Computing
  • Mar 25 Spring break
  • Apr 1
    Class Reports
  • Apr 8 Parallel Signal and Image Processing
  • Apr 15 Parallel Signal and Image Processing
  • Apr 22 Repository stuff and Parallel Debugging
  • Apr 29 Parallel Performance Analysis Tools
  • May 4 Final exam

  • Assignments
  • Assignment 1 (due January 21, 1998) click here.
  • Assignment 2 (due January 28, 1998) click here.
  • Assignment 3 (due February 11, 1998) click here. Solution set for Assignment 3, click here.
  • Assignment 4 (due February 18, 1998) click here.
  • Assignment 5 (due February 25, 1998) click here.
  • Assignment 6 (due March 11, 1998) click here.
  • Assignment 7 (due March 18, 1998) click here. Solution set for Assignment 7, click here.
  • Projects (due April 1, 1998) Preliminary list of projects click here.

  • Class Roster If your name is not on the list or some information is incorrect, please send mail to TA: Song Jin,

    Student Name Email addresss Phone Enrollment Department Acedemic interests

    On-line Documentation and Information about Machines

  • Cray
  • IBM RS6000
  • Intel
  • Intel ASCI Red Paragon
  • SGI Power Challenge
  • Solaris Threads page
  • Catalog of Commercial Hardware and Software Vendors
  • Convex
    • Exemplar
    • Michielse, P. Programming the Convex Exemplar Series SPP system. Parallel Scientific Computing. First International Workshop, PARA '94. Proceedings. Lyngby, Denmark, 20-23 June 1994). Edited by: Dongarra, J.; Wasniewski, J. Berlin, Germany: Springer-Verlag, 1994. p. 374-82.
  • Cray Research
  • Digitial Equipment Corporation ( System Info
  • Hewlett-Packard
  • IBM
  • Sequent
    • Symmetry 5000
    • NUMA-Q
    • Raetz, G.M., Sequentz, G.M., Sequent general purpose parallel processing system, Northcon/87. Conference Record. Portland, OR, USA, 22-24 Sept. 1987)
  • Silicon Graphics
    • Power Challenge
    • Power Challenge (Techical Report)
    • Challenge XL
    • Galles, M.; Williams, E. Performance optimizations, implementation, and verification of the SGI Challenge multiprocessor. Proceedings of the Twenty-Seventh Hawaii Internation Conference on System Sciences Vol. I: Architecture, Wailea, HI, USA, 4-7 Jan. 1994, Edited by: Mudge, T.N.; Shriver, B.D. Los Alamitos, CA, USA: IEEE Comput. Soc. Press, 1994. p. 134-43.
    • Power Series
  • Sun Microsystems

  • Other Parallel Information Sites
  • NHSE - National HPCC Software Exchange
  • Netlib Repository at UTK/ORNL
  • BLAS Quick Reference Card
  • GAMS - Guide to Available Math Software
  • Center for Research on Parallel Computation (CRPC)
  • Supercomputing & Parallel Computing: Conferences
  • Supercomputing & Parallel Computing: Journals
  • High Performance Fortran (HPF) reports
  • High Performance Fortran Resource List
  • Fortran 90 Resource List
  • CMU's list of supercomputing and parallel computing resources
  • J. Wang's Parallel Computing List.
  • Major Science Research Institutions from Caltech
  • Message Passing Interface (MPI) Forum
  • High Performance Fortran Forum
  • PVM
  • Parallel Tools Consortium
  • DoD High Performance Computing Modernization Program
  • DoE Accelerated Strategic Computing Initiative (ASCI)
  • National Computational Science Alliance

  • Related On-line Textbooks
  • Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM Publication, Philadelphia, 1994.
  • PVM - A Users' Guide and Tutorial for Networked Parallel Computing, MIT Press, Boston, 1994.
  • MPI : A Message-Passing Interface Standard
  • LAPACK Users' Guide (Second Edition), SIAM Publications, Philadelphia, 1995.
  • MPI: The Complete Reference, MIT Press, Boston, 1996.
  • Using MPI: Portable Parallel Programming with the Message-Passing Interface by W. Gropp, E. Lusk, and A. Skjellum
  • Parallel Computing Works, by G. Fox, R. Williams, and P. Messina (Morgan Kaufmann Publishers)
  • Computational Science Education Project TextBook.
  • Designing and Building Parallel Programs. A dead-tree version of this book is available by Addison-Wesley.
  • High Performance Fortran (HPF), a course offered by Manchester and North High Performance Computing Training & Education Centre, United Kingdom
  • Parallel Processing Laboratory, Colorado School of Mines
  • The following two texts have sections on computational geometry:

  • F. Thomson Leighton, Introduction to Parallel Algorithms and Architectures: Arrays, Trees, and Hypercubes. Morgan Kaufmann, 1992.
  • John Reif, Synthesis of Parallel Algorithms. Morgan Kaufmann, 1993.
  • For performance analysis:

  • Raj Jain, The Art of Computer Systems Performance Analysis. John Wiley, 1991.
  • Papers on performance analysis tools:

  • Ruth A. Aydt, "The Pablo Self-Defining Data Format," November 1997, click here.
  • Daniel A. Reed, Ruth A. Aydt, Tara M. Madhyastha, Roger J. Noe, Keith A. Shields, and Bradley W. Schwartz, "Pablo: An Extensible Performance Analysis Environment for Parallel Systems", November 1992, click here.
  • Jeffrey K. Hollingsworth, Barton P. Miller, Marcelo J. R. Gongalves, Oscar Naim, Zhichen Xu and Ling Zheng, "MDL: A Language and Compiler for Dynamic Program Instrumentation", International Conference on Parallel Architectures and Compilation Techniques, San Francisco, CA, November 1997, click here.
  • Barton P. Miller, Mark D. Callaghan, Jonathan M. Cargille, Jeffrey K. Hollingsworth, R. Bruce Irvin, Karen L. Karavanic, Krishna Kunchithapadam and Tia Newhall. "The Paradyn Parallel Performance Measurement Tools", IEEE Computer 28(11), (November 1995). click here.
  • Steven T. Hackstadt and Allen D. Malony, "Distributed Array Query and Visualization for High Performance Fortran, February 1996, click here.
  • Jerry Yan and Sekhar Sarukkai and Pankaj Mehra, "Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS toolkit", Software Practice and Experience 25(4), April 1995, 429--461
  • Jack Dongarra