CS 594-29622 Understanding Parallel Architectures:
From Theory To Practice
Spring 2002 - 3 credits - Room 205, Claxton

Jack Dongarra; with help from Micah Beck, Shirley Moore, Dave Cronk, Victor Eijkhout, and Dan Terpstra

Email: dongarra@cs.utk.edu

Office : 413 Claxton

Phone: 865-974-8295

Fax: 865-974-8296

Office hours: Wednesday 11:00 - 1:00, or by appointment

TA: Zizhong Chen  zchen@cs.utk.edu 

       Room 351 Claxton, 974-9985

 

Class Roster

If your name is not on the list or some information is incorrect, please send mail to TA: 
Student Names and Email addresses:

 

YIHUA

BAI

bai@cs.utk.edu

BRANT

BOEHMANN

boehmann@cs.utk.edu

YU

CHEN

ychen1@utk.edu

ZIZHONG

CHEN

zchen@cs.utk.edu

ROBERT

DAY

day@cs.utk.edu

JIN

DING

djin@cs.utk.edu

AMITABHA

GHOSH

aghosh@cs.utk.edu

DAVID

KATZ

katz@cs.utk.edu

WEIRAN

LI

wli@cs.utk.edu

XIANG

LI

xli@cs.utk.edu

JANE

RAZUMOVSKAYA

razumovs@cs.utk.edu

ZHIAO

SHI

shi@cs.utk.edu

DAN

TERPSTRA

terpstra@cs.utk.edu

PHAN

TRAN-Nan

phan@cs.utk.edu

BING

YU

yu@cs.utk.edu

JUNLONG

ZHAO

zhao@cs.utk.edu

 


To find out more about this Course click here.


Lecture Notes: (Tentative outline of the course)

·  January 9th:
     Introduction
     Overview of HPC

     Reading: Overview of Scientific Computing

     Homework: due January 23rd, 2002

·  January 14th; (Note: Lecture is Monday in Room 233 Claxton at 1:30)
     Introduction to MPI (Dave Cronk)

     Appendix A

     Appendix B

     Appendix C

     Homework: due January 30th, 2002

·  January 23rd
     If possible, attend the “Workshop On Grid Computing At UT

1/23, 8:30-12:00, Shilo Room, University Center

Slides from the workshop can be found at here.

·  January 30th

     Parallel Architectures and Programming

     Homework: due February 6th, 2002
     Floating Point Arithmetic

     Reading: Parallel Programming Considerations, Ken Kennedy, et al

     Reading: Overview of Recent Supercomputers, Aad J. van der Steen and Jack J. Dongarra

·  February 6th (this class will start at 12:30 pm and end at 2:30)
     Parallel Architectures and Programming and Floating Point Arithmetic

     Homework: due February 13th, 2002

·  February 13th
     Memory Hierarchy and Cache

     Homework: due February 27th, 2002

·  February 20th
     Linear Algebra

·  February 27th
     Linear Algebra (updated)

     Homework: due March 13th

     Strassen’s Algorithm

·  March 6th
     Sparse Methods in Linear Algebra (Victor Eijkhout)

·  March 13th
      Linear Algebra

·  March 20th
     Spring Break

·  March 27th
     Logistical Computing (Micah Beck)

     Homework

·  April 3rd
     Performance Analysis Tools (Shirley Moore)

     Homework

·  April 10th
     Grid Computing/NetSolve

     Homework: due April 24th

·  April 17th
    Performance Measurement with PAPI (Dan Terpstra)

    Homework: due May 1st

·  April 24th
    No Class

·  May 1st
     Class Project Reports:

1:00 Yihua Bai - Householder implementation

1:20 Zizhong Chen - Redistribution of data

1:40 Robert Day - Divide and Conquer for eigenvalues

2:00 break

2:10 Jin Ding - Data Storage using Xnode

2:30 Xiang Li - Different Givens' method implementations for factorization

2:50 Tran-Nan Phan - Conjugate Gradient

3:10 break

3:20 Zhiao Shi - FFT

3:40 Bing Yu - Instruction-level parallelism

4:00 Junlong Zhao - Different Parallel Matrix multiply implementations

 

 


  • Projects (due May 1, 2002)
    Preliminary list of projects click here.

Additional projects can be found here.


Additional Reading Materials


Message Passing Systems.

The PVM home page.


The MPI home page.
This is the best place to get info on MPI from, including implementations and the MPI forum itself.

The implementation that you should use, that is installed on the TORC (Tennessee Oak Ridge Cluster) cluster is MPICH .
A duplex postcript version of the MPI 1.1 API (with thanks to the LAM team) is available.


Other useful reference material

·         Processor specifications see http://www.geek.com/procspec/

    • A good introduction to message passing systems.


J.J. Dongarra, G.E. Fagg, R. Hempl and D. Walker, Chapter in Wiley Encyclopedia of Electrical and Electronics Enginerring,
to appear. ( postscript version )

``Message Passing Interfaces'', Special issue of Parallel Computing , vol 20(4), April 1994.

    • A paper by members of the PVM team on the differences between PVM and MPI.

Geist, G.A, J.A. Kohl, P.M. Papadopoulos, `` PVM and MPI: A Comparison of Features '', Calculateurs Paralleles , 8(2), pp. 137--150, June, 1996.

    • Papers by members of the MPI team on the differences between PVM and MPI.

``Why are PVM and MPI So Different'', William Gropp and Ewing Lusk (submitted to The Fourth European PVM - MPI Users' Group Meeting)

and

``PVM and MPI are completely different'', William Gropp and Ewing Lusk, to appear in the journal Future Generation Computer Systems, 1998.

 


Standard Books on Message Passing

``MPI - The Complete Reference, Volume 1, The MPI-1 Core, Second Edition'',
by Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker, Jack Dongarra, MIT Press, September 1998, ISDN 0-262-69215-5.

``Using MPI,''
by William Gropp, Ewing Lusk, and Anthony Skjellum, published by MIT Press, October 1994; ISBN 0-262-57104-8.

``MPI: The Complete Reference - 2nd Edition: Volume 2 - The MPI-2 Extensions'',
by William Gropp, Steven Huss-Lederman, Andrew Lumsdaine, Ewing Lusk, Bill Nitzberg, William Saphir, and Marc Snir, published by The MIT Press, September, 1998; ISBN 0-262-57123-4.

    •  

On-line Documentation and Information about Machines

Convex

        • Exemplar
        • Michielse, P. Programming the Convex Exemplar Series SPP system. Parallel Scientific Computing. First International Workshop, PARA '94. Proceedings. Lyngby, Denmark, 20-23 June 1994). Edited by: Dongarra, J.; Wasniewski, J. Berlin, Germany: Springer-Verlag, 1994. p. 374-82.
      •  

Cray Research

Digitial Equipment Corporation ( System Info )

Hewlett-Packard

        • Thomas B. Alexander, Kenneth G. Robertson, Deal T. Lindsay, Donald L. Rogers, John R. Obermeyer, John R. Keller, Keith Y. Oka, and Marlin M. Jones II. Corporate Business Servers: An Alternative to Mainframes for Business Computing. Hewlett-Packard Jornal, June 1994, pages 8-33.
        • HP-9000 K400, HP-9000 K200
        • HP-9000 T500
      •  

IBM

Silicon Graphics

        • Power Challenge
        • Power Challenge (Techical Report)
        • Challenge XL
        • Galles, M.; Williams, E. Performance optimizations, implementation, and verification of the SGI Challenge multiprocessor. Proceedings of the Twenty-Seventh Hawaii Internation Conference on System Sciences Vol. I: Architecture, Wailea, HI, USA, 4-7 Jan. 1994, Edited by: Mudge, T.N.; Shriver, B.D. Los Alamitos, CA, USA: IEEE Comput. Soc. Press, 1994. p. 134-43.
        • Power Series
      •  

Sun Microsystems


Other Parallel Information Sites


Related On-line Textbooks

For performance analysis:

      • Raj Jain, The Art of Computer Systems Performance Analysis. John Wiley, 1991.

Papers on performance analysis tools:

      • Ruth A. Aydt, "The Pablo Self-Defining Data Format," November 1997, click here.
      • Daniel A. Reed, Ruth A. Aydt, Tara M. Madhyastha, Roger J. Noe, Keith A. Shields, and Bradley W. Schwartz, "Pablo: An Extensible Performance Analysis Environment for Parallel Systems", November 1992, click here.
      • Jeffrey K. Hollingsworth, Barton P. Miller, Marcelo J. R. Gongalves, Oscar Naim, Zhichen Xu and Ling Zheng, "MDL: A Language and Compiler for Dynamic Program Instrumentation", International Conference on Parallel Architectures and Compilation Techniques, San Francisco, CA, November 1997, click here.
      • Barton P. Miller, Mark D. Callaghan, Jonathan M. Cargille, Jeffrey K. Hollingsworth, R. Bruce Irvin, Karen L. Karavanic, Krishna Kunchithapadam and Tia Newhall. "The Paradyn Parallel Performance Measurement Tools", IEEE Computer 28(11), (November 1995). click here.
      • Steven T. Hackstadt and Allen D. Malony, "Distributed Array Query and Visualization for High Performance Fortran, February 1996, click here.
      • Jerry Yan and Sekhar Sarukkai and Pankaj Mehra, "Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS toolkit", Software Practice and Experience 25(4), April 1995, 429--461
Jack Dongarra
1/7/2002