Tutorial on Performance Evaluation and Benchmarking
Outline
Tutorial Goals
Why do we benchmark?
Application Performance
Performance Prediction
System Optimization
Goals of Performance Modeling
Terms
Execution Time
Execution Time (continued)
Execution Time (Continued)
Communication Speed and Granularity
Theoretical Peak Performance
Why Never Obtained
What’s Wrong With Speedup T1/Tp ?
Speedup
PPT Slide
Strassen’s Algorithm for Matrix Multiply
Strassen’s Algorithm
Speciality/Instability Ratio
Communication Performance
Communication Latency
Log(P) Communication Parameters
Caveats
Discussion of and
Hockney Parameter
Machine Architecture
Amdahl’s Law
Message Passing Factors
Shared Memory Factors
Simple Performance Model
Benchmarking Methods
Current Status of Performance Modeling
Problems with Existing Benchmarks
Vision
Early Benchmark Efforts
Performance Evaluation
Algorithms and Architecture
Algorithm Issues
Blocking
Loop Unrolling
Limitations and Pitfalls
Future Development
The NAS Parallel Benchmarks (NPB)
Linpack Benchmark
TOP 500
Single numbers
The Perfect Club
SPEC Benchmark
The NAS Parallel Benchmarks
SPEC Suite 2 -- (SPEC95)
SPEC Organization
SPEC HPC -High Performance Group
Livermore Loops
Euroben
Genesis
RAPS Benchmarks
URLs
Email: dongarra@cs.utk.edu
Home Page: http://www.netlib.org/utk/people/JackDongarra/