bchop Utility - Collected Slides

Slide 1

1 Title bchop Utility - Title

R.K.Owen,Ph.D.,

National Energy Research Scientific Computer Center
NERSC / MS 50C
One Cyclotron Road
Berkeley, CA 94720
(510)604-5935
rk@owen.sj.ca.us
rkowen@nersc.gov

TITLE

"So Optimization Breaks Your Code ..."

Slide 2

1.1 Abstract bchop Utility - Abstract

TITLE
"So Optimization Breaks Your Code ..."
ABSTRACT

Many compiler options can have an adverse affect on a code. Most notably, various levels of optimization can "break" a code, such that the program gives erroneous results. One option is to compile for the lowest common denominator. The other option is to identify which routines are adversely affected. The local utility, bchop performs a binary chop between object files in two directories (./good and ./bad) compares the output and isolates which object files are causing the differences in output. This helps isolate problems due to changing compiler or preprocessor options. The number of runs performed is approximately 2 E log₂(N), where N is the number of object files and E is the number of object modules causing errors. This utility, bchop, is similar to CRI's atchop for isolating multitasking problems.

Slide 3

1.2 Why? bchop Utility - Why?

Scenario

User gets different results depending on compiler options (debugging vs. optimization)
Code has 117 compilation units
Each execution takes several minutes of C90 CPU time
Not a multitasking problem

Other Scenarios

f90 vs. cf77
Different compiler options
Can't run interactively

Solution

Write a general utility bchop

Slide 4

1.3 Atchop bchop Utility - atchop

bchop just a atchop knock-off?

atchop bchop

Finds multitasking problems only
Can isolate problem loops
Recompiles & loads code each time
Specific to cf77
CPU intensive
Not easy to use

Finds which modules are bad
Modules compiled only once
Loads code each time
Independent of compiler
Less CPU intensive
easier to use
Portable

Slide 5

2 Binary Decomposition bchop Utility - Binary Decomposition

AKA ``interval halving''

Each object file = endpoint in binary tree
O(log(N)) to traverse tree
Order object files
Select according to LSBs
Take subset from ``bad''
Take remainder from ``good''

Slide 6

2.1 Simple Example bchop Utility - Binary Decomposition Example

Order object files
Select ``bad'' according to LSBs
Increase selection level if overall failure

Slide 7

2.2 Single Error bchop Utility - Single Error

Single error (E = 1)
Number of compilation units (N = 16)
Number of comparisons = 2 E log ₂ (N)

Slide 8

2.3 Multiple Errors bchop Utility - Multiple Errors

Multiple errors (E = 2)
Number of compilation units (N = 16)
Number of comparisons < 2 E log ₂ (N)

Slide 9

3 Number of Trials bchop Utility - Number of Trials

Generally,
Number of Trials <= 2Elog₂(N) Where

E = Number of errant object files
N = Number of object files
Efficient if E << N

Slide 10

3.1 Data plot bchop Utility - Data

1,2, and 3 errors
Up to 140 object files

Slide 11

4 How to Use bchop Utility - How to Use

Compare the ``good'' and the ``bad''
The ``good''

Compile the code with the ``good'' options
Make a directory - ./good
Move the object files to the ./good directory
The ``bad''

Compile the code with the ``bad'' options
Make a directory - ./bad
Move the object files to the ./bad directory
The ``ugly''

Create a file listing all affected object files
If necessary create a script to compare ``good'' & ``bad'' output
Collect all the necessary input files together

Slide 12

4.1 Options bchop Utility - Options

Can specify the following:

Custom script for output comparison
Which output for comparison
File for stdin
Executable command line options
Loader & options

Slide 13

4.2 Command Synopsis bchop Utility - Command Synopsis
 

usage: bchop [-v][-V][-1][-d diff_script] \
	[-o outfile][-i infile] \
        [-C cmd_line][-c load_cmd] \
	[-l libs][-L libdirs][-O okobjects] \
        -f objlistfile

 -v              verbose output
 -V              print version
 -1              no binary chopping...go one-by-one
 -d diff_script  script file takes 2 arguments and
                 exits 0 if no difference.
                 def: "diff -b $1 $2 > /dev/null"
 -o outfile      output file to compare.
                 def: stdout
 -i infile       stdin file for runs.
                 e.g. "./a.out < infile"
 -C cmd_line     command line arguments for runs
 -c load_cmd     command to link objects.
                 def: "cc -o "
 -llibs          added libraries for linking objects
 -Llibdirs       additional directories to search
                 for libraries
 -O okobjects    added objects to link with
 -f objlistfile  list of objects to link together
 
Slide 14

5 Implementation bchop Utility - Implementation

Written in ``Standard C''
Compiles on

Crays
Workstations
PCs running Linux
Heavy use of system() function
bchop

Parsing of customization options
Bookkeeping of ``good'' and ``bad'' objects
Pasting strings together for commands
Clean-up

Slide 15

5.1 Future Work bchop Utility - Future Work

Restart capabilities
Tighten SIGINT handling
Multiple -l & -L options
Report Timings (-t)
Use ``bad'' (but OK) objects (-b)
?
Send suggestions (or code patches) to rk@owen.sj.ca.us
Source available by anonymous ftp at
ftp://owen.sj.ca.us/pub/rkowen/

Slide 16

6 Conclusion bchop Utility - Conclusion

Effective & simple tool, finds objects affected by

Change in compiler options
Upgrades of compiler
Change in operating system
User environment

Easy to use
Not CPU intensive
Was written in ~ 3 days
In original task ... found an object that caused differences not even considered by user.

atchop	bchop
Finds multitasking problems only Can isolate problem loops Recompiles & loads code each time Specific to cf77 CPU intensive Not easy to use	Finds which modules are bad Modules compiled only once Loads code each time Independent of compiler Less CPU intensive easier to use Portable