DDT Debugging Guide
Last update: June 28, 2018
DDT is a symbolic, parallel debugger providing graphical debugging of MPI, OpenMP, and Pthreads applications.
Before running any debugger, the application code must be compiled with the "
-g" and "
-O0" options as shown below:
login1$ mpif90 -g -O0 mycode.f90
login1$ mpiCC -g -O0 mycode.c
Enable X11 forwarding: To use the DDT GUI, ensure that X11 forwarding is enabled when you
sshto the TACC system. Use the "
-X" option on the
sshcommand line if X11 forwarding is not enabled in your
sshclient by default.
localhost$ ssh -X firstname.lastname@example.org
Then load the DDT module on the remote system along with any other modules needed to run the application:
$ module load ddt mymodule1 mymodule2
Start the debugger
$ ddt myprogram
If this error message appears:
ddt: cannot connect to X server
then X11 forwarding was not enabled or the system may not have local X11 support. If logging in with the "
-X" flag doesn't fix the problem, please contact the help desk for assistance.
Click the "Run and Debug a Program" button in the "DDT - Welcome" window.
This displays the "DDT-Run" window, where you specify the executable path, command-line arguments, and processor count. Once set, they remain from one session to the next.
Select each of the "Change" buttons in this window, and adjust the job parameters, as follows:
For the "Options" change button, set the MPI Implementation to either "mvapich 1", "mvapich 2", or "openmpi", depending on which MPI software stack you used to compile your program. Click OK.
For the "Queue Submission Parameters" window, fill in the following fields:
Queue default queue is "
Time (hh:mm:ss) Project Allocation to charge the batch job to Cores per node Actually, this is MPI tasks/node
You must set the Project field to a valid project id. When you login, a list of the projects associated with your account and the corresponding balance should appear. The "Cores per node" setting controls how many MPI tasks will run on each node. For example, if you wanted to run with 4 MPI tasks per node and 4 OpenMP threads per task, this would be set to "4way". Click OK, and you will return to the "DDT-Run" window.
Back in the "DDT-Run" window, set the number of tasks you will need in the "Number of processes" box. If you are debugging an OpenMP program, set the number of OpenMP threads also.
Finally, click "Submit". A "DDT-Job Submitted" status box will appear.
When your job is launched by the SLURM scheduler, the DDT GUI will fill up with a code listing, stack trace, local variables and a project file list. Double click on line numbers to set break points, and then click on the play symbol (>) in the upper left corner to start the run.
By starting DDT from a login node you let it use X11 graphics, which can be slow. Using a VNC connection or the visualization portal is faster, but has its own annoyances. Another way to use DDT is through a "remote client" using "reverse connect". The remote client is a program running entirely on your private machine, and the reverse connection means that the client is contacted by the DDT program on the cluster, rather than the other way around.
Download and install a remote client: https://www.allinea.com/developer-tools/debug-allinea-ddt/download. The client version and the DDT version on the TACC cluster have to correspond. Since the latest client is 18.1.x whereas TACC has DDT 18.0.x installed, you need to follow the link for older versions.
Under "Remote Launch" make a new configuration:
Fill in your login name and the cluster to connect to, for instance
stampede2.tacc.utexas.edu. The remote installation directory is stored in the "
TACC_DDT_DIR" environment variable.
Make the connection; you will be asked for your password and two-factor code:
From any login node, submit a batch job where the mpirun line is replaced by:
$ ddt --connect -n 123 ./yourprogram
When your batch job (and therefore your ddt execution) starts, the remote client will ask you to accept the connection:
Your DDT session will now use the remote client.
The DDT user guide is available under the "Help" menu item.