Question

What is a warp? What are the practical implications of having warps as part of a real GPU?

What are their normal sizes? Is this specific size defined in the CUDA standard? Why or why not?

Answer 1

Warp is said to be a set of threads that all share the same code, follow the same execution path and expected to stall at the same places. In other words, it is the smallest unit of execution on the device.

Practical Implementation:

• Singe uses warp specialization to partition the three most expensive kernels in a real-world combustion application.

• Warp-specializing compiler included the necessary algorithms for managing data placement, communication, and synchronization for general warp specialized kernels.

• High performing warp-specialized code are essential to avoid instruction cache thrashing.

Warp Size: Warp size is the number of threads in a warp that a multiprocessor executes concurrently. Normal sizes of the warp are 32 bytes, 64 bytes, 128 bytes

Yes, the CUDA standard defined the specific size of warp as 32.

you can use the following program to determine the warp size:

#include <stdio.h>

int main(void)

{ cudaDeviceProp deviceProp;

if (cudaSuccess != cudaGetDeviceProperties(&deviceProp, 0))

{

printf("Get device properties failed.\n");

return 1;

}

else

{

printf("The warp size is %d.\n", deviceProp.warpSize);

return 0;

}

}