CUDA is of course NVIDIA's toolkit and programming model which provides a development environment for speeding up computing applications by harnessing the power of GPUs. It's not easy to conquer, but here's a few resources to help.

Let's start with the "Cuda Learning" Github repository, promising to get you from basic to advanced in CUDA Programming.

The path emphasizes building a strong base in programming, understanding data structures, mastering C++, and diving into GPU architecture and CUDA-specific optimizations.

In fact it's a good course in general programming as it first goes through C and C++ programming before getting CUDA specific. A look at the curriculum will make that clear:

C Programming
Begin with C programming if you are unfamiliar with it. A solid understanding of C is mandatory before transitioning to
C++ programming.

Data Structures
Learn essential data structures and algorithms, a prerequisite for effective problem-solving and programming.

C++ Programming
Master C++ programming as it serves as a foundation for CUDA development.

Parallel Computing
Understand the basics of parallel computing and modern hardware architectures.

CUDA Programming
Dive into CUDA, learning GPU programming techniques, optimizations, and advanced performance tuning.

Triton, ThunderKittens, Tile-Lang
Explore the Triton, ThunderKittens, Tile-Lang frameworks for GPU programming with efficient performance.

GPU Architecture and Glossary
Familiarize yourself with GPU architecture and terminology to deepen your understanding of hardware capabilities.

It ends up with references to other useful resources, like the one we'll look into below; The "CUDA 120 days challenge".

That's another learning plan covering daily concepts, exercises, pitfalls, and references (including “Programming Massively Parallel Processors”). It should immediately follow after "Cuda Learning" because here you'll do hardcore coding CUDA implementations.

As such as the name says, it's a structured day-by-day plan to master NVIDIA CUDA programming over 120 days. Each day includes:

• Core Topics
• Practical Exercises / Mini-Projects
• Debugging Pitfalls
• Resource Suggestions
  
  

Six Capstone Projects are spread out at Days 20, 40, 60, 80, 100, and 120 to synthesize the skills acquired.

The curriculum is too long to list it here.

Finally, let's have a look at LeetGPU which is an online playground where you can write and run CUDA code. Makers AlphaGPU aim with it to "democratize access to hardware-accelerated languages" by emulating GPUs on CPUs using open-source simulators.

The available Simulation Modes it supports are:

• Functional Mode: Performs a basic simulation of your CUDA program, focusing on correctness and output verification. This is our fastest simulation option, ideal for testing program logic and debugging.
• Cycle Accurate Mode: Provides detailed architectural simulation by modeling the GPU hardware cycle-by-cycle. While slower, this mode offers valuable performance insights and helps predict how your code will perform on actual GPU hardware. The execution time in this mode correlates with real GPU performance.

Both simulation modes support core CUDA Runtime API features, including:

• Constant memory
• Global memory
• Shared memory
• CUDA Streams
• CUDA Events
• Atomic operations

It's a great companion to your studying of the two aforementioned resources, since you can run CUDA code without having to set up anything. The platform is also working on challenges which will be coming very soon.

These resources will get you started with CUDA programming. Along the way you'll also have to become acquainted with GPU terminology. We've got you covered with that too, with Demystifying GPU Terminology which delves into Modal's GPU Glossary that helps to get to grips with the terminology related to NVIDIA GPU hardware and software.

More Information

Cuda-120-Days-Challenge
Cuda-learning
LeetGPU

Related Articles

Demystifying GPU Terminology  

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info