GPU Performance Showdown: MATLAB/Windows vs. CUDA on Tuned NixOS (RTX 4090/5090)

Introduction

Modern high-performance computing (HPC) relies heavily on GPUs to accelerate intensive tasks like matrix multiplication, convolution, and scientific simulations. However, not all software can squeeze maximum performance from GPU hardware. In this post, we compare GPU utilization and raw performance between MATLAB running on Windows and CUDA C/CUDA Fortran running on a tuned NixOS (Linux) environment. Using an NVIDIA RTX 4090 (and looking ahead to the 5090) as our baseline, we’ll explore theoretical vs. actual FLOPS, and explain why MATLAB’s abstraction often leaves GPU power untapped. We also examine how a highly optimized NixOS setup with low-level CUDA code can unlock full GPU throughput, and even touch on Python (NumPy/PyTorch) as a middle ground.

This post is aimed at PhD students and HPC researchers seeking technically precise insights. We’ll use accessible language, clear section headers, a comparison table, and real TFLOPS numbers to drive home the performance differences. Let’s dive in!

RTX 4090/5090: Theoretical Peak vs. Real HPC Performance

NVIDIA’s flagship GPUs offer staggering theoretical compute. The RTX 4090 (Ada Lovelace architecture) delivers about 82.6 TFLOPS of single-precision (FP32) throughput, while its successor, the RTX 5090 (Blackwell architecture), is expected to reach ~104.8 TFLOPS FP32 – roughly a 27% raw compute increase. These figures represent peak shader throughput under ideal conditions. In double precision (FP64), however, GeForce cards are vastly slower: the 4090 achieves only ~1.3 TFLOPS FP64 (just 1/64 of FP32 rate), since consumer GPUs prioritize graphics/AI workloads. Memory bandwidth is also a critical factor: the 4090 provides ~1 TB/s, and the 5090 is rumored to have ~1.8 TB/s (thanks to faster GDDR7 and a wider bus) – a 78% bandwidth boost beneficial for memory-bound tasks.

Theoretical vs. Measured: In reality, achieving those peak TFLOPS is challenging. Optimized GPU kernels (e.g., large matrix multiplications or batched convolutions) can reach a high fraction of peak, whereas poorly optimized code may only use a small percentage of the GPU’s potential. For instance, the High-Performance Linpack (HPL) benchmark (which stresses FP64 performance) manages about 1.18 TFLOPS on an RTX 4090 – impressive given the 4090’s FP64 limitation, but only ~1.4% of its FP32 theoretical peak. On memory-bound workloads (e.g. the HPCG conjugate gradient benchmark), a 4090 sustains ~162 GFLOPS (using ~1.3 TB/s effective memory throughput), similar to a 3090, since both are limited by memory speed rather than compute. These numbers highlight a key point: unleashing a GPU’s full power requires code that’s optimized and tailored to the hardware. High-level environments and OS overhead can make a huge dent in performance, as we’ll see next.

MATLAB on Windows: Ease of Use at the Cost of GPU Utilization

MATLAB, especially on a Windows workstation, offers scientists a convenient, high-level environment with powerful toolboxes. It can tap into GPU computing via the Parallel Computing Toolbox – allowing users to perform computations on a GPU using familiar MATLAB syntax (e.g. simply converting arrays to gpuArray). This abstraction, however, comes with limitations that prevent MATLAB from utilizing CUDA cores efficiently:

Default Double Precision: By default, MATLAB stores all numeric arrays in double precision. On GeForce GPUs like the RTX 4090, this is a recipe for under-utilization – the GPU’s FP64 throughput is 1/64 its FP32 rate (only ~1.3 TFLOPS vs 82.6 TFLOPS). Unless the user explicitly converts data to single precision (or the algorithm uses single internally), MATLAB code will be limited by the GPU’s paltry double performance. In other words, a naive MATLAB code could end up using <2% of the GPU’s compute capability simply due to precision mismatch. Many HPC tasks (e.g. simulations) demand double precision accuracy, which puts MATLAB at a disadvantage on gaming-class GPUs. (Professional GPU architectures like NVIDIA A100/H100 provide high FP64 throughput – but MATLAB on those is rare, and our focus is on 4090/5090 class hardware.)

Overhead and Abstraction Layers: Every time MATLAB dispatches work to the GPU, there is overhead from the MATLAB interpreter and driver. MATLAB cannot inline custom CUDA kernels in your M-file; you rely on built-in functions which internally call NVIDIA libraries. If you perform many small operations, the overhead of launching GPU kernels and transferring data can dominate. MathWorks documentation itself advises to minimize data transfers and keep work on the GPU to reduce overhead. For example, one should create gpuArray data upfront and avoid shuttling data between host and device inside loops. Still, the very nature of MATLAB – an interactive environment – means there’s often an implicit sync and transfer when you gather results. In complex workflows (with many function calls or iterative steps), GPUs may spend a lot of time idle waiting for MATLAB, resulting in low utilization.

Limited Control Over GPU Execution: MATLAB’s high-level functions don’t expose low-level optimizations. You can’t manually tune thread-block sizes, memory access patterns, or use specialized CUDA features. Want to utilize shared memory tiling in a custom way, or implement warp-synchronous algorithms? Not possible in pure MATLAB. There’s arrayfun to create elementwise GPU kernels and support for a subset of functions on GPU, but anything beyond that requires writing a MEX function in CUDA C++ (at which point you’re stepping outside MATLAB’s comfort zone). MATLAB also has fewer options for asynchronous execution – it generally waits for GPU computations to finish before continuing (unless you explicitly use parallel.gpu.wait or batch jobs), which limits overlapping of computation and data movement.

Multi-GPU and Scaling Challenges: Using more than one GPU in MATLAB is not straightforward. It typically requires the Parallel Computing Toolbox and gpuDevice management or parpool to assign different GPUs to different MATLAB workers. This is clunky compared to CUDA C/Fortran where you can directly program multiple GPUs (or use MPI across nodes). As one user put it, “it isn’t easy to use more than one GPU with MATLAB” for simulations. HPC workloads that need scaling to multi-GPU or cluster environments can thus outgrow MATLAB quickly.

Algorithmic Optimizations: Many advanced algorithms (e.g. custom solvers, FFT-based big convolutions, irregular computations) might not have optimized MATLAB equivalents. If an operation isn’t in MATLAB’s GPU-supported list, you’re stuck doing it on CPU or writing your own CUDA code. In contrast, lower-level approaches (C/Fortran with libraries) or even Python with its ecosystem often have a way to implement custom GPU kernels or use community-developed ones.

Bottom line: MATLAB on Windows makes GPU computing easy (“No need to learn low-level CUDA programming” touts one blog), but this ease comes at the expense of performance. It’s a bit like driving a sports car with an automatic transmission and speed governor – simple to operate, but you’re not accessing the engine’s full power. Many users report that translating MATLAB code to a lower-level language yields massive speedups. For example, a radar simulation in the MATLAB Phased Array toolbox was 10× to 4000× slower than an equivalent C++ implementation. Another report showed a satellite orbit simulation in MATLAB took 512 seconds vs. only 0.044 seconds in C++ (a difference of four orders of magnitude). These are CPU examples, but they illustrate how heavy abstraction can cripple performance. On the GPU side, one user ported MATLAB acoustic code to Python and saw similar CPU speed, but then achieved a 30× speedup using GPUs via CuPy – something that would be hard to replicate in MATLAB without significant effort. Clearly, if maximum performance is the goal, MATLAB becomes a limiting factor.

CUDA C/Fortran on NixOS: Unleashing Full GPU Performance

In stark contrast to MATLAB, coding in CUDA C or CUDA Fortran on a tuned NixOS Linux system gives you full control over the GPU and the software environment. This approach is more complex, but it’s the path to peak performance on RTX 4090/5090 hardware for HPC tasks:

Explicit Memory and Thread Management: With CUDA C/Fortran, you directly manage device memory (using CUDA malloc/memcpy or Unified Memory) and launch kernels with specified thread block sizes. This means you can optimize memory accesses, utilize shared memory, and ensure coalesced I/O in ways high-level tools cannot. You can overlap data transfers with computation using CUDA streams, achieving concurrency that MATLAB or Python (by default) often miss. Essentially, you have the keys to make the GPU work at near 100% utilization – issuing back-to-back kernels and data movement to keep all those CUDA cores busy.

Custom Kernels and Libraries: Need a special algorithm? Write a custom kernel tailored to your problem. You’re not limited to pre-baked functions – you are the chef. For many tasks, you can also leverage highly-tuned libraries (cuBLAS, cuFFT, cuDNN, etc.) directly. For example, you can call cuBLAS SGEMM to perform matrix multiplication and achieve ~90+% of theoretical TFLOPS on large matrices. In fact, well-optimized CUDA code often reaches 80–95% of hardware peak for compute-bound kernels (matmul, deep CNN convolutions, etc.), whereas high-level code might achieve lower percentages. The freedom to use low-level intrinsics (e.g., warp shuffle, tensor cores for FP16/INT8) means you can squeeze out every drop of performance. Nothing is abstracted away – which also means nothing is there to protect you from complexity, but HPC experts are usually willing to handle that for speed gains.

Optimized Compilation and System Tuning: Using a tuned NixOS environment provides a further edge. NixOS allows you to precisely configure compilers, libraries, and system settings for reproducibility and optimization. For HPC, you might compile code with architecture-specific flags (e.g., -arch=sm_89 for Ada GPUs, or -O3 -march=native for CPU parts of the code). Researchers at a supercomputing center using Nix have fine control, enabling optimized builds that leverage AVX512, FMA, and other hardware features. This means your CUDA C/Fortran code on NixOS can be compiled to run only the needed services and with maximally efficient instructions. Moreover, NixOS’s purely functional package management ensures that if you set up an HPC software stack (CUDA toolkit, MPI, math libraries), it’s consistent across systems – great for reproducibility and avoiding “it works on my machine” issues.

Lightweight OS, Better Scheduling and NUMA handling: A tuned NixOS (or any minimal Linux for that matter) typically has far less background bloat than a standard Windows install. No Cortana, no Windows Update popping up – nothing that could steal CPU cycles or cause jitter during your GPU computations. Linux also generally offers more predictable thread scheduling for HPC. For example, Linux can smartly keep a thread locked to a specific core (affinity) and handle multi-core, multi-socket NUMA allocation wisely. One Hacker News comment noted a “significant performance benefit” to leveraging NUMA on Linux for Threadripper CPUs, whereas on Windows the NUMA handling was suboptimal. In practice, this means on Linux your CPU threads feeding the GPU or doing post-processing can run at peak efficiency (with memory local to the core, less context switching). Better NUMA behavior is crucial in multi-CPU servers attached to GPUs – Linux/NixOS will let you pin processes to specific NUMA nodes close to the GPU (using numactl or systemd cpuaffinity), reducing latency. Windows HPC Server has made strides, but Linux remains the dominant choice on the TOP500 for these reasons.

Deterministic, Reproducible Environment: HPC researchers care about reproducibility. NixOS shines here by enabling immutable, declaratively-defined environments. You can specify exact versions of CUDA, driver, libraries, even kernel, and anyone (or any cluster node) can deploy the same configuration. This reduces the “it runs faster on machine X than Y” problem – if both run the same Nix environment, they behave identically. Reproducibility doesn’t directly make your code faster, but it means you can optimize once and deploy everywhere. (One computational scientist on r/NixOS noted that reproducibility was a huge reason for adopting NixOS for scientific workflows.)

In summary, using CUDA C/C++ or Fortran on a lean NixOS Linux gives full access to GPU power with minimal overhead. You’ll be programming closer to the metal and can achieve the theoretical performance in practice for well-optimized code. The trade-off, of course, is development time and complexity. It’s worth it when you need to run massive HPC workloads continuously or at scale. After all, when you’re trying to churn through billions of calculations per second, a high-level environment that only utilizes a fraction of the GPU is a bottleneck you can’t afford.

Performance Comparison: MATLAB vs Python vs CUDA (RTX 4090 Case Study)

Let’s quantify the performance differences on a few representative tasks. The table below compares MATLAB on Windows, Python (NumPy/PyTorch) on Linux, and CUDA C/Fortran on NixOS using an RTX 4090. We consider a compute-bound task (large dense matrix multiply) and a memory-bound task (convolution or sparse solver), using single-precision where applicable. All environments use the same GPU hardware; “% of Peak” refers to the fraction of the 4090’s 82.6 TFLOPS FP32 peak.

Notes: All figures are illustrative, assuming code is written in an optimal way for each environment. For the dense matrix multiply, MATLAB and Python ultimately call the same cuBLAS matrix-multiply kernel – so they can approach similar TFLOPS for a single large multiply. The differences arise if additional overhead or suboptimal data layout in MATLAB hinders reaching peak. In our estimate, MATLAB hits ~65–70% of peak in practice for 4096×4096 (still a huge speedup vs CPU, but trailing a bit behind a pure C implementation). Python with PyTorch or CuPy can be nearly as good as C since it’s just a thin layer over optimized CUDA routines (often 85–90% of peak). The tuned C/Fortran version can push slightly closer to the limit (~95%) by maximizing concurrency and minimizing overhead – effectively hitting ~78–80 TFLOPS on this problem (within a few TFLOPS of the theoretical max).

For the convolution (a more memory-bound operation), all approaches tend to be limited by memory throughput (~1 TB/s on 4090). Here, well-written code in any language that uses GPU will saturate the memory bus. PyTorch (with cuDNN library) and CUDA C can achieve near-hardware-limit bandwidth, whereas MATLAB might lag if its convolution isn’t as optimized or if overhead causes slightly less efficient usage. The table shows MATLAB at ~0.8 TB/s vs ~1.0 TB/s for others – in practice, if MATLAB’s conv2 is GPU-enabled, it likely calls an optimized algorithm too, but we assume a bit of overhead or non-optimal kernel could cost ~20% performance.

The multi-step simulation row is perhaps the most critical for HPC researchers. This represents a scenario like a time-stepping PDE solver, involving many successive operations on the GPU. In MATLAB, each step might be a separate GPU call (or worse, some parts not GPU-supported, forcing CPU work). The GPU could be idle a significant portion of the time waiting for MATLAB, resulting in maybe only 30% average utilization. Python fares better, especially if using Numba, CuPy or libraries to combine steps, but pure Python loops would also bottleneck the GPU. A carefully coded CUDA C/Fortran program can orchestrate the entire simulation loop on GPU memory, with non-stop kernel launches and overlapping I/O – yielding near 90%+ utilization of the GPU over the whole run. The net effect is that a simulation that runs for hours on MATLAB might run in minutes with a CUDA code, purely due to efficiency in using the GPU continuously. (Recall the earlier anecdote: a MATLAB satellite constellation sim took 512 seconds vs 0.044 seconds in C++ – while extreme, it underscores how dramatically a well-optimized low-level approach outperforms a high-level one in long-running workloads.)

Why NixOS for HPC? (Optimized Linux vs. Windows)

It’s worth highlighting why we chose a tuned NixOS environment for the CUDA tests instead of Windows or a generic Linux. In HPC, the operating system can influence performance. Windows is generally not the OS of choice for supercomputing – not because it can’t run fast, but because Linux offers flexibility and lower overhead. NixOS, in particular, allows declarative configuration, meaning you can strip down the OS to only what’s needed for your compute jobs. This leads to:

Lower OS jitter: fewer background processes mean your GPU and CPU aren’t interrupted by unrelated tasks. Compute nodes running NixOS can be configured to run at runlevel with essentially just your job and the bare essentials. This consistency yields more predictable latency for operations – a crucial factor when coordinating CPU threads and GPU tasks.

Better CPU utilization for HPC tasks: Linux gives fine-grained control over process scheduling and CPU affinity. As noted earlier, Linux’s NUMA-aware scheduling tends to outperform Windows for multi-core, memory-heavy workloads. You can lock a process to a specific core or socket, ensuring it uses the closest memory and doesn’t migrate (which could invalidate caches). Windows Server does have some of these capabilities (and Windows HPC editions exist), but most HPC practitioners are more familiar with Linux tooling (like mpirun, numactl, perf for profiling) which NixOS supports fully.

Compiler and library optimization: With Nix’s package manager, you can swap in the latest NVIDIA HPC SDK, or build your code with Intel’s compilers or GCC with specific flags easily, without messing up system libraries. You could, for example, have an OpenMPI built with CUDA-awareness and tuned for your InfiniBand network, all defined in your Nix configuration. Such consistency ensures that when you run across a cluster, each node has identical performance characteristics.

In short, NixOS provides a clean, controllable platform to maximize hardware performance, complementing the benefits of CUDA code. It’s not that you couldn’t achieve similar performance on Windows or another Linux – you can – but NixOS makes it easier to eliminate software as the bottleneck, so the focus is purely on algorithmic efficiency and hardware utilization.

Conclusion (TL;DR)

TL;DR: For GPU-heavy HPC workloads, MATLAB on Windows fails to tap the full power of modern GPUs like the RTX 4090/5090. High-level convenience comes at the cost of massive performance overhead – often utilizing only a small fraction of the GPU’s ~80–100 TFLOPS potential. In contrast, low-level CUDA C/Fortran on a lean NixOS Linux can achieve near-theoretical peak performance, especially for compute-bound tasks (e.g., matrix multiplies approaching 80 TFLOPS in practice) and keeps GPUs fed at high utilization. MATLAB’s abstractions (default double precision, limited kernel access, implicit data movement) severely limit GPU throughput, whereas writing native CUDA lets you explicitly manage memory, threads, and asynchronous execution for maximum FLOPS. A tuned NixOS environment further boosts performance with optimized compilers and minimal OS interference, plus better CPU/GPU coordination (NUMA-aware scheduling, no background bloat). HPC researchers: if you’re serious about performance, it’s time to ditch MATLAB for compute-heavy workflows. Use MATLAB for quick prototypes or plotting, but when you need to run a large simulation or ML training for days, invest in writing or using lower-level, optimized code on Linux. Your reward will be orders-of-magnitude speedups – faster research, bigger problems solved, and getting every dollar’s worth out of that 4090/5090 GPU. 🚀 It’s a no-brainer: embrace efficient tools and leave sluggish high-level code behind.