> For a lot of code out of order execution is the only way to extract some amount of parallelism
But that's the thing. A CPU is simply a parallel machine forced to accelerate "sequential" code.
A "truly sequential" code sequence, like linkedList->head->next->next->next cannot be parallelized on a modern CPU. The only stuff that can be parallelized are if-statements / loops (aka: branch prediction: try to do the future speculatively), and anything Tomasulo's algorithm happens to pick up.
Even then: the modern CPU will attempt to parallelize that linked-list access because of Cache prefetching. That's actually why Arrays work so well in today's architectures: because the L1, L2, L3, and DRAM components are working in parallel through Cache Prefetchers (and Arrays are so simple that speculation will certainly be correct).
In effect: people are writing parallel programs. They are just leaving it to the CPU to figure out the parallelism details.
X = Y + Z; A = B + C can be made parallel (independent variables).
X = Y + Z; X = D + E. Also parallel: Write-after-write hazard, so X=D+E can execute first, and just throw away X=Y+Z entirely. Etc. etc.
CPUs simply find these patterns in your code and execute them in parallel.
AMD Zen has 4-integer pipelines + 4-vector pipelines + 2 load/store units per core.
Intel has 8-pipelines of varying capabilities per core. 0, 1, 5 are vector units, but 0 is also the division pipeline. Its a bit more complicated, but its still 8x parallelism that is being fed by the front-end (and reordered into the correct order by the backend).
--------------
Basically: the parallelism does exist in the code that was written. The programmer just hasn't explicitly acknowledged it yet.
But with every "speculative" branch taken, the CPU does work, and then (maybe) throws it away. The GPU-basis of coding is to not do any speculative work at all. Instead, GPUs throw 10-threads per shader (kinda like a core) and SMT the heck out of them.
If thread#0 accesses memory, it will be stuck doing that for 300+ cycles. So thread#1 will take over the core. When thread#1 waits on memory, thread#2 takes over. Etc. etc. By the time thread#9 and #10 roll around, thread#1 probably has its memory ready.
Instead of speculatively executing thread#0, the GPU is provided with alternative work it can do. In any HPC context, there's probably "more work to do" somewhere, so the GPU can remain saturated with work to do. Its naturally a more power-efficient methodology: there's less wasted work.
It just requires the programmer to figure out a lot of tasks that the GPU can do to remain saturated. While a CPU will dedicate more and more resources, all the way towards speculative execution, to accelerate the same task. Even if its highly wasteful.
But that's the thing. A CPU is simply a parallel machine forced to accelerate "sequential" code.
A "truly sequential" code sequence, like linkedList->head->next->next->next cannot be parallelized on a modern CPU. The only stuff that can be parallelized are if-statements / loops (aka: branch prediction: try to do the future speculatively), and anything Tomasulo's algorithm happens to pick up.
Even then: the modern CPU will attempt to parallelize that linked-list access because of Cache prefetching. That's actually why Arrays work so well in today's architectures: because the L1, L2, L3, and DRAM components are working in parallel through Cache Prefetchers (and Arrays are so simple that speculation will certainly be correct).
In effect: people are writing parallel programs. They are just leaving it to the CPU to figure out the parallelism details.
X = Y + Z; A = B + C can be made parallel (independent variables).
X = Y + Z; X = D + E. Also parallel: Write-after-write hazard, so X=D+E can execute first, and just throw away X=Y+Z entirely. Etc. etc.
CPUs simply find these patterns in your code and execute them in parallel.
AMD Zen has 4-integer pipelines + 4-vector pipelines + 2 load/store units per core.
Intel has 8-pipelines of varying capabilities per core. 0, 1, 5 are vector units, but 0 is also the division pipeline. Its a bit more complicated, but its still 8x parallelism that is being fed by the front-end (and reordered into the correct order by the backend).
--------------
Basically: the parallelism does exist in the code that was written. The programmer just hasn't explicitly acknowledged it yet.
But with every "speculative" branch taken, the CPU does work, and then (maybe) throws it away. The GPU-basis of coding is to not do any speculative work at all. Instead, GPUs throw 10-threads per shader (kinda like a core) and SMT the heck out of them.
If thread#0 accesses memory, it will be stuck doing that for 300+ cycles. So thread#1 will take over the core. When thread#1 waits on memory, thread#2 takes over. Etc. etc. By the time thread#9 and #10 roll around, thread#1 probably has its memory ready.
Instead of speculatively executing thread#0, the GPU is provided with alternative work it can do. In any HPC context, there's probably "more work to do" somewhere, so the GPU can remain saturated with work to do. Its naturally a more power-efficient methodology: there's less wasted work.
It just requires the programmer to figure out a lot of tasks that the GPU can do to remain saturated. While a CPU will dedicate more and more resources, all the way towards speculative execution, to accelerate the same task. Even if its highly wasteful.