>In addition, the reason it's hard to express structured data and nonuniform computation is because those aren't things GPU architecture excels at.
If you write simple GPU kernels (like those that Theano/Thrust/Copperhead/NumbaPro let you easily express) then you're mostly stuck "doing uniform operations over large chunks of memory". However, the latest GPUs are packed with features for going beyond this simple model. There's a rich set of global atomic operations, a fast register shuffle, better caching and most importantly: dynamic parallelism via nested kernel invocations. We're not programming for the G80 any more, Keplers can run a much larger swath of programs. Sure, you won't reach the theoretical peak FLOPS by traversing irregular structures via recursive kernels but you might still beat the pants off a CPU.
If you write simple GPU kernels (like those that Theano/Thrust/Copperhead/NumbaPro let you easily express) then you're mostly stuck "doing uniform operations over large chunks of memory". However, the latest GPUs are packed with features for going beyond this simple model. There's a rich set of global atomic operations, a fast register shuffle, better caching and most importantly: dynamic parallelism via nested kernel invocations. We're not programming for the G80 any more, Keplers can run a much larger swath of programs. Sure, you won't reach the theoretical peak FLOPS by traversing irregular structures via recursive kernels but you might still beat the pants off a CPU.