[nzlug] Threading / Parallel Programming Languages

[Paul Campbell]

On Friday 20 October 2006 10:50, Robin Sheat wrote:
> The thing with Fortress (among other things) is that loops are implicitly
> parallel, e.g:

.......

> which is quite useful. After seeing this I wrote (in Perl) a 'map'-like
> function that works across 'n' processors, for some data-mining stuff. It's
> nice because it makes my dual-core much more useful, and if I set the
> threading to 4, then when one or two threads are waiting on IO, the
> remaining two are actually processing. Of course, Perl has closures which
> makes this sort of thing a lot nicer.

I think you have to take these sorts of auto-parallelism with a grain of salt 
probably works for big things, synchronisation overhead kills you for small 
things - for various values of 'big' and 'small' and depending on how smart 
the compiler writer is.

I've spent a lot of time programming in verilog (a vaguely C-like chip design 
language) where parallelism is both explicit and everywhere - after all what 
you're designing is either going to be compiled into gates which are 
inherently parallel and into code for simulation (which may or may not be 
really parallel when run). Experience designing compilers for this and 
similar language shows that using real CPU paralellism in simulation is hard 
because the schedulable entities are so small and the synchronisation costs 
so large in comparison that parallel code (multithreaded on multiple CPUs) 
runs slower .....  (partly it's that rather sad x86 lock prefix).

It means that as CPUs hare off into multi-core/multi-die sorts of solutions 
(largely due to issues the self same designers are fighting with day-to-day 
like the speed of light and other wire-delay sorts of issues) the designers 
of those same chips are suffering because their tools aren't scaling at the 
same rate .....

People build and sell enormous arrays of FPGAs that one can synthesize gates 
for to get around these simulation speed issues.

However - changing the subject a little - one of the main reason why implicit 
parallelism is a good idea is that it's hard for people (even programmers) to 
learn to think in parallel - at least well enough so that you can see the 
timing hazards without looking for them. 

I was a kernel hack for maybe 10 years - getting that stuff right was 
hard .... I fell into chip design for another 10 or so and by the time I came 
back  the timing/synchronisation hazards that used to to be hard to see and 
think about just sort of seem easy - I think it comes from being in an 
environment where you live and breathe this stuff each and every day rather 
than occasionally

	Paul