Parallel Processing on a Pi … it works !

Shamelessly this is my take on the information provided by Prof Simon Cox from Southampton University in his website Steps to make a RaspberryPi Supercomputer

He first recommends some background reading on some parallel processing and the tools we’ll use for this, these can be found at

Once you have digested these and got a rough handle on what we are about to try, we can get down to configuring one of the PI’s with the required tools and libraries

Update the system

First we update out system

sudo apt-get update

Install Fortran

At this stage I skip over some of Prof Cox’s instructions, I don’t know Fortran, having not used it since 1986 and therefore its pointless even installing it

Download MPI Libraries

First we need to download and compile to MPI libraries, so lets create a directory to store the sources

mkdir ~/mpich2
cd ~/mpich2

Then get the sources from their home on the internet

wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.4.1p1/mpich2-1.4.1p1.tar.gz

Next steps is to unpack the compressed file we just downloaded

tar xfz mpich2-1.4.1p1.tar.gz

Compile the Libraries

Make yourself a place to put the compiled stuff – this will also make it easier to figure out what you have put in new on your system. Also you may end up building this a few times…

sudo mkdir mpich2_install

Next make a build directory (so we keep the source directory clean of build things) and change to it

mkdir mpich2_build
cd mpich2_build

Now we are going to configure the build, make the code and install it. Note that because we have decided not to install Fortran, we need to add a couple of extra commands ( –disable-f77 –disable-fc ) to configure, which stop it looking for Fortran

The instructions from Southampton University initially create a stand alone MPI package, but I want to use this almost exclusively with Python and the mpi4py library and this requires the MPI library to be built as a shared library, so we also need to add the  –enable-shared flag to the configure options

sudo /home/keith/mpich2/mpich2-1.4.1p1/configure -prefix=/home/keith/mpich2/mpich2_install --disable-f77 --disable-fc --enable-shared
sudo make
sudo make install

The above 3 commands take some time on a Pi, don’t be phased if a couple hours pass and you are still waiting lines and lines of text scrolling past your console.

Once the 3 lines are complete, MPI is compiled and installed, we can do a couple of house keeping tasks to make it easier to access the various MPI tools now available to us. First off is to add the MPI bin to our path

Add the place that you put the install to your path

export PATH=$PATH:/home/keith/mpich2/mpich2_install/bin

We can make this permanent by adding this to our .profile file

vi .profile

and add the following 2 lines to the bottom

# Add MPI to path
PATH="$PATH:/home/keith/mpich2/mpich2_install/bin"

At this stage you need to make action the changes to your path statement, you can either reboot with

sudo reboot

Or while still logged on

source ~/.profile

We can now check whether things did install or not

which mpicc
/home/keith/mpich2/mpich2_install/bin/mpicc
which mpiexec
/home/keith/mpich2/mpich2_install/bin/mpiexec

Testing the installation

Change directory back to home and create somewhere to do your tests

cd ~
mkdir mpi_testing
cd mpi_testing

Before we can run some tests we need to create a file with the IP addresses of the machines we will distribute the load onto. For the first test this will be this single node, so we only need to add its address. Create file called ‘machinefile’ and put a single line containing the ip address of this node

vi machine

And it should look something like

pislave

Now we can test whether MPI works for you on a single node

mpiexec -f machinefile -n 1 hostname
pislave1

So far so good, its all working, now lets run a simple example which calculates Pi, on a Pi, get it, funny eh ?, ok so not so funny, but its a test

cd /home/keith/mpich2/mpich2_testing
mpiexec -f machinefile -n 2 ~/mpich2/mpich2_build/examples/cpi

You will see out put that looks like

Process 0 of 2 is on pislave1
Process 1 of 2 is on pislave1
pi is approximately 3.1415926544231318, Error is 0.0000000008333387
wall clock time = 0.005259

Thats it, thanks to Prof Cox for a faultless description to make a Super Computer.

Branching out on to multiple nodes

The above set of instructions needs to be carried out on all nodes, once you have tested that all nodes work in a single node setting, we now need to set up the master/slave relationship. All the remaining instructions should now be carried out on your master node, in my instance ‘pimaster’, so first ssh into this Pi

First we need to allow master to log into all the slave nodes without needing password. We’ve already come across the use of SSH in a previous blog. First we need to create a RSA key pair on pi master

ssh-keygen -t rsa -C "keith@pimaster"
Generating public/private rsa key pair.
Enter file in which to save the key (/home/keith/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/keith/.ssh/id_rsa.
Your public key has been saved in /home/keith/.ssh/id_rsa.pub.
The key fingerprint is:
aa:8e:86:43:57:e7:09:27:bc:05:ad:1e:be:7b:ca:c1 keith@pimaster
The key's randomart image is:
+--[ RSA 2048]----+
|      .          |
|     . .         |
|    . o          |
|     B +         |
|    + X S        |
| . ..+ +         |
|...  Eo          |
|o ...o..         |
| o..o++          |
+-----------------+

Now that we have public and private keys we need to copy the public key to each slave node

cat ~/.ssh/id_rsa.pub | ssh keith@192.168.1.201 "cat >> .ssh/authorized_keys"
cat ~/.ssh/id_rsa.pub | ssh keith@192.168.1.202 "cat >> .ssh/authorized_keys"
cat ~/.ssh/id_rsa.pub | ssh keith@192.168.1.203 "cat >> .ssh/authorized_keys"
cat ~/.ssh/id_rsa.pub | ssh keith@192.168.1.204 "cat >> .ssh/authorized_keys"

You can check each of these is working by ssh’ing into each one in turn and then ‘exiting’ back out

ssh pislave1
ssh pislave2
ssh pislave3
ssh pislave4

Now back on Pi Master we need to create a machinefile with the ip addresses of all the slaves

vi machinefile

And enter the addresses of all the nodes

192.168.1.200
192.168.1.201
192.168.1.202
192.168.1.203
192.168.1.204

Now we run the same C program as before, but this time with the modified machinefile containing the ip addresses of our 4 slaves nodes. The first time we run this we’ll be prompted to enter the password for RSA key, after that it shouldn’t ask for it until after you reboot

mpiexec -f machinefile -n 5 ~/mpi/mpich2_build/examples/cpi

You should then see out put such as

Process 0 of 2 is on pimaster
Process 1 of 5 is on pislave1
Process 2 of 5 is on pislave2
Process 3 of 5 is on pislave3
Process 4 of 5 is on pislave4
pi is approximately 3.1415926544231318, Error is 0.0000000008333387

Well thats it, we have a super computer running 5 nodes in a master slave configuration, … wow !!!!

apaik

Hi,
I’m enjoying your blog.
I got stuck at your “mpiexec -f machinefile -n 1 rpi2” line.
I am getting “mpiexec was unable to lauch the specified application as it could not find an executable:
Executable: -f
Mode: rpi2”

Also, if I do “which mpiexec”-> I get “/usr/bin/mpiexec”

Thanks in advance.

keith

Did you reboot after setting up .profile. I noticed I did not have this as part of the instructions
Once you reboot it will pick up the correct path

Your email address will not be published. Required fields are marked *