Ok, an little update on the PiZero Cluster Front:
Now, that OTG is compiled directly into the "next" kernel of RPi, we can use the OTG USB Virtual Ethernet directly - which makes it a lot easier. So I updated this :).
0. Preparing Minibian Jessie Image
I used some old Appliance Image I created from an Minibian Wheezy Image (https://minibianpi.wordpress.com/) earlier this year - for the 1.) section on the RPi Modell B pre 2.0 and RPi Modell A+. For the 2.) section, I used an special Appliance Image I made from an Minibian Jessie Image. However, I will document needed changes here, to get it running from any source. I recommend the Minibian Jessie Image as starting point, with this changes:
apt-get update
apt-get install -y raspi-config keyboard-configuration
raspi-config
# Default Configuration and Expand Filesystem using raspi-config
# Enter Finish and press Yes on Reboot the Device
apt-get install -y rpi-update sudo
apt-get -y dist-upgrade
reboot
rpi-update
reboot
# Create Default User pi
adduser pi
# Enter Password as wanted, i.e. raspberry
# Add user to default groups
usermod -a -G pi,adm,dialout,cdrom,audio,video,plugdev,games,users pi
# Add sbin Paths to pi
echo 'export PATH="$PATH:/sbin:/usr/sbin:usr/local/sbin"' >> /home/pi/.bashrc
# Add user to sudo
visudo
# Add under
# # User privilege specification
# root ALL=(ALL:ALL) ALL
pi ALL=(ALL:ALL) ALL
# Save and Exit
reboot
# Disable root login
sudo passwd -l root
or - and default RPi Jessie Image.
1. Building mpich 3
# Update and Install Dependencies, then reboot
sudo apt-get update
sudo apt-get -y dist-upgrade
sudo apt-get install -y build-essential
sudo reboot
# Make MPICH 3.2
cd ~
wget http://www.mpich.org/static/downloads/3.2/mpich-3.2.tar.gz
tar -xvzf mpich-3.2.tar.gz
cd mpich-3.2
# This will take some time
sudo ./configure --disable-fortran
# This will take several cups of tea ;)
sudo make
sudo make install
# Create SSH on Master, distribute to Slaves
cd ~
ssh-keygen -t rsa –C "raspberrypi"
Default location should be set to /home/pi/.ssh/id_rsa if you're using the standard user pi. Then choose this command to distribute the key to all your "slave maschines":
cat ~/.ssh/id_rsa.pub | ssh pi@IP_OF_SLAVES "mkdir .ssh;cat >> .ssh/authorized_keys"
( Was taken from http://www.southampton.ac.uk/~sjc/raspberrypi/ - he was the original father of the RPi Clusters and his work inspired me already years ago - so please read and support his work :)! Additional infos can be found at http://westcoastlabs.blogspot.co.uk/2012/06/parallel-processing-on-pi-bramble.html)
You could also just
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
to your own authorized files, shutdown your Master Pi after that and clone the card several times for all your Clients. This way, you would only need to do the work once - however, maybe you should release the keys in ~/.ssh/ so that only your Master Pi could command the slaves
2. PiZero on Virtual Ethernet
I build my MPICH as mentioned in 1) on an Minibian Jessie image (SDCard running on an RPi B). Then I installed the new RPi Kernel and prepared the image for OTG Ethernet, using the new information provided by NicoHood:
#install new kernel with OTG support
sudo BRANCH=next rpi-update
# This is required in the config.txt
echo "device_tree=bcm2708-rpi-zero.dtb" | sudo tee -a /boot/config.txt
# Regarding the readme this is applied automatically for a pi zero (and yes it works without) - so not really needed
# echo "dtoverlay=dwc2" | sudo tee -a /boot/config.txt
# Only use this, if the new kernel causes sd card errors
# echo "dtoverlay=mmc" | sudo tee -a /boot/config.txt
# permanently add the g_ether (or any other) module, add it to /etc/modules
echo "dwc2" | sudo tee -a /etc/modules
echo "g_ether" | sudo tee -a /etc/modules
# Add settings to network interfaces
echo '
allow-hotplug usb0
iface usb0 inet static
address 192.168.7.2
netmask 255.255.255.0
network 192.168.7.0
broadcast 192.168.7.255
gateway 192.168.7.1
dns-nameservers 8.8.8.8' | sudo tee --append /etc/network/interfaces
# Some additional tweaks:
Add
# Turn HDMI Off
/usr/bin/tvservice -o
# Turn HDMI Back On
#/usr/bin/tvservice -p
# Turn ACT LED Off on Pi Zero
echo none | tee /sys/class/leds/led0/trigger
echo 1 | tee /sys/class/leds/led0/brightness
to your /etc/rc.local before exit 0 to turn off the HDMI Interface on boot,
as well as the LED of the Pi Zero to use less energy. Found on:
http://www.midwesternmac.com/blogs/jeff-geerling/raspberry-pi-zero-conserve-energy and http://www.midwesternmac.com/blogs/jeff-geerling/controlling-pwr-act-leds-raspberry-pi
This was enough to create an Pi Zero Slave Image.
Shutdown the RPi now with
sudo shutdown -h now
remove the Power and insert the SDcard into your Pi Zero.
On the Master Machine, I did following changes:
# Add settings to network interfaces
echo '
allow-hotplug usb0
iface usb0 inet static
address 192.168.7.1
netmask 255.255.255.0
network 192.168.7.0
broadcast 192.168.7.255' | sudo tee --append /etc/network/interfaces
# Allow Ipv4 Forward
echo 'net.ipv4.ip_forward=1' | sudo tee --append /etc/sysctl.conf
# Install iptables
sudo apt-get install -y iptables
# Define NATing rules
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A FORWARD -i eth0 -o usb0 -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -i usb0 -o eth0 -j ACCEPT
# Save NAT rules / load iptables on interface up
sudo touch /etc/iptables_masq.rules
sudo chown pi:pi /etc/iptables_masq.rules
sudo iptables-save > /etc/iptables_masq.rules
Add
pre-up iptables-restore < /etc/iptables_masq.rules
under the eth0 section in the network interfaces
sudo vi /etc/network/interfaces
i.e.
auto eth0
iface eth0 inet dhcp
pre-up iptables-restore < /etc/iptables_masq.rules
( Info taken from: http://serverfault.com/questions/405628/routing-traffic-on-ubuntu-to-give-raspberry-pi-internet-access )
#After that, I shutdown the RPi via
sudo shutdown -h now
#removed power from it.
Then I attached the Pi Zero to the Hub of Pi B via an Micro USB Cable by using the Micro USB OTG Slot on the Pi Zero, connecting it to the Hub of the Pi Modell B. Next, I powered up the Pi B - and both booted.
I pinged 192.168.7.2 - which was the IP of the Pi Zero - and it answered. Now I only had to use cat ~/.ssh/id_rsa.pub | ssh pi@192.168.7.2 "mkdir .ssh;cat >> .ssh/authorized_keys" from Section 0 to get the SSH Key, created in Step 0 onto the Pi Zero and could use that to automatically login in into the Pi Zero.
With the new IP of the RPi B and Pi Zero in the machinefile of mpich I could then use my both RPis with higher speed and nearly zero costs for cabeling and power :)!
The clou: I don't need an additional powersupply for the Pi Zero - nor network adapters, RJ45 cabling, an switch - only one USB A to USB Micro cable per Pi Zero - and maybe an big, active USB Hub ;)!
Now, I need to get more Pi Zeros - I plan on using an Modell B as Master with an big active USB Hub to support 4 Pi Zeros - or an Modell B+ with an REALLY BEFFY USB Supply to work them all the same RPi - but that would come down to trying this... And I got only one Pi Zero - so I need some more time (or some sponsors?) to get me more RPi Zeros to try and see, whether this approach does scale ;)!
Best thing: This can also be used to try the awesome work of http://blog.hypriot.com/ to build an Docker Cluster from that - cool, ain't it?