[Freifunk] Upgrade Virtual Freifunk Router on VMWare ESXi 6.5

As I mentioned earlier, I use a virtual Freifunk Router as part of my mobile Infrastructure / Server. With this little VM, I can actually run a big-area Freifunk Network with lots of users without having to invest in new TP-Link accesspoints: I am running the VM, outputing the Freifunk LAN to a designated VLAN and use old spare Accesspoints as dumb "Media-Converters" (LAN to WIFI ;)).

But, as I only use this appliance every now and then for different kind of conventions and conferences, I need to upgrade that Appliance manually, to get it straight to the latest version without waiting or reinstalling (and droping the VPN key...).

Actually, that is quite simple in VMWare: Just go to the terminal of said VM and get started. Well. Ok, stop! Before you're doing that, just remember: You're on a VM. You got no excuse at all for not making a backup - so just shutdown your VM, make a snapshot, then turn it back on - and get started. Better safe than sorry ;)!

First, we're going to install wget with SSL support

opkg update
opkg install wget

Then we need to find the latest version of our Freifunk Firmware - as sysupgrade package. I used the gluon-fftr-0.8.4-x86-vmware.vmdk to install the VM, so I need an x86-generic-sysupgrade.img.gz :). I found that thing here: https://github.com/freifunktrier/firmware_store/tree/master/firmware/stable/sysupgrade . However, please bear in mind to use the image from YOUR Freifunk Provider ;). Try to download the image and get to the "RAW FILE" link on Github and use it to download the file to your VM:

cd /tmp
wget -O sysupgrade.img.gz  https://github.com/freifunktrier/firmware_store/blob/master/firmware/stable/sysupgrade/gluon-fftr-0.8.6+jenk_tackin-x86-generic-sysupgrade.img.gz?raw=true

After that, apply the upgrade and reboot:

sysupgrade -v /tmp/sysupgrade.img.gz
reboot

And thats it :)!

Thanks a lot again to Freifunk Trier for supporting my project :).

Solve Windows 10 does not automount USB Drives anymore

My Windows 10 machine started behaving weirdly and did not mount any USB Drives / Harddrives and such anymore. I had to open the Disk Management Tool and mount them manually, which is quite inconvenient. So I was looking up the error and came across the nearly perfect solution at http://woshub.com/windows-doesnt-assign-letters-to-external-and-usb-flash-drives/ . However, I did change it a bit so that the VDS is changed to automatically start on boot via the CLI :).

Oh, and yes, you need to do this from an admin cmd 😉

First thing to check if something is wrong, is if your VDS (Virtual Disk Service) works. If not, set it to start on boot and start the service:

sc query vds
sc config vds start= auto
sc start vds
sc query vds

If automounting still not works, it could be deactivated. Start diskpart, watch the status of automount and if it is disabled, activate it.

diskpart
DISKPART> automount
DISKPART> automount enable
DISKPART> exit

Now everything should be working fine again 🙂

[Gigabyte] BIOS Upgrade on old Gigabyte Motherboards

There are several old Gigabyte Motherboards like the GA-MA Series which uses their BIOS Included Q-Flash Utility for upgrading. This tool tries to access an attached USB Device in a file browser way to give you the choice on which file to flash. Most of the time, you won't be able to access your drive, as it will only be shown as "Floppy B". In truth, this means your USB drive is formated the wrong way: You should have only one partition on that stick, with size LESS than 128 MB and FAT as file system. Yeah, I figured that out the hard way ^^'. It will then be shown as "HDD 0-0" in Q-Flash and will provide your files for easy upgrading 🙂

[Dell] BIOS Upgrade on a Dell Precision T1500

Hi there, I just got hands on an old Dell T1500 workstation. It is not the beefiest monster - but still kicking. And I got it for a bargain :). So, while I was refurbishing it, I wanted to do an BIOS upgrade, like usual. Turned out, Dell only offers a combined "DOS/WINDOWS" Upgrade File. I tried upgrading via an FreeDOS USB Stick, created with Rufus, however - it failed. Ok, lets try Windows: I installed Windows 8.1 x64 - and the tool "worked" - however, even after reboot, nothing had changed. Reset CMOS, Load Default in BIOS, nothing. Darn... Well.. All the Dell support stuff for this machine was around Windows 7-ish versions, so I thought "last chance" 😉 - and yes! It worked:

You need to install Windows 7 x64 and upgrade the Bios 2.0.2 to 2.4.0 via your Windows install. DOS seems to be not working - and Windows 8.1 won't work either. Also for good measure, load the default settings before upgrading and leave all other settings (especially the disabled fancy CPU stuff!) untouched.

Also, you'll see directly if it works: During the Win 7 upgrade, it disabled the USB mouse I was using and it took way longer. On Windows 8.1 - I could move the mouse as I wished. Oh, and one last thing: Administator rights, please ;).

CUDA and Tensorflow in Docker

In this howto we will get CUDA working in Docker. And - as bonus - add Tensorflow on top! However, please note that you'll need following prereqs:

GNU/Linux x86_64 with kernel version > 3.10
Docker >= 1.9 (official docker-engine, docker-ce or docker-ee only)
NVIDIA GPU with Architecture > Fermi (2.1)
NVIDIA drivers >= 340.29 with binary nvidia-modprobe

We will install the NVIDIA drivers in this tutorial, so you should only have the right kernel and docker version already installed, we're using a Ubuntu 15.05 x64 machine here. For CUDA, you'll need a Fermi 2.1 CUDA card (or better), for tensorflow a >= 3.0 CUDA card...

Which Graphicscard Model do I own?
lspci | grep VGA
sudo lshw -C video

Output i.e.:

product: GF108 [GeForce GT 430]
vendor: NVIDIA Corporation

You should lookup on google if it works with cuda / Fermi 2.1, i.e. on https://developer.nvidia.com/cuda-gpus

GeForce GT 430 - Compute: 2.1

Ok, that one works!

I got additional infos from: https://www.geforce.com/hardware/desktop-gpus/geforce-gt-430/specifications

CUDA and Docker?

You can find out more about that topic on https://github.com/NVIDIA/nvidia-docker

Getting it to work will be the next step:

Download right CUDA / NVIDIA Driver

from http://www.nvidia.com/object/unix.html
I choose Linux x86_64/AMD64/EM64T, Latest Long Lived Branch version: 375.66, but please check in the description of the file, if your graphics card is supported!

After Download, install the driver:
chmod +x NVIDIA-Linux-x86_64-375.66.run
sudo ./NVIDIA-Linux-x86_64-375.66.run

It will ask for permission, accept it. If it gives info that the nouveau driver needs to be disabled, just accept that, in the next step, it will generate a blacklist file and exit the setup. Afterwards, run

sudo update-initramfs -u

and reboot your server. Then, rerun the setup with

sudo ./NVIDIA-Linux-x86_64-375.66.run

You can check the installation with

nvidia-smi

and get an output similar to this one:

Mon Jul 24 09:03:47 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66                 Driver Version: 375.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 430      Off  | 0000:01:00.0     N/A |                  N/A |
| N/A   40C    P0    N/A /  N/A |      0MiB /   963MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0                  Not Supported                                         |
+-----------------------------------------------------------------------------+

which means that it worked!

Install nvidia-docker and nvidia-docker-plugin
wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.deb
sudo dpkg -i /tmp/nvidia-docker*.deb && rm /tmp/nvidia-docker*.deb
Test nvidia-smi from Docker
nvidia-docker run --rm nvidia/cuda nvidia-smi

should output:

Using default tag: latest
latest: Pulling from nvidia/cuda
e0a742c2abfd: Pull complete
486cb8339a27: Pull complete
dc6f0d824617: Pull complete
4f7a5649a30e: Pull complete
672363445ad2: Pull complete
ba1240a1e18b: Pull complete
e875cd2ab63c: Pull complete
e87b2e3b4b38: Pull complete
17f7df84dc83: Pull complete
6c05bfef6324: Pull complete
Digest: sha256:c8c492ec656ecd4472891cd01d61ed3628d195459d967f833d83ffc3770a9d80
Status: Downloaded newer image for nvidia/cuda:latest
Mon Jul 24 07:07:12 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.66                 Driver Version: 375.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 430      Off  | 0000:01:00.0     N/A |                  N/A |
| N/A   40C    P8    N/A /  N/A |      0MiB /   963MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0                  Not Supported                                         |
+-----------------------------------------------------------------------------+

Yep, you got it working in Docker!

Running an interactive CUDA session isolating the first GPU
NV_GPU=0 nvidia-docker run -ti --rm nvidia/cuda
Input our first Hello World program
echo '#include <stdio.h>
// Kernel-execution with __global__: empty function at this point
__global__ void kernel(void) {
// printf("Hello, Cuda!\n");
}
int main(void) {
// Kernel execution with <<<1,1>>>
kernel<<<1,1>>>();
printf("Hello, World!\n");
return 0;
}' > helloWorld.cu
Compile it within the Docker container
nvcc helloWorld.cu -o helloWorld
Execute it...
./helloWorld
and you get,...
Hello, World!

Congrats, you got it working!

Encore, Tensorflow
Getting Tensorflow to work is straight forward:
nvidia-docker run -it -p 8888:8888 tensorflow/tensorflow:latest-gpu

It will output something like:

Copy/paste this URL into your browser when you connect for the first time, to login with a token:
http://localhost:8888/?token=d747247b33023883c1a929bc97d9a115e8b2dd0db9437620

you should do that 🙂

Then enter the 1_hello_tensorflow notebook and run the first sample:

from __future__ import print_function
import tensorflow as tf
with tf.Session():
    input1 = tf.constant([1.0, 1.0, 1.0, 1.0])
    input2 = tf.constant([2.0, 2.0, 2.0, 2.0])
    output = tf.add(input1, input2)
    result = output.eval()
    print("result: ", result)

by selecting it and clicking on the >| (run cell, select below) Button.
This worked for me:

result: [ 3. 3. 3. 3.]

however... sadly not the GPU was calculating the results as shown by the Docker CLI:

Kernel started: 2bc4c3b0-61f3-4ec8-b95b-88ed06379d85
[I 07:31:45.544 NotebookApp] Adapting to protocol v5.1 for kernel 2bc4c3b0-61f3-4ec8-b95b-88ed06379d85
2017-07-24 07:32:17.780122: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-07-24 07:32:17.837112: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-07-24 07:32:17.837440: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties:
name: GeForce GT 430
major: 2 minor: 1 memoryClockRate (GHz) 1.4
pciBusID 0000:01:00.0
Total memory: 963.19MiB
Free memory: 954.56MiB
2017-07-24 07:32:17.837498: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0
2017-07-24 07:32:17.837522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y
2017-07-24 07:32:17.837549: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1003] Ignoring visible gpu device (device: 0, name: GeForce GT 430, pci bus id: 0000:01:00.0) with Cuda compute capability 2.1. The minimum required Cuda capability is 3.0.

So, CUDA >= 3.0 devices only for tensorflow 🙁 - but, it still works, as it is using the CPU (however, not as fast as it could :/)

Infos taken from:

https://github.com/NVIDIA/nvidia-docker
https://developer.nvidia.com/cuda-gpus
https://hub.docker.com/r/tensorflow/tensorflow/

[FreeBSD] Some personal notes for emergencies

Mounting zfs on FreeBSD 9.3 LiveFS for recovery

# List zfs pools
zpool import
-> i.e. syspool
# mount zfs pool syspool
zpool import -f -R /mnt syspool
# create mnt folder for new folder
mkdir /tmp/home
# mount data from zfs pool syspool
mount -t zfs syspool/DATA/home /tmp/home
# work on stuff and umount /tmp/home
# then umount zfspool
zfs umount -a

Format a harddrive on FreeBSD 9.3 and mount it as folder

# format drive with ufs
newfs /dev/da1
# create tmp folder
mkdir /tmp/mnt
# mount new drive
mount -t ufs /dev/da1 /tmp/mnt
# copy files
cp -R /tmp/home/* /tmp/mnt/

ezjail stuff

# show jails
ezjail-admin list
# Get to console
ezjail-admin console JAILNAME
# Start / Stop
ezjail-admin start JAILNAME
ezjail-admin stop JAILNAME
# No Autorun
ezjail-admin config -r norun JAILNAME
# Autorun
ezjail-admin config -r run JAILNAME

Jail Folder on FreeBSD: /usr/jails/JAILNAME
Internal Jail Folder on FreeBSD: /usr/jails/JAILNAME/*/var/HEREISHOME

[resinOS] Build resinOS from scratch

As the time of writing, resinOS is available for Download at Version 2.0.6+rev3.dev for Raspberry Pi 3. This build, however, is nearly 2 weeks old and in the meantime, something great happend: Docker has finally updated to Version 17.03.1 - upgraded from the old ~10 (ten-ish) version - which was not that cool (and without Swarm ;)). So, it is a good idea to get to know how to build your own resinOS in case you really want to live on the bleeding edge ;).

Install Dependencies (Ubuntu 16.04 LTS)

sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
     build-essential chrpath socat cpio python python3 python3-pip python3-pexpect \
     xz-utils debianutils iputils-ping libsdl1.2-dev xterm

goto /, because this build will create very long filenames

cd /

clone the repo, maybe some root power is needed here 😉

git clone https://github.com/resin-os/resin-raspberrypi
cd resin-raspberrypi
git submodule update --init --recursive

you would be done here and could build your own resinOS with the build command,
however, if you really want to pull the latest upgrades...

cd layers/meta-resin
git checkout master
git pull
cd ../..

finally build resinOS for Raspberry Pi 3

./resin-yocto-scripts/build/barys -r --shared-downloads $(pwd)/shared-downloads/ --shared-sstate $(pwd)/shared-sstate/ -m raspberrypi3                       

after quite some time, you'll find the image in

build/tmp/deploy/images/raspberrypi3/resin-image-raspberrypi3.resinos-img

 

There is quite a lot of stuff you can change on your resinOS, so be sure to check out https://resinos.io/docs/custombuild/ for more documentation on that topic. Have fun :)!

[resinOS] Dockerize your own Flask GUI with resinOS

I have been working with Docker and resin.io as well as resinOS for quite some time now and I am actively using those services actively for different projects: The possibility to just throw away a container and build it anew in record breaking time is just awesome and also the fact that I can now just ship images to some device, deploy it there - and it will just work. As some of you know, I am working as volunteer in different NPOs, i.e. the RepairCafé Trier in which I repair Computers and Mobile phones for free, the PiAndMore / CMD e.V. on which I teach interessted guys and gals about Raspberry Pi and Linux (and even did two presentations on resin.io and resinOS, both linked on this blog - and as video :)) - and also one other NPO which is trying to give information about Japanese Culture. For those events I already created an Tickting System called JCTixx, which uses QR Codes to sell and check tickets for those named festivals. However, I created the Scanner Hardware in 2010, just starting my second year in an Apprenticeship as IT Systemselectronics guy, without anything as a Raspberry Pi available. However, being a bit creative, I just took some old Linux routers, threw the good old OpenWRT on those and soldered directly to the its testpoints / CPU to get the GPIO pins I needed ;). That worked very well - however, nowadays, TLS 1.2 and such eat away those small MIPS cores - and I could - with a lot of work - just get them working with said crypto in 2015. However, being 7 years old now, I need new Scanners - and more flexible way of delivery software as well. So resinOS it is! 🙂

Full disclosure: As I am currently developing the Hardware for the Scanners and I am always looking for some Raspberry Pis and other hardware, I stumbled upon resin.ios #blog4swag program and decided to make this tutorial because of two reasons: 1.) I actually need an dockerized GUI for the scanners and therefore am working on a good solution to the problem - and as usually - I want my dear readers to get some interesting stuff to read. 2.) Maybe I can acquire some hardware support from the resin guys which then would help me out to get my NPO helped :). So in the end, everyone wins and no one dies - to cite Doctor Who :).

So, lets get started!

1.) Getting some GUI working at all
I saw some users experimenting with Docker and Chromium along the x86 systems - however, the first resin project I stumbled upon which really made me think about putting the GUI within a container, was resin.ios ElectronJS example. This thing really rocked, but I saw that I needed to work on something a bit easier which would just deploy some sort of GUI - and being as simple as it gets. So I started to look around and stumbled across different projects, namely
RPiBrowser-resin.io and resin-electron-app. Those two were a great starting point and using resin.ios ElectronJS example as well, I started hacking the code together, which I finally submitted to this Git Repo:
docker-raspbian_xinit. With the great help of Gergely Imreh from the resinIO Team (thanks a lot :)!) I got the project working and pushed it onto Docker Hub. To walk you through the Git Repo: I made two versions, one nailed to the release of the 26.04.2017 Raspbian and one made from the latest version. I basically grabbed resin.ios Raspbian from Docker Hub, injected the qemu files (which they did show in one tutorial) and used this as a starting point. It enables Docker Hub to build the Raspberry Pi Images, so that you can use them on your Pi.

If you want to build the docker-raspbian_xinit image yourself, you need to remove the RUN [ "cross-build-start" ] and RUN [ "cross-build-stop" ] command from the Dockerfile :).

The Dockerfile itself builds a basic system with xinit / xorg, alsa and even touchscreen support (xinput-calibrator). It generated german locales and a user called pi (which is both not needed, but I included it as matter of personal preference), copys the needed files and enables systemd. Systemd will start the xinit-docker.service which invokes the start.sh which does prepare the system, i.e. adds the container name to the /etc/hostname, so that sudo will work, activates its own SSH server on port 22, includes a touchscreen calibiration file - if available, creates a very needed config folder, removes files from old X11 sections, sets volume to 100% on speakers and phones and then starts the xinit process with the launch_app.sh :).

The launch_app.sh then works in the xinit context and does xinit specific stuff: It disables screen blanking, sets the keyboard to the created german locales (you can comment this out as well if you want or overwrite the file ;)) - then - finally - starts the matchbox-window-manager without titlebar, but WITH keyboard and mousesupport and launches....
... gedit.
Yeah. Right. I know that is a real disappointment. But I needed a small tool which would not add too much file size and show that keyboard and mouse are working - so I just went for gedit ^^'. Sorry if you were hoping to find something really awesome and cool at this place. But nonetheless, it works, and this image as well as the Git Files should serve as a starting point for your own Xinit adventure - so, thats the reason :).

After I got it finally working, I thought about my personal usecase: I will be using a lot of Python and thought about using Flask as a GUI. However, Flask is just a webframework and does not have the ability to shown something "GUI-like" - thus needing some kind of Webbrowser - and this part could be found in shape of pywebview. Pywebview just includes some website, app or similiar into a small GUI Frame with Webkit Browser in it. Cool! Exactly what I needed. However, I did not have time get to work on my own UI - and wanted to jumpstart the project by getting the Docker Container to work - so I decided to grab a cool small Flask Web GUI project and use this to showcase how easy it is to built a self-starting Docker Container GUI - on resinOS. And with that in mind, I went for the very cool helloflask Calculator by Grey Li.

2.) Getting a Flask GUI working - using 1. 😉
Ok, as soon as I got my Xinit project working, I decided to use this a base image, just overwriting changes in the system and injecting the files needed to run the pywebview'd Calculator.
With that in mind, the Dockerfile became quite small (you'll find the Source Files on Github and the Image on Docker Hub as well :))

FROM nmaas87/docker-raspbian_xinit:jessie-20170426
MAINTAINER Nico Maas <mail@nico-maas.de>
ENV DEBIAN_FRONTEND noninteractive
RUN [ "cross-build-start" ]

RUN apt-get update \
    && apt-get install -yq --no-install-recommends \
        python3-pip python3-pyqt5 python-gi gir1.2-webkit-3.0=2.4.9-1~deb8u1+rpi1 gir1.2-javascriptcoregtk-3.0=2.4.9-1~deb8u1+rpi1 libjavascriptcoregtk-3.0-0=2.4.9-1~deb8u1+rpi1 libwebkitgtk-3.0-0=2.4.9-1~deb8u1+rpi1 \
    && apt-get autoremove -qqy \
    && apt-get autoclean -y \
    && apt-get clean && rm -rf /var/lib/apt/lists/* && mkdir /var/lib/apt/lists/partial \
    && pip3 install Flask pywebview \
    && mkdir /usr/src/app/templates /usr/src/app/static

# copy program
COPY src /usr/src/app

# start init system
ENV INITSYSTEM on

RUN [ "cross-build-end" ]

I just needed to include the cross-build-start and end for Docker Hub again, installed python3-pip and pyqt5 dependencies with webkit (in a special version, otherwise it did not work...) and then installed Flask and pywebview. I then proceded to inject the pywebview-erized Calculator:

def start_server():
    app.run(host="0.0.0.0",port="80");
 
if __name__ == '__main__':
    t = threading.Thread(target=start_server)
    t.daemon = True
    t.start()
 
    webview.create_window("Calculator", "http://127.0.0.1:80/")
    webview.toggle_fullscreen()
 
    sys.exit()

I had to create a start_server function to let Flask run in its own thread while pywebview would show the GUI in fullscreen mode and connect to the Flask server.

As last step, I need to rewrite the launch_app.sh

#!/bin/bash

# Disable DPMS / Screen blanking
xset -dpms
xset s off
xset s noblank

# Change Keyboard Layout from US to German
setxkbmap de

# Debug Tools
#xinput --list
#evtest

# Start Window Manager
sudo matchbox-window-manager -use_cursor yes -use_titlebar no & 
#sudo matchbox-window-manager -use_cursor no -use_titlebar no &

# Start Payload App
#gedit
python3 /app/app.py

As seen, I only changed the "Start Payload App" line and now initialize Python 3 with the pywebview/Flask/Caculator app.

And thats it :).

 

To use this app with resinOS, just go to resinos.io, download the latest 2.0.3 release for the RPi, flash the image to your SD Card using i.e. etcher, boot your RPi and ssh to your system IP, using user root and port 22222. From then on, you can just run the app via

docker run --name pywebview --privileged --restart=unless-stopped nmaas87/docker-raspbian_pywebview:jessie-20170426

or you upload the Git src to the /mnt/data folder and build your own version of this pywebview using

docker build -t pywebview .

Please do NOT forget to comment out the RUN [ "cross-build-start" ] and RUN [ "cross-build-end" ] commands!

After that worked, you can start the app via

 

docker run --name pywebview --privileged --restart=unless-stopped pywebview

 

You can also use the app with resin.io by creating an resin.io account, a new RPi App, pushing either the latest or jessie-20170426 tag to resin - and it should built and work :). However, I am more a fan of the flexibility resinOS offers in terms of developing Docker Apps - so I decided to describe this way here.

And with that said, you can now starting working on your own GUI Apps - working in resinOS / resin.io on your RPi or similar device :)! Have fun - and if you'll excuse me, I now have 4 JCTixx Ticket Scanners to build ^^'.