[resinOS] Build resinOS from scratch

As the time of writing, resinOS is available for Download at Version 2.0.6+rev3.dev for Raspberry Pi 3. This build, however, is nearly 2 weeks old and in the meantime, something great happend: Docker has finally updated to Version 17.03.1 - upgraded from the old ~10 (ten-ish) version - which was not that cool (and without Swarm ;)). So, it is a good idea to get to know how to build your own resinOS in case you really want to live on the bleeding edge ;).

Install Dependencies (Ubuntu 16.04 LTS)

sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
     build-essential chrpath socat cpio python python3 python3-pip python3-pexpect \
     xz-utils debianutils iputils-ping libsdl1.2-dev xterm

goto /, because this build will create very long filenames

cd /

clone the repo, maybe some root power is needed here 😉

git clone https://github.com/resin-os/resin-raspberrypi
cd resin-raspberrypi
git submodule update --init --recursive

you would be done here and could build your own resinOS with the build command,
however, if you really want to pull the latest upgrades...

cd layers/meta-resin
git checkout master
git pull
cd ../..

finally build resinOS for Raspberry Pi 3

./resin-yocto-scripts/build/barys -r --shared-downloads $(pwd)/shared-downloads/ --shared-sstate $(pwd)/shared-sstate/ -m raspberrypi3                       

after quite some time, you'll find the image in

build/tmp/deploy/images/raspberrypi3/resin-image-raspberrypi3.resinos-img

 

There is quite a lot of stuff you can change on your resinOS, so be sure to check out https://resinos.io/docs/custombuild/ for more documentation on that topic. Have fun :)!

[resinOS] Dockerize your own Flask GUI with resinOS

I have been working with Docker and resin.io as well as resinOS for quite some time now and I am actively using those services actively for different projects: The possibility to just throw away a container and build it anew in record breaking time is just awesome and also the fact that I can now just ship images to some device, deploy it there - and it will just work. As some of you know, I am working as volunteer in different NPOs, i.e. the RepairCafé Trier in which I repair Computers and Mobile phones for free, the PiAndMore / CMD e.V. on which I teach interessted guys and gals about Raspberry Pi and Linux (and even did two presentations on resin.io and resinOS, both linked on this blog - and as video :)) - and also one other NPO which is trying to give information about Japanese Culture. For those events I already created an Tickting System called JCTixx, which uses QR Codes to sell and check tickets for those named festivals. However, I created the Scanner Hardware in 2010, just starting my second year in an Apprenticeship as IT Systemselectronics guy, without anything as a Raspberry Pi available. However, being a bit creative, I just took some old Linux routers, threw the good old OpenWRT on those and soldered directly to the its testpoints / CPU to get the GPIO pins I needed ;). That worked very well - however, nowadays, TLS 1.2 and such eat away those small MIPS cores - and I could - with a lot of work - just get them working with said crypto in 2015. However, being 7 years old now, I need new Scanners - and more flexible way of delivery software as well. So resinOS it is! 🙂

Full disclosure: As I am currently developing the Hardware for the Scanners and I am always looking for some Raspberry Pis and other hardware, I stumbled upon resin.ios #blog4swag program and decided to make this tutorial because of two reasons: 1.) I actually need an dockerized GUI for the scanners and therefore am working on a good solution to the problem - and as usually - I want my dear readers to get some interesting stuff to read. 2.) Maybe I can acquire some hardware support from the resin guys which then would help me out to get my NPO helped :). So in the end, everyone wins and no one dies - to cite Doctor Who :).

So, lets get started!

1.) Getting some GUI working at all
I saw some users experimenting with Docker and Chromium along the x86 systems - however, the first resin project I stumbled upon which really made me think about putting the GUI within a container, was resin.ios ElectronJS example. This thing really rocked, but I saw that I needed to work on something a bit easier which would just deploy some sort of GUI - and being as simple as it gets. So I started to look around and stumbled across different projects, namely
RPiBrowser-resin.io and resin-electron-app. Those two were a great starting point and using resin.ios ElectronJS example as well, I started hacking the code together, which I finally submitted to this Git Repo:
docker-raspbian_xinit. With the great help of Gergely Imreh from the resinIO Team (thanks a lot :)!) I got the project working and pushed it onto Docker Hub. To walk you through the Git Repo: I made two versions, one nailed to the release of the 26.04.2017 Raspbian and one made from the latest version. I basically grabbed resin.ios Raspbian from Docker Hub, injected the qemu files (which they did show in one tutorial) and used this as a starting point. It enables Docker Hub to build the Raspberry Pi Images, so that you can use them on your Pi.

If you want to build the docker-raspbian_xinit image yourself, you need to remove the RUN [ "cross-build-start" ] and RUN [ "cross-build-stop" ] command from the Dockerfile :).

The Dockerfile itself builds a basic system with xinit / xorg, alsa and even touchscreen support (xinput-calibrator). It generated german locales and a user called pi (which is both not needed, but I included it as matter of personal preference), copys the needed files and enables systemd. Systemd will start the xinit-docker.service which invokes the start.sh which does prepare the system, i.e. adds the container name to the /etc/hostname, so that sudo will work, activates its own SSH server on port 22, includes a touchscreen calibiration file - if available, creates a very needed config folder, removes files from old X11 sections, sets volume to 100% on speakers and phones and then starts the xinit process with the launch_app.sh :).

The launch_app.sh then works in the xinit context and does xinit specific stuff: It disables screen blanking, sets the keyboard to the created german locales (you can comment this out as well if you want or overwrite the file ;)) - then - finally - starts the matchbox-window-manager without titlebar, but WITH keyboard and mousesupport and launches....
... gedit.
Yeah. Right. I know that is a real disappointment. But I needed a small tool which would not add too much file size and show that keyboard and mouse are working - so I just went for gedit ^^'. Sorry if you were hoping to find something really awesome and cool at this place. But nonetheless, it works, and this image as well as the Git Files should serve as a starting point for your own Xinit adventure - so, thats the reason :).

After I got it finally working, I thought about my personal usecase: I will be using a lot of Python and thought about using Flask as a GUI. However, Flask is just a webframework and does not have the ability to shown something "GUI-like" - thus needing some kind of Webbrowser - and this part could be found in shape of pywebview. Pywebview just includes some website, app or similiar into a small GUI Frame with Webkit Browser in it. Cool! Exactly what I needed. However, I did not have time get to work on my own UI - and wanted to jumpstart the project by getting the Docker Container to work - so I decided to grab a cool small Flask Web GUI project and use this to showcase how easy it is to built a self-starting Docker Container GUI - on resinOS. And with that in mind, I went for the very cool helloflask Calculator by Grey Li.

2.) Getting a Flask GUI working - using 1. 😉
Ok, as soon as I got my Xinit project working, I decided to use this a base image, just overwriting changes in the system and injecting the files needed to run the pywebview'd Calculator.
With that in mind, the Dockerfile became quite small (you'll find the Source Files on Github and the Image on Docker Hub as well :))

FROM nmaas87/docker-raspbian_xinit:jessie-20170426
MAINTAINER Nico Maas <mail@nico-maas.de>
ENV DEBIAN_FRONTEND noninteractive
RUN [ "cross-build-start" ]

RUN apt-get update \
    && apt-get install -yq --no-install-recommends \
        python3-pip python3-pyqt5 python-gi gir1.2-webkit-3.0=2.4.9-1~deb8u1+rpi1 gir1.2-javascriptcoregtk-3.0=2.4.9-1~deb8u1+rpi1 libjavascriptcoregtk-3.0-0=2.4.9-1~deb8u1+rpi1 libwebkitgtk-3.0-0=2.4.9-1~deb8u1+rpi1 \
    && apt-get autoremove -qqy \
    && apt-get autoclean -y \
    && apt-get clean && rm -rf /var/lib/apt/lists/* && mkdir /var/lib/apt/lists/partial \
    && pip3 install Flask pywebview \
    && mkdir /usr/src/app/templates /usr/src/app/static

# copy program
COPY src /usr/src/app

# start init system
ENV INITSYSTEM on

RUN [ "cross-build-end" ]

I just needed to include the cross-build-start and end for Docker Hub again, installed python3-pip and pyqt5 dependencies with webkit (in a special version, otherwise it did not work...) and then installed Flask and pywebview. I then proceded to inject the pywebview-erized Calculator:

def start_server():
    app.run(host="0.0.0.0",port="80");
 
if __name__ == '__main__':
    t = threading.Thread(target=start_server)
    t.daemon = True
    t.start()
 
    webview.create_window("Calculator", "http://127.0.0.1:80/")
    webview.toggle_fullscreen()
 
    sys.exit()

I had to create a start_server function to let Flask run in its own thread while pywebview would show the GUI in fullscreen mode and connect to the Flask server.

As last step, I need to rewrite the launch_app.sh

#!/bin/bash

# Disable DPMS / Screen blanking
xset -dpms
xset s off
xset s noblank

# Change Keyboard Layout from US to German
setxkbmap de

# Debug Tools
#xinput --list
#evtest

# Start Window Manager
sudo matchbox-window-manager -use_cursor yes -use_titlebar no & 
#sudo matchbox-window-manager -use_cursor no -use_titlebar no &

# Start Payload App
#gedit
python3 /app/app.py

As seen, I only changed the "Start Payload App" line and now initialize Python 3 with the pywebview/Flask/Caculator app.

And thats it :).

 

To use this app with resinOS, just go to resinos.io, download the latest 2.0.3 release for the RPi, flash the image to your SD Card using i.e. etcher, boot your RPi and ssh to your system IP, using user root and port 22222. From then on, you can just run the app via

docker run --name pywebview --privileged --restart=unless-stopped nmaas87/docker-raspbian_pywebview:jessie-20170426

or you upload the Git src to the /mnt/data folder and build your own version of this pywebview using

docker build -t pywebview .

Please do NOT forget to comment out the RUN [ "cross-build-start" ] and RUN [ "cross-build-end" ] commands!

After that worked, you can start the app via

 

docker run --name pywebview --privileged --restart=unless-stopped pywebview

 

You can also use the app with resin.io by creating an resin.io account, a new RPi App, pushing either the latest or jessie-20170426 tag to resin - and it should built and work :). However, I am more a fan of the flexibility resinOS offers in terms of developing Docker Apps - so I decided to describe this way here.

And with that said, you can now starting working on your own GUI Apps - working in resinOS / resin.io on your RPi or similar device :)! Have fun - and if you'll excuse me, I now have 4 JCTixx Ticket Scanners to build ^^'.

[RaspPi] How To Compile OpenWRT Trunk with latest Raspberry Pi Kernel 3.10 for the RPi on Ubuntu 12.04

EDIT: This post is from 30.09.2013 and should be the "how to" to this post. However, I never released it for a now unknown reason. Because of the interest in this topic, I still will publish it today so that it can stand as some kind of documentation of what I did back then. However, I might not work anymore.

This is the latest Guide.

What are we doing here?
Building OpenWRT Trunk with the offical RPi 3.10 (next) Kernel.

Packages:
You can use some packages of the offical OpenWRT 12.09 Attitude Adjustment repo.
However, most kernel modules and stuff has to be built but yourself, as I
got no repo. Sorry!

Dependencies needed

sudo apt-get install build-essential subversion git-core libncurses5-dev zlib1g-dev gawk flex quilt libssl-dev xsltproc libxml-parser-perl mercurial bzr ecj cvs unzip

Prepare Enviroment (Trunk)

cd ~
mkdir rpi_openwrt
cd rpi_openwrt
git clone git://git.openwrt.org/openwrt.git
cd openwrt
mkdir dl

Install Packages

./scripts/feeds update -a
./scripts/feeds install -a

Check Enviroment

make defconfig
make prereq

Download Kernel and Prepare:

cd /tmp

wget https://github.com/raspberrypi/linux/archive/rpi-3.10.y.tar.gz

tar xfvz rpi-3.10.y.tar.gz
mv linux-rpi-3.10.y linux-3.10.13
tar -cJvf linux-3.10.13.tar.xz linux-3.10.13/
mv linux-3.10.13.tar.xz ~/rpi_openwrt/openwrt/dl/

Prepare Target

cd ~/rpi_openwrt/openwrt/target/linux/brcm2708/

Remove the RPi Kernel specific patches

we don't need them because we use the ORIGINAL kernel

- and not some retailored standard kernel!

mv patches-3.10 patches-3.10.bkp
mkdir patches-3.10

check wheter the makefile does try to catch the right kernel version

vi Makefile

change LINUX_VERSION to LINUX_VERSION:=3.10.13

Prepare Generic Patches

cd ../generic/patches-3.10

Normally it should be ok to just remove some of the patches like this.

if make target/linux/{clean,prepare} V=99 does fail in patching stage

you would need to remove another patch here

but for the time beeing, lets kill these files which are known to

wreck it:

rm 063-arm-fix-fiq-vivt.patch
rm 880-gateworks_system_controller.patch

#Make
cd ~/rpi_openwrt/openwrt/
make menuconfig # Choose BCM2708 Chipset, Raspberry Pi Board, Exit and Save
make target/linux/{clean,prepare} V=99 # does extract kernel and patch it
make kernel_oldconfig # builds some tools
make kernel_menuconfig # configures kernel
make menuconfig # to change RPi Patterns, i.e.:

Packages rep:

Image configuration -> Version configuration options -> Release Repo to:

http://downloads.openwrt.org/attitude_adjustment/12.09/%T/generic/packages

Main Partition Size from 48 to 64:

Target Options -> Root File System Size, change from 48 to 64 MB or more

Exit and Save

make V=99 -j 3

Files are in ~/rpi_openwrt/openwrt/bin/brcm2708
( i.e. openwrt-brcm2708-sdcard-vfat-ext4.img )
New build:
make kernel_menuconfig
make menuconfig
make V=99 -j 3

Raspberry Pi and its Hardware Random Number Generator (RNG)

Due to a Stackoverflow Post I got to know the fact the BCM2708 / BCM2835 contains a Hardware Random Number Generator (RNG). Two blogposts described how to setup this little module, however, they were outdated, as the needed kernel module is directly baked into the latest 4.x kernel, which ends with the fact that /dev/hwrng already exists :).

The only thing left to do is the installation of the rng-tools, which is easy as pie:

sudo apt-get update
sudo apt-get install -y  rng-tools

After that, we also need to point rng-tools to the right source, by adding

HRNGDEVICE=/dev/hwrng

to /etc/default/rng-tools (need root rights for that :))

After that, the service can be restarted and used:

sudo service rng-tools restart

PS: This RNG seems to be available on all other RPis as well, not only RPi 1 🙂

 

Other articles:

http://scruss.com/blog/2013/06/07/well-that-was-unexpected-the-raspberry-pis-hardware-random-number-generator/

http://fios.sector16.net/hardware-rng-on-raspberry-pi

[Raspberry Pi] Warning - Kernel 4.4.38 breaks boot on RPi 1 & 2

About 14 days ago, RPi Kernel Version 4.4.38 was published. However, something went very wrong somewhere: Raspbery Pi Models 1 and 2 do not boot anymore. As a quickfix I would recommend to download the 4.4.37 Kernel from the Github Repo (https://github.com/raspberrypi/firmware/) and replace the boot Partition on your RPi 1 or 2 SDCard with the /boot path from the 4.4.37 ZIP file - and it should boot again.
If you're RPi is still working - do not update your kernel until this problem is solved! (Issue on Github).

EDIT: Reason for the issue was mostly the open

device_tree=

configuration in the config.txt

Removing this option solved the problem.

Raspberry Pi Pixel - on your x86/x64

The guys over at the Raspberry Pi Foundation - in cooperation with the MagPi Team came up with a cool project: They ported the new "Pixel" desktop from Raspbian over to Debian, so you could use it on your normal x86/x64 machine. The DVD is included in the latest MagPi edition, however, you can also download the DVD from the Raspberry Pi Website, the link is in the article: https://www.raspberrypi.org/magpi/debian-pixel-mac-pc/

PS: Also the "root" directory of the pixel_x86 seems to be: http://downloads.raspberrypi.org/pixel_x86/images/

Raspberry Pi Remote Desktop

To use the RPi via Remote Desktop, different protocols and programs exist, i.e. VNC. Due to simplicity of use, however, I mostly chose XRDP, which works with the Microsoft Remote Desktop Tools. However, you must not install it via sudo apt-get install xrdp - as this will install it with realvnc and break xrdp (Package error which has not been fixed yet in the recent "Pixel" Release of Raspbian...).

Instead, use:
sudo apt-get install tightvncserver xrdp

As it will install tightvnc first and xrdp will use it after that as source. You can then use mstsc on your Windows Machine or any other RDP Viewer to access your RPis Desktop.

PS: Yes, the new Pixel Version of Raspbian does contain an VNC Server which can be enabled very easily via the GUI "Raspberry Pi Configuration" Tool, if you want to use VNC directly without RDP. However, I did chose this way, as the old VNC server installations never worked for me and I got used to this kind of "workaround" (more or less a "works-for-me"(tm) solution)