Sonarqube is cool, as the generated results can not only be viewed via the web interface, but also via an plugin in IntelliJ or other Jetbrains IDEs. The installation is quite simple, you just need to download the latest plugin on this website: https://plugins.jetbrains.com/plugin/7238 and install it in the way described here: https://github.com/sonar-intellij-plugin/sonar-intellij-plugin. After that, you can easily hunt down and eliminate those errors :)!
Category: Computer
Computer Stuff
[Docker] Sonarqube with Gogs/Drone to check software quality
Sonarqube is an really useful tool to check for different kind of errors in all kinds of programming languages. If you want to check out Sonarqube in a livedemo, just go over to https://sonarqube.com/. In the last two parts of my Docker series we created an gogs Git Repo as docker container and then went over to integrating gogs with drone (Version 0.4), an CI Build Server - also as Docker container. Today, we want to install Sonarqube as a tool to check upon our software quality, so that - as soon as we push an update to our gogs project, drone will execute an analysis using Sonarqube - which will give us information about different kind of errors and code quality.
# We will start by creating the needed folders:
sudo mkdir /var/sonarqube
sudo mkdir /var/sonarqube/data
sudo mkdir /var/sonarqube/extensions
sudo chown -R yourusername:yourusername /var/sonarqube
# Then we change our drone docker-compose.yml, so that drone and sonarqube will be started at the same time
sonarqube:
restart: unless-stopped
image: sonarqube:lts-alpine
volumes:
- /var/sonarqube/data:/opt/sonarqube/data
- /var/sonarqube/extensions:/opt/sonarqube/extensions
environment:
- SONARQUBE_HOME=/opt/sonarqube
- SONARQUBE_JDBC_USERNAME=sonar
- SONARQUBE_JDBC_PASSWORD=sonar
- SONARQUBE_JDBC_URL=
ports:
- "9000:9000"
- "9092:9092"
drone:
restart: unless-stopped
image: drone/drone:0.4.2
volumes:
- /var/drone:/var/lib/drone
- /var/run/docker.sock:/var/run/docker.sock
env_file:
- ./dronerc
ports:
- "8000:8000"
links:
- sonarqube
After that, we can start the service with docker-compose up -d and see Sonarqube on http://IPADDRESS:9000 (needs some time to load...).
To do an check of i.e. an Java Project, we need to write an new pom.xml file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.appexample</groupId>
<artifactId>appexample</artifactId>
<version>1.0</version>
<name>phpTest</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<sonar.language>java</sonar.language>
<sonar.sources>src</sonar.sources>
<sonar.exclusions>src/test/test.php, src/test/test/*</sonar.exclusions>
</properties>
</project>
and we need a new .drone.yml
cache:
mount:
- /drone/.m2
build:
main:
image: maven:3-jdk-8-onbuild
commands:
- mvn sonar:sonar -Dsonar.host.url=http://IPOFTHESONARQUBESERVER:9000 -Dmaven.repo.local=/drone/.m2
- echo "Sonarqube has been completed."
- mvn clean install -Pcoverage -Dmaven.repo.local=/drone/.m2
- mvn package -Dmaven.repo.local=/drone/.m2
- mvn test -Dmaven.repo.local=/drone/.m2
- echo "Build has been completed."
debug: true
And thats it :). Login into drone, activate your repo as valid CI repo and after that - with every push to that gogs repo, a new sonarqube analysis should be performed.
[Ubuntu] Use Molly-Guard to stop shooting your own leg
If you're working on some dozens of linux servers (or even more than 100,.. as in my case), you end up doing administration via SSH - which is the way to go. And chances are, that you'll get dozens of SSH connections open in dozens of tabs and you did some updates on some of those servers and want to restart this thing with a quick sudo reboot now...
I won't lie if I say, it happend more than once that I accidentally rebooted the wrong server - at least that was the case more than a year ago.
For the last year, since I have been using Molly-Guard - that did not happen once. Why? Because Molly-Guard does stop the reboot command if it detects that you're issuing it from an SSH console - and asks for the server name. If you're entering it correctly - it will reboot. If you're in a frenzy, doing your "sudo reboot now" and enter name serverB while you're on serverA - yep, Molly-Guard will stop you from shooting yourself in the leg. Neat, ain't it?
Oh - and the best part? Ease of use: sudo apt-get install molly-guard
Thats it, you're set, bye.
Nope. Really. No configuration needed. Just install that baby and be safe :)!
[Docker] CI with Drone and Gogs
Now that we have a running Gogs installation for our source code (see here) - we can use Drone to build working software from those repos. Drone is an Continious Integration System, that comes in two flavors: Drone.io - which is the hosted service you could use - or Drone - as self-hosted service. We want to use the later one. To install drone in our existing Docker setup (with gogs already installed) we need to complete following steps:
# Create docker-compose.yml in ~/drone/
cd ~
mkdir drone
cd drone
vi docker-compose.yml
# Copy this content into your docker-compose.yml file
drone:
restart: unless-stopped
image: drone/drone:0.4.2
volumes:
- /var/drone:/var/lib/drone
- /var/run/docker.sock:/var/run/docker.sock
env_file:
- ./dronerc
ports:
- "8000:8000"
# Create the dronerc file
vi dronerc
# Copy this content into your dronerc file
# replace yourserverip with the ip or dns name of your server, i.e. example.com
# replace the gogsport with the port of the http port of the gogs installation, i.e. was 3000 in our example
REMOTE_DRIVER=gogs REMOTE_CONFIG=https://yourserverip:gogsport?open=false DEBUG=true
# Create the needed folders
sudo mkdir /var/drone
sudo chown -R yourusername:yourusername /var/drone/
After that, an docker-compose up will start the service, you can end it via STRG + C and really start it in deattached mode with docker-compose up -d
You can then go to http://yourserverip:8000 and log in into drone with your gogs login and allow access.
The current readme for drone can be found on http://readme.drone.io/usage/overview/ and you'll need to include a .drone.yml in your repos to really build something.
In my example, I used this .drone.yml
cache:
mount:
- /drone/.m2
build:
main:
image: maven:3-jdk-8-onbuild
commands:
- mvn clean install -Pcoverage -Dmaven.repo.local=/drone/.m2
- mvn package -Dmaven.repo.local=/drone/.m2
- mvn test -Dmaven.repo.local=/drone/.m2
- echo "Build has been completed."
debug: true
and this pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany.app</groupId>
<artifactId>WhereIsMyPi</artifactId>
<version>1.0</version>
<packaging>jar</packaging>
<name>Where is my Pi</name>
<url>http://www.nico-maas.de</url>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<!-- Build an executable JAR -->
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.4</version>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<classpathPrefix>lib/</classpathPrefix>
<mainClass>WMP</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>
to build my "WhereIsMyPi" project :).
Happy building!
[Ubuntu] Secure your Apache 2 Reverse Proxy
We got an Apache 2, working as Reverse Proxy to some Docker instances (we won't talk about the nginx vs Apache stuff here for the same reasons we won't talk about vi vs emacs vs xyz ;)) - and somehow we realized that our apps are a little bit too sensitive to allow them from any ip.
First, we want to activate the needed modules. Normally that should not be necessary, but for sake of completeness:
sudo a2enmod mod_authz_core
sudo a2enmod mod_authz_host
Secondly, we want to allow them only from trusted ips. We do redirect them to the docker instances via ProxyPass - but need to create an Location / "catcher" - otherwise we could not use the mod_authz to deny other ips :).
<VirtualHost *:80> ServerAdmin johndoe@example.com ServerName hex.example.com ServerAlias hex RedirectMatch ^/$ https://example.com <Location / > <RequireAll> Require ip 192.168.1.0/24 192.168.2.0/24 192.168.3.0/24 </RequireAll> </Location> ProxyPass "/" "http://127.0.0.1:8020/" ProxyPassReverse "/" "http://127.0.0.1:8020/" </VirtualHost> <VirtualHost *:443> ServerAdmin johndoe@example.com ServerName hex.example.com ServerAlias hex <Location / > <RequireAll> Require ip 192.168.1.0/24 192.168.2.0/24 192.168.3.0/24 </RequireAll> </Location> ProxyPass "/" "http://127.0.0.1:8020/" ProxyPassReverse "/" "http://127.0.0.1:8020/" # Alias /static /srv/example_sw/sw/public_html/ SSLEngine on SSLCertificateFile /etc/ssl/certs/hex.example.com.pem SSLCertificateKeyFile /etc/ssl/private/hex.example.com.key SSLCertificateChainFile /etc/ssl/chains/example-ca-chain.pem </VirtualHost>
That way, hosts from other subnets than 192.168.1.0, 2.0 and 3.0 won't be able to access the proxy and therefore our app :)!
Cisco Prime Infrastructure 3.1 Cheat Sheet
A little cheat sheet for myself. All commands can be used via SSH or Shell:
Show Config: show run
Show Inventory: show inventory (Does show i.e. how many CPUs and RAM is installed. This does match normally to the Version of Cisco Prime. i.E. 8 vCPUs and 16 GB RAM -> Express Plus Type)
Show status of prime: ncs status
Backup OS and Application: backup PI311 repository defaultRepo
Backup Application only: backup PI311appOnly repository defaultRepo application NCS
Activate OS Shell: shell
Locations of different files
defaultRepo: /localdisk/defaultRepo
Config: /storedconfig
License Files: /opt/CSCOlumos/licenses
To transfer the backup files to a safe place, just use scp :).
[Ubuntu] Radsecproy for secure Radius over WAN
Chances are you going to need an radius Auth over WAN - because your Radius and Identity Mngmnt is hosted in the security of the local datacenter of your corp... but the client (i.e. an network switch) is somewhere over the rainbow WAN. You *could* just pipe the radius traffic over the internet - but there be dragons: radius communication is unencrypted. So... just no.
Enter radsecproxy: Radsecproxy is - as the name implies, an radius proxy - which needs to be installed on both servers (the local one in your company, now called SERVER, and the remote one with the switch attached, now called CLIENT) - and does encrypt the communication between both server parts (over WAN i.e.) via TLS.
1.) Install radsecproxy on Server ( sudo apt-get install radsecproxy )
2.) Create CA with generate-CA.sh (in /etc/radsecproxy/) [ https://github.com/owntracks/tools/blob/master/TLS/generate-CA.sh - please change keybits to 4096 bits, thanks! ]
3.) Create Certs (Server, Client) with generate-client.sh (in /etc/radsecproxy/) [ at the end of this post, http://rockingdlabs.dunmire.org/exercises-experiments/ssl-client-certs-to-secure-mqtt - please change keybits to 4096 bits as well! 🙂 ]
4.) Configure /etc/radsecproxy.conf [UPPERLETTERS are constants which you need to change]
# Master config file for radsecproxy
sourceTLS IPADDR_OF_SERVER
listenTLS IPADDR_OF_SERVER:2083
LogLevel 3
LogDestination file:///var/log/radsecproxy/radsecproxy.log
LoopPrevention on
tls default {
CACertificateFile /etc/radsecproxy/ca.crt
CertificateFile /etc/radsecproxy/SERVER_NAME_FQDN.crt
CertificateKeyFile /etc/radsecproxy/SERVER_NAME_FAQN.key
}
client CLIENT_NAME {
host IPADDR_OF_CLIENT
type tls
certificatenamecheck off
secret PW_OF_CLIENT_RADSEC
}
server SERVER_NAME_auth {
host IPADDR_OF_SERVER:1812
type udp
StatusServer on
secret PW_OF_SERVER_FOR_RADIUS
}
server SERVER_NAME_acct {
host IPADDR_OF_SERVER:1813
type udp
StatusServer on
secret PW_OF_SERVER_FOR_RADIUS
}
realm * {
server SERVER_NAME_auth
accountingserver SERVER_NAME_acct
}
# example config for localhost, rejecting all users
client 127.0.0.1 {
type udp
secret TEST_SECRET
}
realm * {
replymessage "User unknown"
}
5.) sudo service radsecproxy restart
6.) Install radsecproxy on Client ( sudo apt-get install radsecproxy )
7.) Copy client cert and ca.crt to Client /etc/radsecproxy
8.) Configure /etc/radsecproxy.conf [UPPERLETTERS are constants which you need to change]
#sourceUDP 127.0.0.1
sourceUDP IPADDR_OF_CLIENT
listenUDP *:1812
listenUDP *:1813
LogLevel 3
LogDestination file:///var/log/radsecproxy/radsecproxy.log
LoopPrevention on
tls default {
CACertificateFile /etc/radsecproxy/ca.crt
CertificateFile /etc/radsecproxy/CLIENT_NAME_FQDN.crt
CertificateKeyFile /etc/radsecproxy/CLIENT_NAME_FQDN.key
}
client CLIENT_NAME {
#host 127.0.0.1
host IPADDR_OF_CLIENT
type udp
secret CLIENT_RADIUS_SECRET
}
client SWITCH_NAME {
host SWITCH_IP
type udp
secret SWITCH_RADIUS_SECRET
}
server SERVER_NAME {
certificatenamecheck off
host IPADDR_OF_SERVER
type tls
StatusServer on
secret PW_OF_CLIENT_RADSEC
}
realm * {
server SERVER_NAME
accountingserver SERVER_NAME
}
# example config for localhost, rejecting all users
client 127.0.0.1 {
type udp
secret TEST_SECRET
}
realm * {
replymessage "User unknown"
}
9.) sudo service radsecproxy restart
10.) If you now point your switches to the CLIENT_IP with the correct credential, it should go via the radsecproxy to your main radius server and get the connection working. Please pay attention that on your CLIENT site no radiusd daemon is allowed to run, as it would block the ports needed for radsecproxy / radius. Make use of the radsecproxy log files to see, wheter the two radsecproxy servers do connect and talk to each other :).
[Review] Western Digital WD Labs PiDrive 314GB
Today, I want to do my first review on this blog, concerning the Western Digital / WD Labs PiDrive 314GB. I received this nice little drive from the guys over at PiAndMore as a gift for my three presentations at the PiAndMore 9 (among other loot ;)). So, what is this PiDrive and what is so special about it? Well it is in fact a small 2.5 HDD with integrated Sata to USB 3.0 Controller (with the matching "two-piece" connector). It was specially designed by WD to work with the Raspberry Pi Series and gives the RPi 314 GB of space (hence the name PiDrive - however, there is an 1 TB Version available as well).
So what did I get?
The PiDrive itself, a USB Micro to USB A Connector cable, as well as a power brick. Unfortunately, the power brick turned out to be a UK Version, not a German one, so I needed to get an adapter for that. The Brick itself was a beefy (2A@5V) supply, probably meant to power the RPi and PiDrive at the same time.
I loved the "Pi" finish at the package.
The package itself included the PiDrive (wrapped in a antistatic bag, already removed here ;)) and an business card with some tips on "getting started"
The Drive itself - small, lightweight and quite nice to look at.
Ok, after unpacking, I did some "Google Research" as I normally do when it comes to Harddisks and SSDs. A bunch of people (mostly with 1 TB Versions) already explained they were having problems with the Firmware of the Drive and that there already were an update available. So to not get into problems myself, I turned to WDs Support Website (http://support.wdc.com/downloads.aspx?lang=en) and downloaded the "WD Universal Firmware Updater for Windows". I plugged the included USB Micro Cable into the PiDrive and connected it to my Laptop and saw that this Drive also got an *REALLY BRIGHT* white LED as Status Indicator. However, the LED was blinking on/off continuously and I decided that the Drive probably did not got enough juice from the one USB Port. So I combined an old 2.5 HDD cable (USB A to 2x USB A Y Cable) with an USB Micro OTG Cable, so that I got 2x USB A connectors on the one side (which plugged into my Laptop) and the USB Micro connector on the other side, which connected to the PiDrive. Now I got an steady white lit LED - and I could find the Drive in the Device Manager. After I fired up the Firmware Updater, I directly got the info about a new firmware [1.0.12] and got the drive flashed. Everything went fine :).
So, from that point on, I wanted to use the Drive with one of my Raspberry Pis. As I still got some old RPi Model B, v2.0, I tried to give it a try - but it did not work. The LED was just flashing again, and it seems like I would need an active USB Hub or my special cable to get it working [it did also not show up in dmesg or blkid]. Then I realized, that I already had seen a special connector cable from WD (this one: http://wdlabs.wd.com/products/raspberry-pi-accessories/#pidrive_cable) which had sadly not been sent to me. This cable does directly plug into the power brick and does power up the RPi as well as the PiDrive, so that the RPi did not need to supply about 1 A via the USB port to the PiDrive. Bummer.
Next thing I saw, was the max_usb_current=1 which had to be added to the /boot/config.txt on the RPi card. This - however, does only work from RPi+ Modells on onwards (so RPi A+/B+, RPi 2 B) and should not be needed on the RPi 3, as this is already "on by default". Why does this only works on that versions? They got an little Mosfet included there to GPIO 38 which does activate a kind of "high power mode" for the USB controller. So to see wheter your little modification did work, just try gpio -g read 38 and 1 will tell you, that it worked. An lsusb -v 2>/dev/null | grep -e 'MaxPower' -e 'Bus [0-9]' will also tell you, how much power every device on the RPi draws (Thanks to https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=105502&start=25).
So, after I saw that my RPi 3 got its max_usb_current enabled, I rebooted it with the PiDrive attached to the included USB cable. However, the Drive LED did also blink like on the RPi B v2.0, which was really strange. And it did also not show up in the blkid nor in dmesg. So I did unconnect the PiDrive and used an TeckNet 0.3m USB Micro USB cable - and voilá - it worked right out of the box! With sudo blkid I could see my connected drive and did find out that it already had been mounted to /media/USERNAME/WD_PiDrive_314GB.
So, I did not yet do any speed tests and stuff, but I am pretty happy with this little drive. I am really looking forward to play with it a little bit more. Is it worth the price? Probably. I really like WD drives, I make really good use of WD Blues in my Desktops, WD Greens in some Backup Systems and WD Reds in my NAS/Small Servers. I never had any problems with WD Drives and so I am looking forward to a "nice smooth ride" with this HDD (hopefully ;)) as well.
Thanks a lot for Western Digital to giving these PiDrives out :)!
More infos about i.e. the Berry Boot can be found here: http://wdlabs.wd.com/Support/#berryboot / http://www.berryterminal.com/doku.php/berryboot, as well as some nice step-by-step setup in there forums: https://community.wd.com/t/how-to-install-raspbian-on-pidrive-using-windows-pc-step-by-step/134723
[PiAndMore] resin.io Presentation
Here is my presentation to resin.io @ PiAndMore 9 (Trier, 11.06.2016)
resinio_PiAndMore9.pdf (1,03 MB, PDF)
Videorecording of the talk can be found here
[PiAndMore] platformIO Presentation
Here is my presentation to platformIO @ PiAndMore 9 (Trier, 11.06.2016)
platformIO_PiAndMore9.pdf (830 KB, PDF)
Videorecording of the talk can be found here