Reflashing Deltaco Smartplug SH-P01 to Work With Home Assistant

This is a documentation of a ongoing work where I’m trying to get Deltaco Smartplug SH-P01 to work with Home Assistant.

Since the Home Assistant does not have a native support for the device I’m planning to:

  • Create an ESPhome firmware with the configuration I found from the Home Assistant discussion board
  • Flash the firmware with tuya-convert
  • Finally, control the device with Home Assistant

Creating ESPhome Firmware

Since I already had Docker installed on my laptop I entered

docker pull esphome/esphome

and got the ESPhome Docker image. After this I created relay_1.yaml with following content:

esphome:
  name: relay_1
  platform: ESP8266
  board: esp01_1m

# Your WiFi SSID and passphrase is defined here
# https://esphome.io/components/wifi.html
wifi:
  ssid: "YOUR_WLAN_SSID"
  password: "YOUR_WLAN_PASSPHRASE"

# Enable fallback hotspot (captive portal)
# In case the device can't connect the host defined above
# it starts to work as an access point with these settings
# https://esphome.io/components/captive_portal.html
  ap:
    ssid: "Smartplug Deltaco 1"
    password: "RANDOM_PASSWORD_NEEDED_IN_CASE_THE_DEVICE_CANT_CONNECT_WLAN"

captive_portal:

# Enable logging
logger:

# Enable Home Assistant API
# https://esphome.io/components/api.html
api:
  password: "RANDOM_PASSWORD_FOR_HOME_ASSISTANT_TO_CONNECT"

# Enable Over The Air update component
# https://esphome.io/components/ota.html
ota:
  password: "RANDOM_PASSWORD_FOR_ESPHOME_OTA_UPDATES"

binary_sensor:
  - platform: gpio
    pin:
      number: GPIO13
      mode: INPUT_PULLUP
      inverted: True
    name: "Deltaco SH-P01 Button"
    on_press:
      - switch.toggle: deltaco_relay_1
  - platform: status
    name: "Deltaco SH-P01 Status"

output:
  - platform: esp8266_pwm
    id: deltaco_smartplug_blue_led
    pin:
      number: GPIO5
      inverted: True
      
switch:
  - platform: gpio
    name: "Deltaco SH-P01 Relay"
    id: deltaco_relay_1
    pin: GPIO12

light:
  - platform: monochromatic
    name: "Deltaco SH-P01 blue LED"
    output: deltaco_smartplug_blue_led

This was taken from the discussion board referred above. All I needed to do was to edit the SSIDs and passwords and compile a new firmware:

docker run --rm -v "${PWD}":/config -it esphome/esphome relay_1.yaml compile

This printed a lot of debug information and ended with following lines:

Building .pioenvs/relay_1/firmware.bin
Retrieving maximum program size .pioenvs/relay_1/firmware.elf
Checking size .pioenvs/relay_1/firmware.elf
DATA: [===== ] 45.3% (used 37148 bytes from 81920 bytes)
PROGRAM: [==== ] 41.5% (used 425048 bytes from 1023984 bytes)
Creating BIN file ".pioenvs/relay_1/firmware.bin" using ".pioenvs/relay_1/firmware.elf"
========================= [SUCCESS] Took 29.83 seconds =========================
INFO Successfully compiled program.

And yes, the compiled firmware was at relay_1/.pioenvs/relay_1/firmware.bin! All I needed to do is to get this firmware to the device.

Flashing the Firmware with tuya-convert

The tuya-convert offers a docker image to make the flashing, but I could not make it work. Therefore I installed the script and required packages directly to my Ubuntu as instructed in the README:

git clone https://github.com/ct-Open-Source/tuya-convert
cd tuya-convert
nano -w config.txt
# Changed value WLAN to my device name, wlx7cdd901255af
# Enumerate your device names with "ifconfig"
sudo ./install_prereq.sh

At this point I disconnected by WLAN from the local access point so it was available for tuya-convert. I also copied the ESPhome firmware.bin to tuya-convert/files/relay_1.bin. Then it was time to start the flashing script:

sudo ./start_flash.txt

When instructed I joined my phone to new WLAN access point “vtrust-flash” and after this was done I reset my SH-P01. It took me a while to understand why it did not respond when I pressed the button for the magic 6 seconds. Finally it turned out that the device already had connected to the AP.

The script did the trick. First it downloaded the firmware backup and after that it asked which binary file I wanted to install. After a while the relay_1.bin was uploaded.

Installing Home Assistant

Since I have a Xen server I installed the Home Assistant to a fresh VM running Debian Stretch. The Home Assistant configuration was quite uneventful as everything went as planned.

Since the server and the SH-P01 were on the same network segment (same LAN) the switch was discovered right away. All I needed to do was to give the API password I defined in relay_1.yaml and wrote to the ESPhome firmware.

Everything worked: the relay, the light and the button.

Flashing More Devices

To add more devices just make a copy of the ESPhome configuration (relay_1.yaml) and edit the esphome:name -setting:

esphome:
  name: relay_2
  platform: ESP8266
  board: esp01_1m

The device names must be unique and they’re used for example as a hostname of the device. Home Assistant creates its ID:s from device names (e.g. “Deltaco SH-P01 Relay” becomes “switch.deltaco_sh_p01_relay”). In case you’re worried about these entity ID:s you might want to give distinctive names to avoid overloading (“switch.deltaco_sh_p01_relay_2”, “switch.deltaco_sh_p01_relay_3” etc.). You might also want to change some of the passwords. After this just repeat the steps with the new firmware.

The sample configuration toggles the relay when the button is pressed. If you don’t want this behaviour but want to control the relays by the Home Assistant automations just remove this codeblock:

on_press:
  - switch.toggle: deltaco_relay_1

Starting MariaDB on Ubuntu Bionic causes timeout

Starting MariaDB 10.1 causes timeout. Syslog says:

Apr  5 20:55:53 megis systemd[1]: Starting MariaDB 10.1.38 database server...
Apr  5 20:55:53 megis mysqld[6892]: 2019-04-05 20:55:53 140074458918016 [Note] /usr/sbin/mysqld (mysqld 10.1.38-MariaDB-0ubuntu0.18.04.1) starting as process 6892 ...
Apr  5 20:55:54 megis kernel: [  980.249533] kauditd_printk_skb: 2 callbacks suppressed
Apr  5 20:55:54 megis kernel: [  980.249535] audit: type=1400 audit(1554486954.089:38): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld" name="run/systemd/notify" pid=6892 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=121 ouid=0
Apr  5 20:57:23 megis systemd[1]: mariadb.service: Start operation timed out. Terminating.

There can be more lines where AppArmor says

audit(1554487043.725:40): apparmor="DENIED" operation="sendmsg" info="Failed name lookup - disconnected path" error=-13 profile="/usr/sbin/mysqld"

Some notes in the net suggest that you should set the TimeoutSec=infinity in the systemd configuration, but this did not help in my case. I had to disable the AppArmor for /usr/sbin/mysqld. It wasn’t enough to put it to complain mode. Here are the instructions:

Get apparmor-utils

If you can’t execute “sudo aa-disable” you have to install the AppArmor utilities:

sudo apt install apparmor-utils

Create AppArmor profile for MariaDB

By default the AppArmor profile for /usr/sbin/mysqld is empty which causes “aa-disable” to fail. Add following lines to “/etc/apparmor.d/usr.sbin/mysqld”:

/usr/sbin/mysqld {
}

Disable AppArmor for /usr/sbin/mysqld

After this say:

sudo aa-disable /usr/sbin/mysqld

Writing keyboard layout for xorg – Notes

I had a privilege to be the first person in universe to implement Skolt Sami keyboard for Linux. I had two references:

The job was pretty straight-forward: change the keyboard combinations in the file.

Keycodes

The idea of xkb file is to translate keycodes to characters. The first step is to identify the keys in the keyboard. Each key produces its own keycode. For a start I used Figure 2 in this documentation to get an idea ofo the keycodes. However, my keyboard layout (and the documented Sami layout) differed from fig 2.

To get a keycode for a key:

$ xev -event keyboard

KeyRelease event, serial 28, synthetic NO, window 0x2200001,
 root 0x12d, subw 0x0, time 85909731, (-350,-52), root:(244,267),
 state 0x0, keycode 38 (keysym 0x61, a), same_screen YES,
 XLookupString gives 1 bytes: (61) "a"
 XFilterEvent returns: False

The xev utility gives you the keycodes but this code cannot be used as such in your xkb definitions where the keys are identified with a hex tuplet (e.g. AC01) or a tag (e.g. LSGT). To get your xkb-compliant key identifier:

$ xkbcomp :0 keyboad-mapping.txt
$ grep 38 keyboard-mapping.txt
 <AC01> = 38;
 <PROP> = 138;
 <I238> = 238;
 key <I238> { [ XF86KbdBrightnessUp ] };
 { [ 38, 18 ] },
 left= 382;

Wow! Now I know that A in my keyboard -> keycode 38 -> AC01 in xkb definition.

Characters not defined in keysymdef.h

Most of the xkb howtos I browsed explained that all the symbols are defined in the keysymdef.h. Well, that wasn’t the case with Sami characters. Instead of using the defined keyboard symbols I had to use Unicodes. Say we want to implement Đ character.

  1. If possible, try to get the character to a clipboard. I ended up using Windows implementation to get all three different hyphens right.
  2. Go to http://www.utf8-chartable.de/unicode-utf8-table.pl and locate the character there. This evil beast was hiding in the page “Latin Extended-A”.
  3. Get the Unicode code point (for Đ it is U+0110) which can be used in the xkb definition (“U0110”).

When implementing my second keyboard layout I found a search tool which helped me a lot in finding exotic characters. Workflow:

  1. Start a Windows with selected keyboard layout in a virtual machine. Start a Notepad.
  2. Push a key to see what character appears to the notepad. Copy and paste it to search tool.
  3. The Unicode number can be entered directly to the keyboard definition but I tried to find the symbol from the keysymdef.h whenever I could. It makes keyboard definition file more readable.
  4. Repeat with Shift+Key, AltGt+Key and Shift+AltGr+Key.

VoidSymbol

To avoid strange characters appear when you type the undefined combinations use “VoidSymbol”. So if you define keys like this:

 key <AC01> { [ a, A ] };
 key <AC02> { [ s, S ] };

and press AltGr+A or AltGt+Shift+S you might get some random characters To avoid these chars use VoidSymbol to make sure that no definions are inherited (or whatever neat feature causes this):

 key <AC01> { [ a, A, VoidSymbol, VoidSymbol ] };
 key <AC02> { [ s, S, VoidSymbol, VoidSymbol ] };

Using xrdp Desktop for Everyday Life (Ubuntu Xenial)

Preface: I Need the Latest xrdp

During the holidays the keyboard of my laptop started to indicate that there is no eternity in the world. And especially not among the cheapo laptops. Since a new laptop with a decent screen and processor would const a fortune, I decided to try another approach. Why don’t I install a desktop Linux to my Xen server and use it for daily tasks? In this setup the laptop wouldn’t need to have a capable processor.

I wanted a setup where the connection would be direct without any servers between. I don’t want anyone to follow my mediocre node.js stumbles or follow my bank sessions. TeamViewer and similar helpdesk tools are no-go.

I did some testing with NoMachine NX, but it felt a bit clumsy although the setup was painless. Using X was not sufficient option since you want to hear the sounds when watching YLE Areena, right?

Finally I ended up working with xrdp, a Linux realisation of Microsoft’s Remote Desktop Protocol. The protocol supports sound and its performance should be enough for multimedia. Well, it quickly turned out that the pre-packaged version of xrdp (the server) did not have support for sound. It didn’t support mouse wheel either which is another “must”. Thus I needed to compile the latest version myself.

The Beef: Compiling xrdp With Sound

Compiling xrdp

Get the source from xrdp site. My version was 0.9.4. First compile xrdp. Make sure you have build-essential installed and give it a try:

cd xorg/
./configure
make
sudo make install

Compiling X11 with RDP support

It turns out that the pre-packaged Xorg does not have RDP support and we need to compile X11 with RDP support as well:

apt-get install xsltproc flex bison libxml2-dev python-libxml2
cd xorg/X11R7.6/
./buildx default

Install (copy) the binary to /usr/local/bin as root:

cp rdp/X11rdp /usr/local/bin

Now start (again, as root):

xrdp-sesman
xrdp

By default xrdp server supports multiple session types. Since I want to provide the RDP protocol only I have to edit /etc/xrdp/xrdp.ini. The session types are in the bottom (after [Channels] section).

Leave only X11rdp connection type:

[X11rdp]
name=X11rdp
lib=libxup.so
username=ask
password=ask
ip=127.0.0.1
port=-1
xserverbpp=24
code=10

Getting sound to work

Building Pulseaudio sound modules requires a bit extra work.

First, get the version of your Pulseaudio installation:

$ pulseaudio --version
pulseaudio 8.0

Get the pulseaudio source and unpack it somewhere. I used ~/src/pulseaudio and now I have ~/src/pulseaudio/pulseaudio-8.0. Now go to forementioned directory and compile your Pulseaudio:

apt-get install intltool libjson-c-dev libsndfile1-dev
./configure --without-caps
make

Now back to xrdp:

cd xrdp/sesman/chansrv/pulse

Edit PULSE_DIR (in xrdp/sesman/chansrv/pulse/Makefile) to point to the path you have your Pulseaudio source. In my example:

PULSE_DIR = /home/matti/src/pulseaudio/pulseaudio-8.0

Then simply:

make

Now you have module-xrdp-sink.so and module-xrdp-source.so. Copy them to Pulseaudio libs (as root)

cp *.so /usr/lib/pulse-8.0/modules/

Add following lines to your /etc/pulse/default.pa:

load-module module-xrdp-sink
load-module module-xrdp-source
set-default-sink xrdp-sink
set-default-source xrdp-source

Note! In my case pavucontrol crashed the RDP connection for whatever reason. If you encounter problems you can list sinks and sources with these commands:

$ pacmd list-sinks | grep -e 'name:' -e 'index'
 * index: 1
 name: <xrdp-sink>
$ pacmd list-sources | grep -e 'name:' -e 'index'
 index: 1
 name: <xrdp-sink.monitor>
 * index: 2
 name: <xrdp-source>

You can restart the xrdp server by restarting xrdp and xrdp-sesman.

Linux Client

At this point your Linux desktop can be connected using Windows Remote Desktop Client. Since none of the pre-packaged rdp clients did not work, had lag or did not support sound i decided to build latest FreeRDP. I personally decided latest nightly build offered from their CI at https://ci.freerdp.com/job/freerdp-nightly-binaries/.

To open a connection I enter:

/opt/freerdp-nightly/bin/xfreerdp /kbd:0x0000041D /sound /v:[MY SERVER NAME]

The /kbd value uses Swedish keyboard layout. You can get the possible values by entering:

/opt/freerdp-nightly/bin/xfreerdp /kbd-list

You don’t need to edit the keyboard settings in the target machine. Actually editing them made FreeRDP to segfault.

Epilogue: Was it Worth it?

The original goal was to use the remote desktop as a workstation for daily use: browsing the Net, reading emails and doing some garage programming. Was this setup good enough for this?

No.

There is slight lag with the display: when you close a window it can take 1-2 seconds before the screen updates. Changing the virtual desktop is sluggish. The mouse roller is inaccurate. I used this for one or two days and then destroyed the whole thing.

Acknowledgements

Configuring Chinese Ethernet-controllable 2-relay board

I bought some weeks ago a Ethernet-controllable 2-relay board. While my Chinese Top Seller could not provide any documentation for the item I had to find things myself.

The default IP for the device is 192.168.1.100. To control the device with my Linux device I found a nice Python script sr-201-relay. Since we know the IP of the board we can get rest of the configuration:

$ python sr-201-relay.py 192.168.1.100 config
ip=192.168.1.100
netmask=255.255.255.0
gateway=192.168.1.1
(unknown)=
power_persist=0
version=931
serial=XXXXX50E35000000
dns=192.168.1.1
cloud_server=connect.tutuuu.com
cloud_enabled=0
cloud_password=(not-sent)

Excellent! Now we can change the network settings to suit with my LAN:

$ python sr-201-relay.py 192.168.1.100 gateway=0.0.0.0
$ python sr-201-relay.py 192.168.1.100 dns=0.0.0.0
$ python sr-201-relay.py 192.168.1.100 ip=192.168.2.11
$ python sr-201-relay.py 192.168.1.100 reset

First I tried to make sure the device does not reach Internet and finally I set a static IP from the correct LAN.

The script offers methods to turn relays on (closed) and off (open):

$ python sr-201-relay.py 192.168.2.11 close:1
$ python sr-201-relay.py 192.168.2.11 status
relay status: 1-closed 2-open 3-open 4-open 5-open 6-open 7-open 8-open
$ python sr-201-relay.py 192.168.2.11 open:1
$ python sr-201-relay.py 192.168.2.11 status
relay status: 1-open 2-open 3-open 4-open 5-open 6-open 7-open 8-open

 

HST-kortin käyttöönotto: Ubuntu Trusty

Tässä ohjeessa kerrotaan miten HST kortti otetaan käyttöön niin, että sillä voi kirjautua Firefoxilla viranomaisten sähköisiin verkkopalveluihin.

  1. sudo apt-get install opensc
  2. Käynnistä Firefox (esimerkissä 40.0.3)
  3. Edit > Preferences > Advanced > Certificates > Security Devices > Load > Module Filename
    • 32-bittisessä anna: /usr/lib/i386-linux-gnu/opensc-pkcs11.so
    • 64-bittisessä anna: /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so
  4. OK

Lähde: http://www.linux.fi/wiki/HST

Paketista asennettuna opensc kysyy kirjautumisen yhteydessä molemmat PIN-koodit. OpenSC sisältää kirjaston onepin-opensc-pkcs11, joka kysyy vain PIN1:n. Tämän saa vain kääntämällä OpenSC:n uudelleen. Onneksi homma on helppo ja ohjeet ovat hyvät. OpenSC on asennettava ohjeiden mukaisesti /usr -haaraan. /usr/local ei toimi.

 

Merikartat ja peruskartat Androidiin (offline)

Näillä ohjeilla voit tehdä Android-tabletille (tai puhelimelle) meri- ja peruskartat, joiden käyttö ei vaadi verkkoyhteyttä. Valitsemasi kartat ladataan maksuttomista karttapalveluista tietokoneella ja siirretään tablettiin. Voit siis ladata vain ne alueet, joita oikeasti tulet käyttämään. Jos et halua ladata karttoja etukäteen, katso toinen ohje.

Karttaselaimena käytetään erinomaista OruxMaps-ohjelmistoa, joka on maksullinen.

Liikenneviraston karttapalvelusta ladattavat merikartat eivät sisällä saarien nimiä tai veneväyliä, mutta alla oleva asetustiedosto lataa ne Maanmittauslaitoksen rajapinnasta.

mobac_sample_seamap

Jos resepti tuntii työläältä, kokeile vaikka Maastokarttoja.

A) Valmistelevat toimenpiteet

Kartat ladataan maksuttomalla MOBAC-ohjelmalla. Se on kirjoitettu karttatiedostojen lataukseen ja pakkaukseen eri ohjelmille.

  1. MOBAC tarvitsee Javan.
    • Windows: Lataa Java. Se on maksuton.
    • Linux: Tarvitset Javan. OpenJDK käy, esim. Ubuntussa asenna paketti “openjdk-7-jre”.
  2. Lataa MOBAC. Tuloksena on ZIP-tiedosto.
    • Windows: Pura ZIP-tiedosto esim. työpöydälle.
    • Linux: Pura ZIP-tiedosto esim. kotihakemistoosi.
  3. Tarvitset vielä meri- ja peruskarttojen määrittelytiedostot. Löydät ne tästä tiedostosta. Pura tiedostot MOBAC-ohjelman hakemistoon “mapsources”:
    mobac_mapsources1 mobac_mapsources2

B) Valitse ja lataa kartat

MOBAC-ohjelman toimintaperiaate on yksinkertainen. Sen avulla voit tehdä atlaksia (karttakokoelmia), jotka sisältävät yhden tai useampia suorakaiteen muotoisia karttoja.

  1. Käynnistä MOBAC:
    • Windows: Kaksoisnapsauta MOBAC-ohjelman hakemistosta löytyvää tiedostoa “Mobile Atlas Creator.exe”
    • Linux: Aja “start.sh”, esim. terminaalissa “sh start.sh”.
  2. Anna ensimmäiselle atlakselle jokin kuvaava nimi tai esim. “Testi”. Valitse formaatiksi “OruxMaps Sqlite”.
    mobac_name_atlas
  3. Oletuksena sinulle näytetään karttaa “OpenStreetMap MapQuest” (ks. vasemman ylänurkan Map Source -valinta). Siirry tämän kartan avulla haluamallesi alueelle (esim. Suomenlahdelle).
    • Liikuta karttaa pitämällä hiiren kakkosnappia alhaalla.
    • Zoomaa hiiren rullalla tai näytön yläreunassa näkyvällä zoom-valitsijalla.
      mobac_goto_finland
  4. Kun olet oikealla alueella ja zoom levelillä 10, valitse Map Source -valinnalla “Merikartta Liikennevirasto S57”.
  5. Rajaa haluamasi alueet (hiiren vasen näppäin), valitse zoom levelit ja valinta atlakseen (valinnan jälkeen vasemmalta Add selection).
    • Valitut alueet (selection) vastaavat ikään kuin karttalehtiä, kun taas atlakset ovat kartastoja.
    • Tabletin ohjelma osaa valita automaattisesti oikean karttalehden.
    • Mitä isomman zoom levelin (vasen ylänurkka, 0-18) otat mukaan, sen enemmän tilaa kartta-aineisto vie (ja kartan tekeminen kestää). Kannattaa kokeilla jättää levelit 15-18 pois, niin säästyy tabletilta levytilaa.
      mobac_select_layer
  6. Kun olet tyytyväinen atlakseen napsauta “Create atlas”. Se lataa valitsemasi kartat . Tämä kestää kauan, koska karttatiilet (kartan palaset) haetaan yksitellen palvelimelta. Mitä isompia zoom leveleitä olet valinnut selectioneihin, sen enemmän ladattavaa on.
  7. Valmis tavara tulee mobac-ohjelman hakemistoon “atlases”, jossa on alihakemisto kaikille tehdyille atlaksille.

C) Kartat Androidiin

Tässä vaiheessa olet tehnyt tietokoneella haluamasi kartat. Viimeisessä vaiheessa näytät kartat tabletissa.

  1. Lataa Androidille kartat näyttävä ohjelma OruxMaps.
  2. Käynnistä OruxMaps, jotta se tekee karttahakemistot tabletille. Voit sulkea OruxMapsin saman tien.
  3. Seuraavaksi voit siirtää valmiit kartat MOBAC-ohjelman hakemistosta “atlases” tabletille.
    • Näet tabletin hakemiston seuraavasti: Asetukset (kolme pistettä näytön ylänurkassa) > Kaikki asetukset > Kartat > Kartat hakemisto: esim. “/storage/emulated/0/oruxmaps/mapfiles/”.
    • Voit tehdä siirron esim. muistikortin tai Google Driven avulla. Kätevä apuväline tässä on Cheetah Mobilen File Manager.
  4. OruxMapsissa karttavalikko (karttaikoni) > Switch map > Offline > oma Atlaksesi jokin layer. Jos karttasi ei näy, paina yläreunan reload-painiketta.

D) OruxMapsin käyttö

OruxMapsissa on tärkeä säätää karttojen zoom-tasot oman päätelaitteen näytölle sopivaksi. Zoom-tasot näkyvät näytön oikeassa alanurkassa:

oruxmaps_zoomlevel

Zoom level on 16, joka on suurennettu digitaalisella zoomilla 136%:iin. Tässä näkyvät zoom levelit ovat samoja, jotka näkyvät MOBAC-karttoja tehdessä.

  • Zoom leveliä voi vaihtaa nipistämällä karttaa kahdella sormella.
  • Digital zoomia voi säätää äänenvoimakkuusnäppäimillä.

Pieni digital zoom auttaa, jotta karttamerkit näkyy sujuvasti. Zoom-asetuksia voi säätää: Asetukset (kolme pistettä näytön ylänurkassa) > Kaikki asetukset > Kartat > Zoom asetukset:

  • Pyöritys ele: Ota tämä pois päältä, jos et halua kääntää karttoja.
  • Äänenv. näppäimet: Valitse tämä, niin voit säätää digitaalisia zoomeja äänenvoimakkuuksilla.
  • Viimeistele nipistys zoom: Valitse tämä, niin zoom level vaihtuu loogisesti.

 

Reinstalling GRUB After Windows Update (non-EFI)

This post explains how to revert the GRUB after your Linux (Ubuntu 14.10)/Windows (7) dual-boot workstation boots directly to Windows the Windows update. This procedure is valid for non-EFI (legacy) workstations. For EFI workstations see another post.

  1. Get a sysresccd boot image and boot the workstation from it.
    • Note that you cannot make an bootable USB stick using dd but executing an installation script as explained in the documentation.
    • This recipe was tested on 4.5.3.
  2. Find your Linux root partiton using fdisk:
    1. fdisk /dev/sda (sdb, sdc…)
    2. p (prints the partiton table)
  3. In this example we found the suspected root partition from /dev/sdb2. It has the boot flag on and the partition type is “Linux”.
  4. mkdir /tmp/root
  5. mount /dev/sdb2 /tmp/root
  6. ls /tmp/root (make sure that this contains the root partition)
  7. grub2-install –root-directory=/mnt /dev/sda
  8. reboot
  9. As soon as you get your Ubuntu up and running re-instal grub:
  10. sudo grub-install /dev/sda

kernel-qemu for running RaspBi on QEMU

Some days ago I was in desperate need to run Raspberry Pi on my Ubuntu. As you might expect, there has been several others with similar need and i quickly ran to a number of blog posts explaining how to do this with QEMU. Maybe the best I could find is at http://paulscott.co.za/blog/full-raspberry-pi-raspbian-emulation-with-qemu/.

However, all these documents had a similar drawback. For some reason you cannot just make QEMU to boot from the Raspbian image file but you need a kernel file (often referred as kernel-qemu) which is given with -kernel parameter to QEMU. All these blogs pointed at currently no-existing blog at xecdesign.com and did not explain where this magical kernel file came from.

It appears that you have to do apply a number of patches (armhf? qemu?) to stock kernel to make it boot in QEMU. Of yourse, you have to cross-compile the kernel to ARM.

Finally I found the kernel-qemu inside the pre-packaged QEMU+Raspbian Windows installation at sourceforge. If you just need the kernel-qemu, go ahead and download it. This is probably the one that origins from xecdesign.com. There is another alternative (http://elinux.org/File:ZImage.7z) which gives you 3.6.1 kernel (download).

However, both the two have same drawback. You don’t get modules and they lack some important modules. For example the USB stack does not work.

Block YouTube with iptables

My son uses connects the Internet via our local WLAN network. To help him to just lag on the sofa and browse YouTube for hours we decided to block the video service from his mobile. It appears to be quite easy to block video services using squid but a bit harder to find the streaming server IPs. I have now found a solution which appears to work.

First we want to redirect the pass-through packages from the son’s mobile to the chain “funban”.

$IPTABLES -N funban
$IPTABLES -A FORWARD -m mac --mac-source 11:22:33:aa:bb:cc -j funban

Now we just have to build necessary rules to the chain.

#!/bin/bash
IPTABLES=/sbin/iptables
function block_ips {
 for THIS_IP in $1
 do
 # XXX.XXX.XXX.XXX/XX
 if [[ $THIS_IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\/[0-9]+$ ]]; then
 $IPTABLES -A funban -d $THIS_IP -j DROP
 fi
 # XXX.XXX.XXX.XXX
 if [[ $THIS_IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
 $IPTABLES -A funban -d $THIS_IP -j DROP
 fi
 done
}

function accept_ips {
 for THIS_IP in $1
 do
 # XXX.XXX.XXX.XXX/XX
 if [[ $THIS_IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+\/[0-9]+$ ]]; then
 $IPTABLES -A funban -d $THIS_IP -j ACCEPT
 fi
 # XXX.XXX.XXX.XXX
 if [[ $THIS_IP =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
 $IPTABLES -A funban -d $THIS_IP -j ACCEPT
 fi
 done
}

function get_ips {
 echo "`dig ${1} A | grep -E '^[^;]' | grep -o -E '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'`"
}

# Flush funban chain
$IPTABLES -F funban

# Block all Google-related IPs. The "AS15169" is taken from
# http://networktools.nl/asinfo/google.com
block_ips "`whois -h whois.radb.net -- '-i origin AS15169' | grep ^route\:`"
# Add these IPs to make google search to work (NOTE: This is not sufficient and blocks Google searches)
accept_ips `get_ips www.google.com`

My firewall server runs this script hourly to make sure the changes in the IPs don’t open unexpected possibilities to our YouTube addict. The blockade will surely affect other Google services as well. We possibly have to add more accept_ips commands to the script.