#Home Assistant: DIY Smart Home - tips and tricks (SSL, HAProxy & Let's Encrypt)
Home Automation, Raspberry Pi | 12 min | 130956
Table of contentShowIn the last years, I wrote several articles about Home Assistant (HA) and the possibility to replace the Zigbee hubs from different companies in your home using Zigbee2MQTT. Some of those articles are listed here (don't visit the links, they are outdated):
- Zigbee: Xiaomi Sensors using Raspberry Pi (without Gateway!)
- Zigbee: Flashing a CC2531 dongle using a Raspberry Pi
- Raspberry Pi: Xiaomi Sensors & Home Assistant (Hass.io)
- Raspberry Pi: Controlling Philips HUE Lights (ZigBee & CC2531)
- Zigbee: Reading Xiaomi Sensors with a Raspberry Pi (no Xiaomi gateway needed!)
Those articles were very popular at that time but, they need to be updated!
I'm using HA for about 3 years now and in this article:
- I'll describe my setup and learnings that I got during this time;
- I'll include the steps to add an SSL layer to the HA setup.
A Dockerized service adds the SSL layer (reverse proxy) and automatically renews the SSL certificate when needed. This setup enables you to seamlessly connect your HA to Google Assistant without using any external server or service in the middle.
HA is officially compatible with the following hardware:
generic-x86-64 odroid-c2 odroid-n2 odroid-xu qemuarm qemuarm-64 qemux86 qemux86-64 raspberrypi raspberrypi2 raspberrypi3 raspberrypi4 raspberrypi3-64 raspberrypi4-64 tinker khadas-vim3
and more options (e.g. orange-pi, rock-pi, and more.). I'm using a Raspberry Pi 4 for my setup, so the steps below will be focus on this architecture, but they can be easily ported to other architectures.
HA installation
If you are newbie with HA, you should know that you can install HA in four different ways. I prefer the Supervised way. But, if you don't have enough experience in Linux and Docker, you can go straight ahead with the OS way - meaning hass.io. Why do I prefer the Supervised? Well, because I still have access to Raspberry Pi OS and I can install or start other things or services (e.g. HAProxy - continue reading). This means, I lose the possible advantages of a managed OS but I gain flexibility.
To install HA using the Supervised way, you need to get Raspberry Pi OS (as I said, I focus on the Raspberry Pi architecture) running and you need to install Docker. This can be done after connecting e.g. over SSH to the Raspberry Pi and by typing the following on a Terminal:
curl -sSL https://get.docker.com | sh sudo usermod -aG docker pi sudo reboot
After installing Docker, you should install the following requirements:
sudo apt-get install -y jq wget curl udisks2 libglib2.0-bin network-manager dbus libpcap0.8 libdbus-glib-1-dev apparmor ppp
and the latest OS-Agent following the steps described here: OS Agent Installation.
The next step is to install the HA supervisor by typing:
wget https://github.com/home-assistant/supervised-installer/releases/latest/download/homeassistant-supervised.deb dpkg -i homeassistant-supervised.deb
If you are running Raspberry Pi OS with Kernel >= 5.10 (type
uname -a
to check that info). Add the following at the end of the line inside the/boot/cmdline.txt
file and reboot the Raspberry Pi.lsm=apparmor
this activates AppArmor support of the Kernel.
Then, you will have HA running on
http://[RASPBERRYPI-IP]:8123
. Thus, point your browser to that address and start the configuration.Note: If you connect your Raspberry Pi to your router via Wi-Fi, due to thenetwork-manager
package, the MAC address of your Wi-Fi adapter is going to change on every reboot -I don't know why-. This causes the IP to change as well, if are using DHCP, which is the default option of the interface. I didn't find any solution to that problem (obviously a fix IP would help), so I connected the Raspberry Pi using a LAN cable to the router, and reserved the IP to the MAC address of the LAN network adapter of the Raspberry Pi. Update @Fotios thanks for the comment. The problem is related with the MAC address randomization of the network-manager package. Describe for Fedora in this article.Zigbee2MQTT Hardware
A setup with a Raspberry Pi and a Zigbee dongle or a Home Assistant Yellow can replace multiple Zigbee hubs/gateways that you may have running on your home. Xiaomi, Bosch, Philips Hue, IKEA, Osram, Sonoff, and others can be replaced with this setup. More important is that you can combine the devices (sensors and actors) from different companies using automation rules, scenes, scripts, and more. You save electricity because you only need a Gateway (Raspberry Pi) and you don't have to buy everything from the same company, which can be sometimes very expensive. These Zigbee devices are compatible with this setup.
For the setup, I used to use the cheap
CC2531
Zigbee dongle as I mentioned here. You need to flash the chip using a Raspberry Pi or a CC-Debugger. However, the dongle is no longer recommended because of the outdated chip and the performance. Last year, SONOFF has presented the SONOFF Zigbee 3.0 USB Dongle Plus. This dongle uses a CC2652P, which is pre-flashed with the right firmware (as Zigbee coordinator), and it comes with an aluminum housing and an antenna, which increases the Zigbee range (gain up to 5dB with default firmware). Thus, for my actual setup, I've chosen the SONOFF Zigbee 3.0 USB Dongle Plus.Note 1: FYI - Zigbee 3.0 devices can be added to an Echo Plus, Echo Show 2, Echo Studio, Echo 4G. It's a different setup as with HA, but not a lot of people know that those Amazon Echos have a Zigbee hub integrated.
Note 2: If you want to update the firmware of the SONOFF Zigbee 3.0 USB Dongle Plus, follow the instructions provided in this tutorial. You should update the firmware if you have Philips HUE lights that don't have a reset button.
Note 3: WI-FI/WLAN devices are also compatible with HA. Sonoff/eWeLink or Tuya/SmartLife can be added to the setup using cloud integrations or as I will explain, local integrations.
Note 4: ZHA is another Integration that you can install to connect to Zigbee sensors. I prefer Zigbee2Mqtt because it is more flexible, and support more devices. But, ZHA is easier to set up without additional software (MQTT).Firmware Upgrade (optional)
As I said, the SONOFF Zigbee 3.0 USB Dongle Plus comes pre-flashed with the Z-Stack 3.x.0 coordinator firmware. But, if you need to use Touchlink for resetting devices this only works with firmware newer as Z-Stack_3.x.0_20220103. Thus, you need to upgrade the firmware otherwise you'll get the error:
SRSP - AF - interPanCtl after 6000ms
To do that, I used this tutorial and followed
Method 2
and theManual mode
option. That means I opened the housing and pressed the boot button while I connect the dongle in one of my Windows computers. Yes, you need a computer running Windows because you will use the SmartRF(TM) Flash Programmer 2 (v1.8.2) that runs only on Windows. The latest firmware is available here, and for the CC2652P you need the hex file inside theCC1352P2_CC2652P_launchpad_*.zip
file.
Fig. 1: Philips HUE LightStrip resetted using TouchlinkZigbee2MQTT Installation
To install Zigbee2MQTT on a running HA, you need to add the repository:
https://github.com/zigbee2mqtt/hassio-zigbee2mqtt
to the add-on repositories. This installation can be only made following these steps if you installed HA as Supervised installed or the OS/hass.io. These two options support add-ons. Thus, add the above repository URL in the ADD field located under:
Configuration
(menu on the left) →Add-ons, Backups & Supervisor
(option in the middle panel) →Add-on Store
(blue button on the right corner) →⋮
(top right corner) →Repositories
(dropdown menu) →- Paste the URL in the
ADD field
(overlay popup) and click on theADD
button.
The repository includes two add-ons:
- zigbee2mqtt is the stable release (recommended),
- zigbee2mqtt-edge is the dev branch, so you can install the edge version if there are features or fixes in the dev branch that are not yet released.
These two add-ons will be listed in the add-on list (
Configuration
→Add-ons, Backups & Supervisor
). You need to install only one of them by clicking on its box and selecting install. I installed the stable version, but if you're willing to be a beta tester, select the dev option.Note: The readme description, does not reflect the git commit comments (edge v1.22.0-3, stable v.1.22.2-1).Zigbee2MQTT needs also an MQTT-Broker (thus the name) that can be installed separately (as a service) or activated as a HA add-on. In the add-on list, you can find the
Mosquitto broker
. Click on the box and install it.You should activate the Advanced Mode, it gives you more information (logs) to debug, and more options to check if you are configuring something wrong (e.g. a check for theconfiguration.yaml
file).
Go tohttp://[HA-IP]:8123/profile
and turn on theAdvanced Mode
switch.
Activate also the Multi-factor Authentication Modules especially if you are going to open the HA to the Internet. To do that, turn on theAuthenticator app
switch.Add-on Configurations
Before you start the installed add-ons, you should configure them first.
MQTT Broker
To configure the MQTT add-on, go to:
Configuration
(menu on the left) →Add-ons, Backups & Supervisor
(option in the middle panel) →- Click on the
Mosquitto broker
(add-on in the middle panel) → - Click on the
Configuration
tab (top of the page)
and add the following inside the
Options
field:logins: - username: [user] password: [password] customize: active: false folder: mosquitto certfile: fullchain.pem keyfile: privkey.pem require_certificate: false mqtt: base_topic: zigbee2mqtt user: user password: password
Change the options
[user]
and[password]
to a secure combination and keep it safe, you'll need them for the next step. Then, click on theSave
button.After configuring the
Mosquitto broker
add-on, you can start it by going back to theInfo
tab and clicking on theStart
button.Zigbee2MQTT
To configure the MQTT addon, go to:
Configuration
(menu on the left) →Add-ons, Backups & Supervisor
(option in the middle) →- Click on the
Zigbee2mqtt
(add-on in the middle panel) → - Click on the
Configuration
tag (top of the page)
and change the following inside the
Options
field:[...] permit_join: true mqtt: base_topic: zigbee2mqtt user: [user] password: [password] serial: port: /dev/ttyUSB0 [...]
The user and password under
mqtt:
field you should have the same value that you've written for theMosquitto broker
add-on. Theport: /dev/ttyUSB0
config works for the SONOFF Zigbee 3.0 USB Dongle Plus, because it has a CP2102N. Other dongles connect using the/dev/ttyAMA0
port. Thepermit_join: true
config allows every Zigbee device to join the network if requested. After you add your devices (sensors and actors), you should set this option tofalse
. Finally, change thenetwork_key
to increase your network security. Do that before starting the add-on and adding some devices to your network. If you've already done that, you will need to pair all devices again if you change the network key :(.[...] network_key: - 1 - 1 - 1 - 1 - 1 - 1 [...]
Again, after configuring the Zigbee2MQTT add-on, you can start it by going back to the
Info
tab and clicking on theStart
button.You can usedmesg
command on Linux hosts to find the Zigbee dongle. If possible you should use the/dev/serial/by-id/
path of the stick, instead of/dev/tty*
. The path/dev/tty*
can change. The/dev/serial/by-id/
path doesn't change.The Zigbee2MQTT add-on has a nice and simple UI to add and configure Zigbee devices. To open the UI go to the
Info
tab and click on theOPEN WEB UI
. There, you check and update the Zigbee devices, look at the network, and more. There is a button/option (permit join (all)
) you don't need to press it to add new devices if you set thepermit_join: true
option. Then, after you added all devices and set this option tofalse
, you can add devices by pressing that button. A countdown appears, and you'll be able to add a new device during the countdown.Note 1: Every time you modify the configuration of a specific add-on (`Options` field), you need to restart so that your changes take effect.
Note 2: Devices that are connected to an AC supply (wall switches, curtain controllers - both with neutral) work as routers. Thus, these are good to have to extend and stabilize your network.Pair Devices to Zigbee2MQTT
You can pair devices to Zigbee2MQTT by pressing the pair button. It is different from device to device: how many times or if it's a long or a short press. The README booklet that comes with the device usually has that information. Check that, or simply press the button a couple of times. But, as I said, read the README booklet. Maybe you need to reset the device first (e.g. by Aqara sensors you can do that by pressing the reset button for 5 seconds).
Fig. 2: Example of a Zigbee NetworkTuya, Sonoff, and Xiaomi BLE devices
Using this setup, you can add devices from Tuya, Sonoff and Xiaomi BLE Sensors. I use the following integrations to do that:
- Tuya Integration - (devices controlled using Smartlife app on your smartphone): for WiFi devices, the integration uses the cloud and the update rate for the attributes (e.g. current, voltage, and power consumption) is not reliable. I used to used it, but not anymore.
- LocalTuya - (devices controlled using Smartlife app on your smartphone): this integration works local (via LAN) but, it is not officially supported. Thus, you need to install it using HACS and you need the local key of the devices. You can get those keys using Tuya IoT Cloud/API Explorer. A tutorial is available here. I use this integration and recommend it.
- Sonoff LAN - (devices controlled eWeLink app on your smartphone): this integration works local, cloud, or hybrid (configurable) but, it is not officially supported. Thus, you need to install it using HACS. The devices are synchronized from the Cloud and after that, you can switch to local. I use this integration and recommend it.
- Passive BLE Monitor - (sensors that use BLE and are integrated using Mi Home app): this integration works local (via BLE). To read the data from some sensors, you needs an encrytion key. Read the FAQ to get that key. Both methods (the Xiaomi Cloud Tokens Extractor and TelinkFlasher (WebBluetooth) worked for me. I use this integration and recommend it.
HAProxy for HTTPS
As you may have noticed, HA runs on port 8123 as an HTTP-based application. If you want to access your application from outside you should use HTTPS. HTTPS makes that all traffic between you and the HA is encrypted with a key that only the client (e.g. your smartphone's browser) and server (Raspberry Pi running HA) know.
To get an SSL certificate:
- Your HA should be exposed to the Internet. That means you need to forward the ports of the Raspberry Pi in your router.
- You need to point a domain name to your IP address, and for that, you'll need a dynamic DNS.
To do that easyly, I build a Docker image that you can deploy to add an SSL layer (TLS) to HA.
Pull the Docker Image
You can build the Docker image (check the files inside docker-stack) or you can pull it from the Dockerhub register:
docker pull lemariva/haproxy-certbot:1.1.1-SNAPSHOT
To start the service,
docker-compose
must be installed. Otherwise, you need to configure all the variables and settings that you can find inside thedocker-compose.yml
file.sudo apt-get install python3-pip pip3 install docker-compose
Configure the Docker Service
But, before starting the service, edit
[SERVICE_IP]
and[PORT_IP]
variables inside the filedocker-compose.yml
. Both should point to the service that you are forwarding. In my case, that is:SERVICE_IP: 192.168.178.161 SERVICE_PORT: 8123
The IP
192.168.178.161
and port8123
points to my Raspberry Pi address and the standard port of HA, respectively.Then, you can start the service by typing the following:
cd orchestration docker-compose up -d
Configure HA
If you are running HA with version greater than v2021.7.0, you will get a "400 Bad Request" error when I tried to access HA via the HTTP/HTTPS address. A breaking change was added to this version and if you are running a proxy you need to add to the configuration.yaml the following:
# proxy http: use_x_forwarded_for: true trusted_proxies: - [PROXY_IP]
If your proxy is running on another machine, you need to change
[PROXY_IP]
with the IP address of that machine. But, if you are running the proxy on the same machine that HA is running, you need to change[PROXY_IP]
with the Docker internal IP of the container. You can get that by typing the following:docker container inspect `docker ps -aqf "name=haproxy-certbot"` | grep "\"IPAddress\": \"1"
Note: this works if you didn't change the name to the container (LOAD_BALANCER_NAME
) inside the .env file. Otherwise, you need to change that.You get something like this:
pi@homeassistant:~ $ docker container inspect `docker ps -aqf "name=haproxy-certbot"` | grep "\"IPAddress\": \"1" "IPAddress": "172.26.0.2",
The
172.26.0.2
is the IP address that I needed.Get the first certificate
To get a certificate from Let's Encrypt, you need to forward the ports 80 and 443 of your computer to the Internet and you need a URL pointing to your router (basically a domain name). Internet providers usually change the IP address that you get every 24 hours. Thus, you need to get a dynamic DNS service. I use NoIP, it's free but you need to confirm your host every 30 days (no big deal). If you are thinking of buying a subscription, you can get a 5 dollars discount using the promo code
REFER5
, After opening the ports on your router and getting a domain name that points to your router, you can get the first certificate from Let's Encrypt. To do that, you need to get inside the Docker container using:docker container exec -it `docker ps -aqf "name=haproxy-certbot"` /bin/bash
and run the following commands:
/usr/bin/certbot certonly -c /usr/local/etc/letsencrypt/cli.ini --agree-tos --email [YOUR_EMAIL] --domains [YOUR_DOMAIN]
haproxy-refresh Replace
[YOUR_EMAIL]
and[YOUR_DOMAIN]
with your valid email address and the domain that points to your router. If everything goes as planned, you will get a valid SSL certificate for your HA system.Then, you can integrate your HA to e.g. Google Assistant following the steps in this tutorial.
Conclusions
This article updates all my articles about Home Assistant. The setup described in this article can replace every Zigbee hub that you already have in your home and can be integrated with your Google Assistant. Additionally, sensors and actors that connect to the Wi-Fi such as Tuya or Sonoff or even Xiaomi BLE devices can be also integrated and combined with the Zigbee devices in automation rules, scenes, and scripts.
The Google Assistant is possible due to a HAProxy configuration that adds an SSL layer to HA. In this article, I've described the steps to start a Docker container running HAProxy and acquire an SSL certificate from Let's Encrypt.
I also included tips and some learnings that I've acquired while I was configuring my setup during the last couple of years. I hope this helps you to set up your home, but remember Good home automation never annoys but is missed when it is not working (source).
We use cookies to improve our services. Read more about how we use cookies and how you can refuse them.
Steve 01.17.2022
I like your pi set up instructions. And for assist++ do you also show system or application gui's? For instance I use Google home and open it all day long to adjust devices. If that ends up command line I would wish to avoid that solution.
Fotios 01.18.2022
Hi, great article. I've been meaning to give HA a try and this was really informative.
FYI, the MAC address issue you have is called MAC address randomization and it's a security feature. It's described more in this fedora article along with some ways to configure it. Things might be slightly different in other OSes, but it's a starting point for you.
https://fedoramagazine.org/randomize-mac-address-nm/
Cheers!