Implementing Real-Time data pipelines on the Cloud. DIY Projects from data generation, acquisition, storage up to visualization. In this section, you walk through examples of a new transformative, more collaborative way of doing data science on the Cloud.
Fig. 1: Data diagrams on Google Data Studio
This is the third tutorial from the series "implementing real-time data pipelines - from generation to models" and the other tutorials are the following:
Two weeks ago, I published a tutorial that explains how to connect an M5Stack running MicroPython to the Google Cloud Platform using the IoT Core, and I did mention that upcoming tutorials will examine the following topics:
- Collecting and synchronizing external data (weather from OpenWeatherMap) and other sensors -window/door status, sneezing detector-.
- Saving the data to a NoSQL database
- Displaying the obtained data on Google Data Studio.
This tutorial is about managing a Docker Engine remotely using Portainer connected to the protected Docker daemon socket (
TCP port 2376). By default, you can manage Docker locally through a non-networked UNIX socket (option
-v /var/run/docker.sock:/var/run/docker.sockwhile running Portainer). But, if you want the Docker Engine to be reachable through the network in a safe manner, you need to enable TLS by specifying the
--tlsverifyflag and pointing Docker’s
--tlscacertflag to a CA certificate. Then, the daemon only accepts connections from clients that are authenticated by a certificate si...
Cloud Platforms | 7 min | 5646
The last days, I was looking for an online project management tool with bug tracking system. The best platform that I know is Jira from Atlassian. Jira offers a bug tracking and agile project management, however, it costs between $8.33 and $10 (annual/monthly payment) for the standard version with under 10 accounts. We use Jira in Finealyze and I am very happy with it, but I was looking for a free alternative for myself. Then, I found OpenProject, which describes itself as "the leading open source project management software".
OpenProject is offered in three different ways: Community, Cloud an...
This part 2 of the tutorial will allow you to get data from the smart power outlets and send it to the Google Cloud Platform (GCP) throught the Google IoT Core and using an ESP32 programmed with Zerynth (Python). This data is not usually available to the user (directly, you need to use the power outlet application), and it is sent to the company servers.
If you want to check, if your power outlets are compatible for this tutorial, they should have the...
There are lot of tutorials about connecting the ESP32 to a cloud service, but I planned to connect these boards using MicroPython directly to the cloud. That means, I didn't want to use a hardware or software bridge. I wanted to use the Google IoT Core, but MicroPython doesn't support JSON Web Token (JWT), which is needed for the device authentication. I tried to write the library myself, but it didn't work, because of the needed dependencies and the small mem...