InfluxData sponsored this post.
I managed to hook up solar panels at my house over a year ago, but soon realized I had a problem: I wanted more access to how certain metrics were displayed, such as how much power and energy the panels were able to generate from the sun. I was also interested in correlating this data with temperature, cloudiness and sun intensity.
As a way to retrieve these metrics from the solar panels, I found that the Fronius solar inverter, which transforms DC electricity coming from solar panels to grid-compatible AC, has a push service. This push service can be configured to periodically send HTTP POST or FTP requests with the data encoded in JSON to the provided endpoint — exactly what I needed.
I was also already using the InfluxDB database for storing other metrics from IoT devices and thought that it would be cool to just persist the inverter-produced data there. However, at the time Telegraf (the official data collector software for InfluxDB) didn’t support parsing JSON (and other data formats) data sent via HTTP POST/PUT requests. As these types of requests are the de-facto standard, I decided to contribute and create a http_listener_v2 plugin which would support this use case.
Essentially, the plugin registers HTTP POST/PUT requests. The data from the request is parsed in the specified format and then forwarded to InfluxDB or other supported databases.
In this blog post, I will explain how I now am able to use the http_listener_v2 Telegraf plugin I created to record and analyze the power my house’s solar panels generate. By following along, you will learn about the basic architecture of the InfluxDB’s TICK Stack and understand how to set up the plugin yourself.
Fronius Inverter Settings
The settings of the Fronius inverter can be reached via its WiFi access point. The section that interests me is the push service. This service periodically sends data (electricity power, energy and voltage) to specified endpoints via HTTP. Here is my configuration:
When saved, the interface says that a connection is not possible because Telegraf is not yet listening on the specified server port. I am going to fix this in the next paragraphs.
Setting Up the TICK Stack
The easiest way to get Telegraf up and running is to use the sandbox which spins up the whole TICK Stack (Telegraf, InfluxDB, Chronograf and Kapacitor) in Docker. All components of the TICK Stack can be started by calling ./sandbox up in the command line. However, before that I need to expose port :8080 for Telegraf in the docker-compose.yml file:
|
Configuring Telegraf
In the next step, we need to configure the Telegraf server to listen for the incoming JSON data sent by the inverter.
Here’s how I set up the http_listener_v2 plugin in the Telegraf’s config:
|
NOTE: if you’re using the sandbox, the config file should be located as telegraf/telegraf.conf (the file is in toml format). After making any changes to the config, don’t forget to restart the sandbox with ./sandbox restart.
All options for http_listener_v2 plugin can be found here.
In order to understand how the json_time_format gets interpreted, read more about the date and time parsing in this paragraph of the Go documentation.
Processing Input Data
After receiving the HTTP POST request, the http_listener_v2 plugin parses the data based on data_format option (all available formats can be found here). In my case, it’s the JSON parser. The parser flattens each JSON object into an InfluxQL query. The flattening is done by joining the JSON keys with an underscore ‘_’. For instance:
|
This JSON object is flattened to Body_P_PV=764.0, Body_P_Load=258.0.
To simplify the naming of the data fields, the processor.rename plugin can be used to drop the unnecessary prefixes in the measurement field names:
|
Inspecting Data Series with Chronograf
Finally, we can observe the collected time series data with Chronograf. By default, Chronograf should be on localhost:8888.
It’s easy to get an overview of all of the data in the Explore tab. There you can either click through the databases, measurements, and fields or write an InfluxQL query yourself. You can also choose Flux, which is a new programming language specifically designed for working with data series. If you would like to see your visualizations sometime again, you can create a dashboard in the Dashboards tab. Check the out-of-the-box examples for inspiration.
Below is the visualization of the data sent from my solar inverter. The green line represents generated energy, and yellow represents consumed. We can clearly see that there’s not much energy generated in the middle of winter.
Although creating data visualizations is quick and effortless in Chronograf, you can look at Grafana for a more advanced graphics solution. It works really well with InfluxDB.
Conclusion
To recap, I have created a new input plugin http_listener_v2 for Telegraf and demonstrated how it was used with the Push Service of Fronius solar inverter. The JSON data from the inverter was parsed, processed and then visualized in Chronograf and Grafana.
The plugin can be used in other cases as well. It can be configured to save non-numeric values as tags or parse other data formats such as CSV.
Feature image via Pixabay.
InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
