[Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www

[Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - Printable Version

+- WeatherDuino Forum (https://www.meteocercal.info/forum)
+-- Forum: Weather Stations - WeatherDuino Pro2 Line (https://www.meteocercal.info/forum/Forum-Weather-Stations-WeatherDuino-Pro2-Line)
+--- Forum: General Talking (https://www.meteocercal.info/forum/Forum-General-Talking)
+--- Thread: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www (/Thread-Pro2-Plus-RX-Integrating-RESTful-Sensor-Data-working-solution-for-local-wifi-www)

Pages: 1 2 3 4

[Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - t3gathome - 04-06-2018

Hi Werk_AG and hello world,

this is my first thread&post at this forum at the same time.
I am a friend of engolling and I heared about this awesome project because of him.
Since my bachelors thesis was about sth. like a weather station I wanted one to have in my flat instead of moving my personal one, preinstalled at my parents house.
And my decision fell on weatherduino Smile

.
In my flat now I already have a TX, a Pro2 Plus RX and a WD board. (But I need some sensors...)
This is a really nice project with so much working out of the box Werk_A. Kudos!
BUT I missed the feature of implementing more sensors.
I already got the main impression, that backward compatibility is a big point (and I understand that very well) in changing/adding things.
So I tried to improve - not to change Smile

.

This is a possibility of including RESTful sensor data (over wifi) in the Pro2 Plus RX board.

In my case, I was sad not to have the local SHT31 data of my wireless display transmitted to the main station.
For the display there is only a one way communication in the meaning of 433, so I thought I need sth. else.

So I created a Webserver on the wireless display that provides the data of temp/hum in JSON format. (shure, can be adjustet to whatever data we want)
And I implemented a web client on the Pro2 Plus RX board that takes the sensor JSON data (shure, can be adjustet to whatever data we want) of the wireless display (or any other IOT JSON formatted sensor) via http get
right now the routine, that takes the data (in the RX station), is in routines and is a new case of sensor: #if (INSIDE_TH_SENSOR == 8) // T/H over Wifi
to achieve a more stable connection and usability, I also changed the hostname of the WD automatically to the string "WirelessDisplay-"+relay_ID (than it is unique and clearly detectable in the network)

If you are interested in my solution, than I can share my already prepared manual Smile

.

In addition this solution also works over the web - not only the intranet/local wifi. (If the ip/the hostname is static and the software-predefinded port is open)
engolling and me successfully tested that tonight Smile

. --> So I successfully received his WD sensor data over the web on my RX (main station) board!

regards,
t3g

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - werk_ag - 05-06-2018

Hello t3gathome,

Welcome to the forum and thank you for your collaboration.

Of course I'm interested in your developments/enhancements! I see some applications for them.
I don't know if they will have any implications on the overall operation of the system, but there are only one way to know, which is doing tests.
If you would like to share the code, when possible I will give a look to it.
I may have some suggestions/ideas: It could be interesting trying to make that the T/H sensor of the Wireless Displays could be mapped as an extra sensor. After seeing the code I will try to figure a way of doing it.

Quote:...to achieve a more stable connection and usability, I also changed the hostname of the WD automatically to the string "WirelessDisplay-"+relay_ID (than it is unique and clearly detectable

Keep in mind that if more than one Wireless Display is used, all of them will use the same relay_ID, so this may no work when multiple Wireless Display units are used at the same time. Anyway, that isn't a problem, as can be solved by many ways.

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - t3gathome - 05-06-2018

Hi Werk_AG,

nice to hear that.
I will try to add the changes in my post.
Probably the line numbers may be incorrect, so I added some example lines.
And if you would paste some parts/functions somewhere else --> feel free Big Grin

.
I am new to this environment, and I am not so deep into it yet...

This is the RX part:

Code:
Install ArduinoJson library from Benoit Blanchon

Main file:

Between lines 67&69:

#include <ESP8266.h>

#include "VPcrc.h"

Add:

#include <ArduinoJson.h>

Between lines 531&539:

    #endif 

    // Check latest software version

Add:

    if (wifi.enableMUX()) {

        Serial.print("multiple ok\r\n");

    } else {

        Serial.print("multiple err\r\n");

    }

Config_Options.h:

After:

// --------------------------------------------------------------------------------------

//   Define the type of your inside Temperature / Humidity Sensor

// --------------------------------------------------------------------------------------

Replace with:

#define INSIDE_TH_SENSOR   8   // 0= BME280, 1= HTU21D, 2= SHT21, 3= SHT31, 8=T/H over Wifi

Routines.ino:

After line 378:

Add:

  #if (INSIDE_TH_SENSOR == 8)  // T/H over Wifi

    #if (ENABLE_INTERNET == 1)

      uint8_t buffer[1024] = {0};

      static uint8_t mux_id = 3;

      //Change HOSTNAME String if differs: (wifi.createTCP(&mux_id,HOST_NAME, HOST_PORT,0)) {

      if (wifi.createTCP(mux_id, "WirelessDisplay-"+String(Relay_ID), 80, 0)) {

        //Serial.print("create tcp ok\r\n");

      } else {

        Serial.print("create tcp err\r\n");

      }

      //Change HOSTNAME if differs: s_Relay_ID String

      String s_Relay_ID = String(Relay_ID);

      char *prefix = "GET / HTTP/1.1\r\nHost: WirelessDisplay-";

      char *postfix =" host:80\r\nConnection: close\r\n\r\n"; 

      String url = prefix +s_Relay_ID+ postfix;

      const char *get_json = url.c_str();

      //Some Examples instead of const char *get_json = url.c_str() above: Attention: IP/Hostname must be the same as in wifi.createTCP

      //char *get_json = "GET / HTTP/1.1\r\nHost: 192.168.1.140 host:80\r\nConnection: close\r\n\r\n";

      //char *get_json = "GET / HTTP/1.1\r\nHost: ESP-17F41F host:80\r\nConnection: close\r\n\r\n";

      //char *get_json = "GET / HTTP/1.1\r\nHost: 93.229.107.176 host:80\r\nConnection: close\r\n\r\n";

      wifi.send(mux_id,(const uint8_t*)get_json, strlen(get_json));

      //first receive for header

      uint32_t len = wifi.recv(&mux_id,buffer, sizeof(buffer), 1000);

      //second receive for payload

      len = wifi.recv(&mux_id,buffer, sizeof(buffer), 1000);

      if (len > 0) {

        StaticJsonBuffer<300> jsonBuffer;

        JsonObject& root = jsonBuffer.parseObject(buffer);

        if (!root.success())

        {

          Serial.print("ParseObject() failed");

          return;

        }

        TemperaturaInt = (int16_t)(root["InsideT"]); 

        Output_H_Int   = (int16_t)(root["InsideH"]);  

      }

      if (wifi.releaseTCP(mux_id)) {

        //Serial.print("release tcp ok\r\n");

      } else {

        Serial.print("release tcp err\r\n");

      }

    #endif

  #endif

and this is the WD part:

Code:
Install ArduinoJson library from Benoit Blanchon

Main file:

Between lines 43&44:

#include <ESP8266WiFi.h>

#include <ESP8266mDNS.h>

Add:

#include <ESP8266WebServer.h>

#include <ArduinoJson.h>

Between lines 172&173:

  WGForecast forecasts[maxForecasts]; 

#endif  

Add:

  //---------------------------------------------------------

  //     Start WebServer

  //---------------------------------------------------------

  ESP8266WebServer server(80);    // set Server Port

  void set_data() {

      StaticJsonBuffer<200> jsonBuffer;

      JsonObject& jsonObj = jsonBuffer.createObject();

      char JSONmessageBuffer[200];

      if (Display.InsideT == 0)

          server.send(204);

      else {

          jsonObj["InsideT"] = Display.InsideT; //inside T for example

          jsonObj["InsideH"] = Display.InsideH; //inside H for example

          jsonObj.prettyPrintTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));

          #ifdef PRINT_DEBUG_WIFI

            Serial.println(JSONmessageBuffer);

          #endif

          server.send(200, "application/json", JSONmessageBuffer);

      }

  }

Between lines 275&277:

  connectWifi();

  if (WiFi_connected)

Add:

  //  Handle events

  server.on("/", HTTP_GET, set_data);

  server.begin();               // Start the webserver

Between lines 359&361:

  if (ask433.available()) ReceiveDataRF();

  // --- Things inside this, run once per minute

Add:

server.handleClient();

Routines:

Between lines 71&72:

  delay(100);

  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);

Add:

WiFi.hostname("WirelessDisplay-"+String(Relay_ID));

So I always tried to show the important lines, in between there should be added sth.
e.g. WiFi.hostname MUST be before WiFi.begin...

If you have any questions - I will help you out and could explain every loc.

I hope it will help to enhance this great project a bit Smile

.

You can test if the WD provides data by typing in the IP or the hostname in your browser and it should write sth. like:
{
"InsideT": 248,
"InsideH": 604
}
So these are the internal values (multiplied by 10) --> in JSON double values could also easily be transmitted... but I would stay at your data format!

regards,
t3gathome

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - t3gathome - 05-06-2018

Oh I did not know, thath many WDs can have the same relay ID, or that it is dependent on the RX... As I only have one of each...
Yes I wanted to simplify it and shorten the configs (or the need to)...
If this won't work, then we can easily change it - or can go e.g. to the name we give for OTA updates hostnames ?!?

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - werk_ag - 05-06-2018

Thank you for the code. It looks to me quite understandable.
During the next weekend I will try to do something with this code. Interesting idea! Smile

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - werk_ag - 05-06-2018

(05-06-2018, 22:36)t3gathome Wrote: Oh I did not know, thath many WDs can have the same relay ID, or that it is dependent on the RX... As I only have one of each...
Yes I wanted to simplify it and shorten the configs (or the need to)...
If this won't work, then we can easily change it - or can go e.g. to the name we give for OTA updates hostnames ?!?

Using the OTA name may be one of the possibilities... then the client will try to contact every pre-defined (in config perhaps????) WD units, and retrieve the T/H for each one. I'm just theorizing. First I must try to figure if I can find a way to map the readings to extra sensors without having to write a lot of extra code (memory constrains on the receiver)

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - t3gathome - 06-06-2018

(05-06-2018, 22:41)Werk_AG Wrote: Thank you for the code. It looks to me quite understandable.
During the next weekend I will try to do something with this code. Interesting idea!

Cool, I am looking forward to it Big Grin

And concerning to the WDs and data of them in general, there are many things possible. Because of extra sensors (the data memory/layer) and so on, I am happy that you have a look at it now directly... I did not want to break any memory constraints!

I was very interested in the humidity data of the Display, because I have it in my bath... That's where the idea was born.
So I can live with any solution which could include the data of the Display in the whole weatherduino environment, that fits best according to your experience Wink

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - werk_ag - 07-06-2018

(06-06-2018, 20:34)t3gathome Wrote: ...
I was very interested in the humidity data of the Display, because I have it in my bath... That's where the idea was born.
...

The bathroom always has been a place where many ideas was born. Smile

(06-06-2018, 20:34)t3gathome Wrote: ...
So I can live with any solution which could include the data of the Display in the whole weatherduino environment, that fits best according to your experience
...

Instead of replacing the data of the local T/H sensor on the RX by the data of the local Wireless Display sensor, I'm thinking in allow to map those data to any of the seven Extra Sensors. User can choose which one by using the WeatherDuino mapping scheme.
Taking it even a step further, T/H data from up to four WD units can be mapped to any of the seven Extra Sensors. I'm sure you are seeing the possibilities... Cool

Note the new "Source 5"!

Code:
// --------------------------------------------------------------------------------------

//   Extra Sensors Mapping

// --------------------------------------------------------------------------------------

// ---- Define the source and sensor type / number for each of the seven extra sensors, insert 9 if not used.

//      Sensors SHT31, SHT21 or HTU21D are type 0, sensors SHT1x or DHT22 are type 1

// 

// ---- Source 0 means TX unit0, 1 means TX unit1... 

// ---- Source 5 is for Temperature/Humidity from the Wireless Display units

// ---- Source 6 (Reserved)

// ---- Source 7 is for BIOS Sensor (Only Temperature is shown)

// ---- Source 8 is for Temperature/Humidity from the Air Quality Monitor

//

// ---- First field = Sensor source, Second field = Sensor type (0 or 1 for TX units) or number

// ---- Example:

// ----                          ES0   ES1   ES2   ES3   ES4   ES5   ES6     

//#define EXTRASENSORS_SOURCE { {0,1},{1,1},{5,1},{5,0},{9,9},{9,9},{8,0} }

//               {0,1} - First extra sensor (ES0), will be sensor 1 from TX unit0

//               {1,1} - Second extra sensor (ES1), will be sensor 1 from TX unit1

//               {5,1} - Third extra sensor (ES2), will be T/H data from WiFi Sensor 1 (Wireless Display 1)

//               {5,0} - Fourth extra sensor (ES3), will be T/H data from WiFi Sensor 0 (Wireless Display 0)

//               {9,9} - Fifth extra sensor (ES4), not used - not mapped

//               {9,9} - Sixth extra sensor (ES5), not used - not mapped

//               {8,0} - Seventh extra sensor (ES6), will be Temperature / Humidity from the Air Quality Monitor 

//

//-----                        ES0   ES1   ES2   ES3   ES4   ES5   ES6     

#define EXTRASENSORS_SOURCE  { {0,1},{0,0},{5,0},{1,0},{9,9},{9,9},{8,0} }

#define EXTRASENSORS_SOURCE { {0,1},{0,0},{5,0},{1,0},{9,9},{9,9},{8,0} }

With this we will have T/H from Wireless Display 0 mapped to third Extra Sensor... can be to any other!

Maybe by the weekend I will have a test version. Would you like to test it?

Currently I'm fine tuning some things.
uint8_t buffer[1024] = {0}; This is too large, system will crash in 2 or 3 days or even less.
StaticJsonBuffer can also be reduced to 100 bytes (50 may be enough, have to check it better).

Execution time is also a crucial item in all WeatherDuino routines.

Code:
create tcp OK

204

637

Routine took: 174 mS

create tcp OK

204

637

Routine took: 264 mS

create tcp OK

204

637

Routine took: 136 mS

create tcp OK

204

637

Routine took: 346 mS

create tcp OK

203

638

Routine took: 85 mS

This is a new routine for the RX, where almost all of your code is included.

Code:
//---------------------------------------------------------

//     Get Temperature / Humidity from WiFI sensors (Usually the local T/H sensor on Wireless Display units)

//---------------------------------------------------------

#if (ENABLE_INTERNET == 1 && ENABLE_GET_TH_FROMWD == 1)

  void getTHfromWD()

    {

      uint8_t mux_id = 3;

      uint32_t uploadApiTimer = millis();       // Used to check how long it takes to execute

      for(byte i=0; i<WIFIsensors; i++)

        {

          if (wifi.createTCP(mux_id, WIFIunit_IPaddress [0] [i], 80, 0))      

            {

              Serial.print(F("create tcp OK\r\n"));

              const char* prefix = "GET / HTTP/1.1\r\nHost: ";             

              const char* postfix =" host:80\r\nConnection: close\r\n\r\n";

              String ipaddress = WIFIunit_IPaddress [0] [i];

              String url = prefix + ipaddress + postfix;

              wifi.send(mux_id,(const uint8_t*)url.c_str(), strlen(url.c_str()));

              //first receive for header

              memset(temp_buffer, 0, sizeof(temp_buffer));       

              uint16_t len = wifi.recv(&mux_id, temp_buffer, sizeof(temp_buffer), 1000);

              //second receive for payload 

              memset(temp_buffer, 0, sizeof(temp_buffer));

              len = wifi.recv(&mux_id, temp_buffer, sizeof(temp_buffer), 1000);

              if (len > 0)

                {

                 StaticJsonBuffer<100> jsonBuffer;

                 JsonObject& root = jsonBuffer.parseObject(temp_buffer);

                 if (root.success())

                  {

                    Serial.println( (int16_t)(root["T"]));

                    Serial.println( (int16_t)(root["H"]));

                    WiFi_THdata [i][0] = (int16_t)(root["T"]);

                    WiFi_THdata [i][1] = (int16_t)(root["H"]);                     

                  }         

               }                      

             }

          wifi.releaseTCP(mux_id);            

        }

      for (byte i=0; i<7; i++)

       {

         if (ExtraS_Source[i][0] == 5)

          {

            ExtraS_Data[i][0] = WiFi_THdata [ExtraS_Source[i][1]] [0];  // Temperature Data

            loopData.extraTemperatures [i] = (ExtraS_Data[i][0] * 0.18)  + 122.5;

            ExtraS_Data[i][1] = WiFi_THdata [ExtraS_Source[i][1]] [1];  // Humidity Data

            loopData.extraHumidities [i]   = (ExtraS_Data[i][1] + 5.0) * 0.1;

          }  

       }

      Serial.print(F("Routine took: ")); Serial.print((long)(millis() - uploadApiTimer));

      Serial.println(F(" mS")); Serial.println();            

    }

#endif

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - markkkk42 - 07-06-2018

(05-06-2018, 22:41)Werk_AG Wrote: Thank you for the code. It looks to me quite understandable.
During the next weekend I will try to do something with this code. Interesting idea!

Looks like you have been thinking in the bathroom too Werk Smile

So much for waiting until the weekend..seems like you are going at full speed

RE: [Pro2 Plus RX] Integrating RESTful Sensor Data - working solution for local wifi/www - werk_ag - 07-06-2018

(07-06-2018, 06:26)markkkk42 Wrote: Looks like you have been thinking in the bathroom too Werk So much for waiting until the weekend..seems like you are going at full speed

Just something I would like to explore, which may turn Wireless Display units even more useful devices.