[41south]

Well that got rid of the bad data - but ...... the information in the LOOP is almost totally fictitious and bears very little if no relationship to the data on the console TFT

Baro, UV, solar, temps are all wrong and compared to before the LOOP packet looks quite small?


LOOP:   2018-10-31 12:31:35 NZDT (1540942295) altimeter: 1053.60746351, appTemp: -2.13558531939, barometer: 1053.16342077, cloudbase: 384.852796518, dateTime: 1540942295, dewpoint: -2.81211985001, heatindex: 0.226223048565, humidex: 0.226223048565, inDewpoint: -0.512212601359, inHumidity: 29.996106225, inTemp: 17.2233038264, maxSolarRad: 950.382756578, outHumidity: 79.9970795266, outTemp: 0.226223048565, pressure: 1053.16342077, radiation: 981.148577966, rain: 0.0, rainRate: 0.0, usUnits: 16, UV: 13.7360800915, windchill: 0.226223048565, windDir: 359.98247716, windGust: 0.000940011594201, windGustDir: 359.98247716, windSpeed: 0.000783342995168

My process was to run the utility - this finished ok according to the serial monitor

Status register :10001100
Manufacturer ID :
1F
Device ID (part 1) :
22
Device ID (part 2)  :
0
Extended Device Information String Length  :
0
Erasing the Flash memory... Please wait!
...
Erase done!

Then reloaded B008

[41south]

Disregard all the above re the incorrect packets, the flash reset appears to have solved the problem

Thanks!!

Also now running compiled with 1.8.4

[werk_ag]

Enjoy.
I wrote the flash memory erase sketch yesterday, thinking in your problem. As it may be useful, will be included on next software releases.

[werk_ag]

When you say do a search on the forum do you mean this one or the Weewx forum?
...

I mean searching on this forum. Should be very old topics, I remember of an user (hvalentim I think) which has used weewx during long time.

[engolling]

Hi Werk,

since I'm also trying WeeWx I have corrupted data logs in the flash.
I tried your erasing sketch but WeeWx still gets old datasets from some depth of the RX

Could it be possible that not the whole memory is deleted?

Regards

[werk_ag]

Hi Werk,

since I'm also trying WeeWx I have corrupted data logs in the flash.
I tried your erasing sketch but WeeWx still gets old datasets from some depth of the RX

Could it be possible that not the whole memory is deleted?

Regards

Hi Engolling

Yes it is. I used a sector erase command which seems to not erase all the pages within each sector (I think that happens because of a know firmware bug present in some series of those Flash Memory chips).
Please use the new version included on the Pro2 PLUS software package.

[engolling]

Good evening,
I did some tests with the WeatherDuino and WeeWx the last days. I could not reproduce a big problem while booting. The worst thing happened was that the communication did not start properly.
But WeeWx gets crazy when there are invalid datasets stored onto the logger flash. I tried again a wrong compiled WeatherDuino software causing wrong data on the logger and it simply did not work anymore. It seems that there is communication between the WeatherDuino and WeeWx indicated by the communication LED but no data is stored.

After erasing the flash chip of the WeatherDuino it works fine again.

So I see two possible solutions: Modify the WeeWx Davis driver that it can handle invalid datasets e.g. in the future or in the past or building a WeatherDuino utility checking the integrity of the stored datasets mainly the timestamp.


Regards,
engolling

[werk_ag]

Answering with a though:
Should those checks be done at WeatherDuino level, by a microcontroller with limited memory resources and running at relatively low clock speed, or at weather software level which usually run on machines with lots of memory and much higher clock speeds?

By other side, datasets with a date in the past are absolutely normal in a logger, if they are more in the past than required by the weather software, then they should be rejected by the software (Cumulus 1 and CumulusMX does this). Datasets with a date on the future, can only happen due to something which will cause that the system clock isn't set properly.  As much as we know, the only thing than can cause wrong timestamps is compiling the software with avr 1.6.22 or 1.6.23,

If you wish, you can try to write an weatherduino routine to do what you propose, but then give a look at the size of the resulting code, and the processing overhead that it will introduce (imagine a full logger download: 512 pages * 5 records).

[engolling]

Of course the drivers of the weather software should be robust enough not to get stuck if anything strange happens and it should be able to repair or ignore any faulty data. This unfortunately does not happen in WeeWx.
The result is strange behaviour of the WeeWx interface.
That's great with Cumulus and that is why I'm using cumulus with my production system up to now.

Nevertheless this faults do not occur very often and it might be the easier solution to program a WeatherDuino data integrity check.
Im thinking of a sketch you can upload if there are any issues walking through the data like the eraser and sorting out any date-time combinations that as simply not possible.
Due to the bug i had the 35th of Oktober written on the Display for example. I don't know which timestamp is saved here, but it can not be good  .
Moreover I'm thinking of the possibility to specify a time range where the data should be kept and the other stuff should be deleted.

Until now I do not know how the logger works or how it knows where its data is stored, but I will have a closer look at it the next days.

@Werk There is no need to do anything. I will try to make a sketch to change the logged data in a way WeeWx can handle it and does not get stuck. I think this is the easiest way to get rid of this WeeWx bug in that seldom cases.

Best regards,
engolling

[werk_ag]

Nevertheless this faults do not occur very often and it might be the easier solution to program a WeatherDuino data integrity check.

I should be a very lucky guy!
In almost 5 years of using WeatherDuino I never saw any of the things we are calling "this faults"...

@engolling, We both know that this question about wrong or invalid records was started with a problem caused by the use of AVR boards manager 1.6.22 and 1.6.23 which messes the system clock which by turn causes that records on the data logger have a wrong timestamp.

Considering that there are hundreds of bug reports related to avr 1.6.22 I'm not concerned it it. Recommendation on Arduino.cc is: if a software not works with 1.6.22 or 1.6.23 revert back to 1.6.21

So, why are we complicating or creating the idea that WeatherDuino stores invalid records on the logger?