Extra Sensors

Introduction

Extra Sensors is a feature already present in Cumulus 1. Ignored for long time by CumulusUtils, with the rise of the Ecowitt ecology and the support of CumulusMX for the GW1000 (and higher) devices, it has been decided to make Extra Sensors a standard module of CumulusUtils.

For the obvious reason of the absence of a log, the lightning sensor will be ignored by CumulusUtils as are additional sensors which can not be logged. You are advised to use the CustomLogs to chart Lightning.

Please read the paragraph operation carefully and report any difficulty in operating the module both in this Wiki, from within the website or otherwise. I think all cases are covered but you never know.

For External Extra Sensors i.e. sensors the user has available outside the CMX system such as e.g. a Geiger Counter see: External Extra Sensors.

In CMX version 3.22.0 - b3211 the Custom Logs were introduced, providing the possibility for the user to create their own logfiles for any value - also strings - which CMX produces in a webtag. This called for a procedure in CumulusUtils to handle those logs. In version 6.21.0 this was realised. the Custom Logs procedure us based on the External Sensors module and has its own wiki page.

Dependency

Unlike other modules the Extra Sensors module is dependent on the ChartsCompiler which means that the file CutilsCharts.def must be present in the utils directory. This may contain a user made charts definition but for a starting user it will be easiest to just copy the file CutilsCharts-default-for-use.def from the distribution and rename it to CutilsCharts.def. Make sure that your CMX configured datafiles and the charts definitions agree, if they don't than you won't see data lines or/and get errors in the console.

User made or just default, the file MUST contain the lines

 ; DO NOT CHANGE ANYTHING BELOW THIS INDICATOR LINE
 ; ExtraSensorCharts

And everything below these lines will be replaced by generated ExtraSensors charts.

The user may not remove the above lines but by setting the parameter

UserModificationExtraSensorCharts=true

the charts will no longer be overwritten and the user may modify the generated charts to his own tasting.

Translation/Language

The translation of of this module is accommodated via two sections in the CUstringsXX.ini file: Compiler (most) and ExtraSensors (some).

You will find all translation tags for the sensor names in the section Compiler and this includes the three values for the lightning detector although those are not really explicit sensors. This is analogue to all standard plot variables.

The translation tags content for the extra sensor names are defaulted with the extra sensor description texts from strings.ini when the tags are created in the CUstringsXX.ini and can then be changed by the user. These modifications will be used in subsequent runs for the ExtraSensors module. As a consequence, the user may choose different descriptions in CMX and in CUtils for the Extra Sensors.

To set back the default values of the tag names - the values in strings.ini - just remove the tags from the CUstringsXX.ini file.

Operation

1. CumulusUtils must be run once with the command ExtraSensors (fastest) or Website to create the required parameters in the section [ExtraSensors] in the inifile.
2. The module requires that the user has Extra Sensor logging enabled in CumulusMX(CMX menu=>Station Settings=>Common Options=>Log Extra sensors must be enabled) AND the user must set the parameter ExtraSensors in the cumulusutils.ini file to true. This last parameter makes it possible to have Extra Sensors operational in CMX but not display them on the web (if you are generating the website). Once both parameters are set to true the module is active.
3. After the first run the following inifile parameters exist (see below): ExtraTemp, ExtraHum, ExtraDP, SoilTemp, SoilMoisture, AirQuality, UserTemp, LeafTemp, LeafWetness. Filling these with e.g. 1,2,3 activates those sensors and a subsequent run will show these under the menu choice Extra.
4. For the CO2 sensor the parameter CO2=, a value 0 (or empty ) or 1 (de-)activates that sensor which subsequently is handled as an extra sensor and will be shown under the menu choice Extra.
5. For the lightning sensor the parameter LightningSensor=true/false (de-)activates that sensor which subsequently is displayed as a normal Extra Sensor in the list. The Lichting sensor is currently not charted as it is not logged in the ExtraSensors log. Use the CustomLogs and its module for this.
6. Finally, to make it work and to bring system in the Extra Sensors, it is required for the user to use strings.ini (copy it from samplestrings.ini in the distribution of CumulusMX) in the CumulusMX directory and give the sensors you want to be handled by CumulusUtils a short meaningfull name.

For the Website Generator this means everything is now automatically available in the menu of the website.

The extrasensorsrealtime.txt file needs to be processed in the CMX Extra Webfiles utility and send to the webroot on the realtime frequency.

Using as a module requires some additional editing to make use of the table with the current values (Note the libraries as well). The charts are always available (see outputs). It is advised not to use this module outside the context of the Website Generator but skilled users can use the module as the basis for their own website (as has been done with AirLink and other modules). If you only want the charts then life is more easy and the ChartsCompiler makes it possible to combine virtually every plotparameter with another plotparameter (of RECENT or EXTRA type)

To receive the data for the ExtraSensors you have to run CumulusUtils UserAskedData. It would be logical to do this on the normal FTP frequency on which all other chart data are generated by CMX and by the requirements of the ChartsCompiler and the AirLink module.

The commands for the above named steps are:

 utils/bin/cumulusutils.exe ExtraSensors         => this command creates the outputfiles. 

or:

 utils/bin/cumulusutils.exe website

and:

 utils/bin/cumulusutils.exe UserAskedData                => this command produces the ExtraSensors datafiles with only those data required for the present and activated sensors.

NOTE: UserAskedData is typically used in an interval through the scheduler. See also the AirLink and the ChartsCompiler
NOTE: CUtils creates its own JSON data files and does not make use of the CMX ExtraSensors datafiles so you can switch those off and prevent too much bandwidth usage.

There are some translations in the section [ExtraSensors] in the language file. The names of the Extra Sensors given by the user in strings.ini are used through the language file translations in the [Compiler] section.

Output

The Extra Sensors module has multiple outputs:

1. extrasensors.txt => contains the activation code for the module, activated by the menu choice
2. extrasensorscharts.txt => contains the charts which go with the extra sensors (analogue to the trend charts produced by CMX with the same periodic generation and the same period shown)
3. extrasensorsrealtime.txt => contains the webtags for the current value of the Extra Sensors used. Has to be entered in Extra Webfiles as source to be processed and FTP'd to the webroot at realtime.
4. extrasensorsdata.json => contains all data for the extra sensors specified (by the command utils/cumulusutils.exe UserAskedData.

In addition the module produces a menu item SC map which, when clicked, activates sensorcommunity.txt which is also output of this module. This loads the global map of sensor.community centered on the coordinates of the users weather station. The user may wish to upload his/her data to the sensor.community project which is beyond the scope of CumulusUtils and CumulusMX.

The user can optionally create the charts himself by setting the parameter UserModificationExtraSensorCharts to true.
The outputfile for the charts MUST always be extrasensorscharts.txt.

Inifile Parameters

The Extra Sensor module has the following inifile parameters (see cumulusutils.ini):

 [ExtraSensors]
 ExtraSensors=false                       => If false the module is skipped, no menu entry will be made
 ParticipatesSensorCommunity=false        => If false, the menu option SC map will not be generated
 CleanupExtraSensorslog=false             => If true will remove old logfiles and keep only the current and the previous month
 UserModificationExtraSensorCharts=false  => If true the Extra Sensor charts will not be overwritten in CutilsCharts.def and give the user the possibility to create his own charts
 LightningSensor=false                    => If true the Ecowitt Lightning sensor is present and will be presented in the module.
                                             (NOTE that this sensor is not logged by CMX and therefore does not have a corresponding chart, see Operation paragraph)
 ExternalExtraSensors=                    => A comma separated list of strings naming the External Extra Sensors the user wishes to use (See External Extra Sensors)

This list of parameters defines which extra sensors are used in your system (combine with strings.txt for the descriptions (Captions) used):

 ExtraTemp=                               =>  fill with comma separated digits from 1 - 8
 ExtraHum=                                =>  fill with comma separated digits from 1 - 8
 ExtraDP=                                 =>  fill with comma separated digits from 1 - 8
 SoilTemp=                                =>  fill with comma separated digits from 1 - 16
 SoilMoisture=                            =>  fill with comma separated digits from 1 - 16
 AirQuality=                              =>  fill with comma separated digits from 1 - 4
 UserTemp=                                =>  fill with comma separated digits from 1 - 8
 LeafTemp=                                =>  fill with comma separated digits from 1 - 2
 LeafWetness=                             =>  fill with comma separated digits from 1 - 2
 CO2=                                     =>  fill with 1 

NOTE: For the Ecowitt CO2 sensor to actually show the parameter above is not enough. There must be a value for CO2 in the database.
NOTE: the Ecowitt AirQuality sensors have an implicit accompanying sensor value AirQualityAvg giving the 24 hr average

Inner working

When run it is first checked if anything has to be done through the parameters ExtraSensors (in cumulusutils.ini) and LogExtraSensors (in Cumulus.ini).

When both parameters are true then sensors are checked from the parameters and the names of the extra sensors in strings.ini are fetched.

A sensor present means the realtime data and a corresponding chart will be created for presentation. The only sensor where a parameter is not the only determinant for its presence on the website is the Ecowitt CO2 sensor. Those names may be changed by the user, but only a value in the logfile for the CO2 value is taken as proof of presence.

A structure is created for each sensor present. Each AirQuality sensor is automatically coupled with an AirQualityAvg sensor (which is not a physical sensor by itself) for presentation. For each CO2 sensor all seven successive sensors (see strings.ini) are also created.

With the list of sensors the extrasensorsrealtime.txt is generated, followed by extrasensors.txt (the code) and the charts are added to CutilsCharts.def. The last line of the userdefined charts is:

 ; DO NOT CHANGE ANYTHING BELOW THIS INDICATOR LINE
 ; ExtraSensorCharts

After this line no changes may be made unless the parameter UserModificationExtraSensorCharts=true. This line marks the beginning of the Extra Sensor Chart definitions. The actual charts code is generated by the compiler. The compiler runs implicitly when generating the website or when generating ExtraSensors as a module.

Data generation is separate from the code generation and should run on the FTP frequency like all other data generators of CumulusUtils and CumulusMX. Run:

utils/cumulusutils.exe UserAskedData

in the scheduler. The compiler and/or the AirLink module require that command as well so it may already be present.