Extra Sensors: Difference between revisions

m
 
(38 intermediate revisions by the same user not shown)
'''NOTE: Under construction / module not released yet'''
 
== Introduction ==
{|align=right
''Extra Sensors'' is a feature already present in Cumulus 1. Ignored for long time by CumulusUtils, with the rise of the [https://www.ecowitt.com/shop/homePage Ecowitt ecology] and the support of CumulusMX for the GW1000 (and higher) devices, it has been decided to make ''Extra Sensors'' a standard module of CumulusUtils.
 
For the obvious reason of the absence of a log, the lightning sensor will be ignored by CumulusUtils as are additional sensors which can not be logged. (likeYou e.g.are LeafTemp3advised andto 4).use the [[Custom Logs|CustomLogs]] to chart Lightning.
''Extra Sensors'' have a certain complexity because, although handled by CumulusMX, they are kind of anarchic: unregulated, CumulusMX does not care, when detected they are always displayed only logged when logging is switched on and then not all of them are logged. There are more webtags than logged sensors etc... This complexity made some additional handling necessary.
 
For the obvious reason of the absence of a log, the lightning sensor will be ignored by CumulusUtils as are additional sensors which can not be logged (like e.g. LeafTemp3 and 4).
 
Please read the paragraph operation carefully and report any difficulty in operating the module both in this Wiki, from within the website or otherwise. I think all cases are covered but you never know.
 
For External Extra Sensors i.e. sensors the user has available outside the CMX system such as e.g. a Geiger Counter see: [[External Extra Sensors]].
 
In CMX version 3.22.0 - b3211 the Custom Logs were introduced, providing the possibility for the user to create their own logfiles for any value - also strings - which CMX produces in a [[Webtags|webtag]]. This called for a procedure in CumulusUtils to handle those logs. In version 6.21.0 this was realised. the [[Custom Logs]] procedure us based on the External Sensors module and has its own wiki page.
 
=== Dependency ===
Unlike other modules the Extra Sensors module is dependent on the [[ChartsCompiler]] which means that the file CutilsCharts.def must be present in the utils directory. This may contain a user made charts definition but for a starting user it will be easiest to just copy the file ''CutilsCharts-default-for-use.def'' from the distribution and rename it to ''CutilsCharts.def''. Make sure that your CMX configured datafiles and the charts definitions agree, if they don't than you won't see data lines or/and get errors in the console.
 
User made or just default, the file '''''MUST''''' contain the lines
; DO NOT CHANGE ANYTHING BELOW THIS INDICATOR LINE
; ExtraSensorCharts
And everything below these lines will be replaced by generated ExtraSensors charts.
 
The user may not remove the above lines but by setting the parameter
UserModificationExtraSensorCharts=true
the charts will no longer be overwritten and the user may modify the generated charts to his own tasting.
 
=== Translation/Language ===
The translation of of this module is accommodated via two sections in the ''CUstringsXX.ini'' file: ''Compiler'' (most) and ''ExtraSensors'' (some).
 
You will find all translation tags for the sensor names in the section ''Compiler'' and this includes the three values for the lightning detector although those are not really explicit sensors. This is analogue to all standard plot variables.
 
The translation tags content for the extra sensor names are defaulted with the extra sensor description texts from ''strings.ini'' when the tags are created in the CUstringsXX.ini and can then be changed by the user. These modifications will be used in subsequent runs for the ''ExtraSensors'' module. As a consequence, the user may choose different descriptions in CMX and in CUtils for the ''Extra Sensors''.
 
To set back the default values of the tag names - the values in ''strings.ini'' - just remove the tags from the ''CUstringsXX.ini'' file.
 
== Operation ==
#CumulusUtils must be run once with the command ''ExtraSensors'' (fastest) or ''Website'' to create the required parameters in the section ''[ExtraSensors]'' in the inifile.
#The module requires that the user has ''Extra Sensor'' logging enabled in CumulusMX(''CMX menu=>Station Settings=>Common Options=>Log Extra sensors'' must be enabled) AND the user must set the parameter ''ExtraSensors'' in the cumulusutils.ini file to ''true''. This last parameter makes it possible to have ''Extra Sensors'' operational in CMX but not display them on the web (if you are generating the website). Once both parameters are set to true the module is active.
#After the first run the following inifile parameters exist (see below): ''ExtraTemp, ExtraHum, ExtraDP, SoilTemp, SoilMoisture, AirQuality, UserTemp, LeafTemp, LeafWetness''. Filling these with e.g. '''1,2,3''' activates those sensors and a subsequent run will show these under the menu choice ''Extra''.
#Finally, to make it work and to bring system in the ''Extra Sensors'', '''it is required''' for the user to use ''strings.ini'' (copy it from samplestrings.ini in the distribution of CumulusMX) in the CumulusMX directory and give the sensors you want to be handled by CumulusUtils '''a short meaningfull name'''. Leave the default value or remove the default value to make an empty string to have CumulusUtils ignore the sensor. '''Default value or empty define a sensor as NOT PRESENT.''' Existing sensors which keep their default names (or have empty entries) will be displayed in the CMX interface but will not be used by CumulusUtils.
#For the CO2 sensor the parameter ''CO2='', a value 0 (or empty ) or 1 (de-)activates that sensor which subsequently is handled as an extra sensor and will be shown under the menu choice ''Extra''.
#For the lightning sensor the parameter ''LightningSensor=true/false'' (de-)activates that sensor which subsequently is displayed as a normal Extra Sensor in the list. The Lichting sensor is currently not charted as it is not logged in the ExtraSensors log. Use the [[Custom Logs|CustomLogs and its module]] for this.
#Finally, to make it work and to bring system in the ''Extra Sensors'', '''it is required''' for the user to use ''strings.ini'' (copy it from samplestrings.ini in the distribution of CumulusMX) in the CumulusMX directory and give the sensors you want to be handled by CumulusUtils '''a short meaningfull name'''. Leave the default value or remove the default value to make an empty string to have CumulusUtils ignore the sensor. '''Default value or empty define a sensor as NOT PRESENT.''' Existing sensors which keep their default names (or have empty entries) will be displayed in the CMX interface but will not be used by CumulusUtils.
 
For the [[Website_Generator|Website Generator]] this means everything is now automatically available in the menu of the website.
 
The ''extrasensorsrealtime.txt'' file needs to be processed in the CMX ''Extra Webfiles'' utility and send to the webroot on the realtime frequency.
 
Using as a module requires some additional editing to make use of the table with the current values (Note [[Modules#Javascript_libraries|the libraries]] as well). The charts are always available (see outputs). It is advised not to use this module outside the context of the [[Website_Generator|Website Generator]] but skilled users can use the module as the basis for their own website (as has been done with AirLink and other modules). If you only want the charts then life is more easy and the [[ChartsCompiler]] makes it possible to combine virtually every plotparameter with another plotparameter (of RECENT or EXTRA type)
 
To receive the data for the ExtraSensors you have to run ''CumulusUtils [[UserAskedData]]''. It would be logical to do this on the normal FTP frequency on which all other chart data are generated by CMX and by the requirements of the [[ChartsCompiler]] and the [[AirLink]] module.
 
The commands for the above named steps are:
''utils/bin/cumulusutils.exe ExtraSensors CompileOnly'' => this command creates the outputfiles.
or:
''utils/bin/cumulusutils.exe website''
''utils/bin/cumulusutils.exe [[UserAskedData]]'' => this command produces the ExtraSensors datafiles with only those data required for the present and activated sensors.
 
NOTE: [[UserAskedData]] is typically used in an interval through the scheduler. See also the [[AirLink]] and the [[ChartsCompiler]]<br/>
NOTE: CUtils creates its own JSON data files and does not make use of the CMX ExtraSensors datafiles so you can switch those off and prevent too much bandwidth usage.
 
There are some translations in the section [ExtraSensors] in the language file. The names of the Extra Sensors given by the user in strings.ini are used through the language file translations in the [Compiler] section.
#extrasensors.txt => contains the activation code for the module, activated by the menu choice
#extrasensorscharts.txt => contains the charts which go with the extra sensors (analogue to the trend charts produced by CMX with the same periodic generation and the same period shown)
#extrasensorsrealtime.txt => contains the webtags for the current value of the Extra Sensors used. Has to be entered in Extra Webfiles as source to be processed endand FTP'd to the webroot at realtime.
#extrasensorsdata.json => contains all data for the extra sensors specified (by the command ''utils/cumulusutils.exe [[UserAskedData]]''.
 
In addition the module produces a menu item ''SC map'' which, when clicked, activates ''sensorcommunity.txt'' which is also output of this module. This loads the global map of [https://sensor.community/en/ sensor.community] centered on the coordinates of the users weather station. The user may wish to upload his/her data to the sensor.community project which is beyond the scope of CumulusUtils and CumulusMX.
 
The user can optionally create the charts himself by setting the parameter ''UserModificationExtraSensorCharts'' to falsetrue. <br>
The outputfile for the charts MUST always be ''extrasensorscharts.txt''.
 
CleanupExtraSensorslog=false => If true will remove old logfiles and keep only the current and the previous month
UserModificationExtraSensorCharts=false => If true the ''Extra Sensor'' charts will not be overwritten in ''CutilsCharts.def'' and give the user the possibility to create his own charts
LightningSensor=false => If true the Ecowitt Lightning sensor is present and will be presented in the module.
(NOTE that this sensor is not logged by CMX and therefore does not have a corresponding chart, see [[Extra_Sensors#Operation|Operation paragraph]])
ExternalExtraSensors= => A comma separated list of strings naming the External Extra Sensors the user wishes to use (See [[External Extra Sensors]])
 
This list of parameters defines which extra sensors are used in your system (combine with strings.txt for the descriptions (Captions) used):
 
ExtraTemp= => fill with comma separated digits from 1 - 8
ExtraHum= => fill with comma separated digits from 1 - 8
ExtraDP= => fill with comma separated digits from 1 - 8
SoilTemp= => fill with comma separated digits from 1 - 16
SoilMoisture= => fill with comma separated digits from 1 - 16
AirQuality= => fill with comma separated digits from 1 - 4
UserTemp= => fill with comma separated digits from 1 - 8
LeafTemp= => fill with comma separated digits from 1 - 2
LeafWetness= => fill with comma separated digits from 1 - 2
CO2= => fill with 1
 
NOTE: For the Ecowitt CO2 sensor to actually show the parameter above is not enough. There must be a value for CO2 in the database.<br/>
NOTE: the Ecowitt AirQuality sensors have an implicit accompanying sensor value AirQualityAvg giving the 24 hr average
 
== Inner working ==
When run it is first checked if anything has to be done through the parameters ''ExtraSensors'' (in cumulusutils.ini) and ''LogExtraSensors'' (in Cumulus.ini).
 
When both parameters are true then sensors are checked from the parameters and the names of the extra sensors in ''strings.ini'' are fetched.
When both parameters are true then the names of the extra sensors in ''strings.ini'' are checked if they deviate from the default. If they deviate it means the user has changed the name which is taken as a proof of presence of an ''Extra Sensor''. A sensor present means the realtime data and a corresponding chart will be created for presentation. The only sensor where the name in ''strings.ini'' is not determinant for its presence is the Ecowitt CO2 sensor. Those names may be changed by the user, but only a value in the logfile for the CO2 value is taken as proof of presence.
 
When both parameters are true then the names of the extra sensors in ''strings.ini'' are checked if they deviate from the default. If they deviate it means the user has changed the name which is taken as a proof of presence of an ''Extra Sensor''. A sensor present means the realtime data and a corresponding chart will be created for presentation. The only sensor where thea name in ''strings.ini''parameter is not the only determinant for its presence on the website is the Ecowitt CO2 sensor. Those names may be changed by the user, but only a value in the logfile for the CO2 value is taken as proof of presence.
 
A structure is created for each sensor present. Each AirQuality sensor is automatically coupled with an AirQualityAvg sensor (which is not a physical sensor by itself) for presentation. For each CO2 sensor all seven successive sensors (see strings.ini) are also created.
 
With the list of sensors the ''extrasensorsrealtime.txt'' is generated, followed by ''extrasensors.txt'' (the code) and the charts are added to CutilsCharts.def. The last line of the userdefined charts is:
; DO NOT CHANGE ANYTHING BELOW THIS INDICATOR LINE
; ExtraSensorCharts
; ExtraSensorCharts
After this line no changes may be made unless the parameter ''UserModificationExtraSensorCharts=true''. This line marks the beginning of the ''Extra Sensor Chart definitions''.
The actual charts code is generated by the compiler. The compiler runs implicitly when generating the website. Otherwiseor itwhen has to be run separately whengenerating ExtraSensors is run on its own and regeneration of the charts code would be required (e.g. when addingas a sensor)module. The command then would be:
utils/cumulusutils.exe ExtraSensors CompileOnly
 
Data generation is separate from the code generation and should run on the FTP frequency like all other data generators of CumulusUtils and CumulusMX.