5,838
edits
Extra Sensor Files: Difference between revisions
Add table of fields for MX
m (→Introduction: add category) |
(Add table of fields for MX) |
||
|
{{Template:Version badge Mx}}{{Version badge 1}}This Wiki page applies to both Cumulus flavours.[[Category:Cumulus MX]][[Category:Cumulus Files]][[Category:MX txt Files]][[Category:Cumulus 1]] <big>This page was however created when only the legacy Cumulus 1 existed, and would benefit from further improvement now that MX is the dominant flavour in use.</big>
=The extra sensor files=
If the ''Extra Sensors'' configuration setting is ticked, see [[#Extra Sensors|instructions later]], files are automatically created with names in the form <tt>ExtraLogyyyymm.txt</tt>. For example, a file for December 2003 is called ''ExtraLog200312.txt''.
The files are pure text files, containing just the digits 0 to 9, and some punctuation. Each file is in '''comma separated value''' (CSV) format with one line per entry. There is some information on these files in the Cumulus 1 help file, in the section “The Data log file”.
''Note that 'comma-separated' does not necessarily mean that a comma is used to separate the values!'' If your computer locale/region uses a semicolon for the list separator, that will be used in these files.
The fields that can appear in the file vary depending on which release you are using. The list of fields available for your release can be seen in a file: <code>Extrafileheader.txt</code> stored in the folder containing the executable. The standard names shown in that list can be [[#General customisation|customised, as described below,]] to names that suit you better.
There are lots of extra sensor fields in the legacy Cumulus, with many more in MX. It is probably technically impossible to have hardware that would use all possible fields, so most files will contain an awful lot of zeroes representing sensors not used in your implementation.
Unfortunately, MX kept compatibility with the legacy Cumulus, and just added extra fields to the end, rather than redesigning the whole concept and creating a new file design that just contains the fields currently being used.
== Uploading the latest log to your web server==
In MX, if you use build 3087 (or later) then you can easily upload the latest log, by using the '''Extra web files''' page in the [[File:Extra web settings.png|Right]][[MX_Administrative_Interface#Extra_Web_Files|interface]]. Simply use '''<currentextralogfile>''' as the ''Local Filename''. Tick the '''UTF8''' checkbox. Optionally, tick '''End of Day'''. Fill in the remote path.
In the legacy Cumulus, see [[Cumulus.ini_(Cumulus_1)#Section:_FTP_Site]], you will manually have to enter the month and year for 'yyyymm' if you want the current log, and you need to do this each month, the whole parameter is therefore in format '''ExtraLocal[0-99]=ExtraLogyyyymm.txt'''. Alternatively, you will have to use a third party tool like [[CumulusToolbox]] to upload the log.
== List of fields in the file for MX==
Fields are numbered from 1, as MX internally treats field zero as the line number.
The letter(s) appearing in brackets after the field number represents how the respective column would be identified in a spreadsheet like Libre Office, Open Office, Excel etc.
<big>This table has some blanks, awaiting completion by a contributor who uses the relevant functionality</big>
{| border="1" class="wikitable"
|-
! style="width:60px" | Field #
! style="width:180px" | Rule
! style="width:80px" | Example
! style="width:450px" | Description
|-
| Colspan="4" style="background: pink;"| The fields in this first section are unchanged from the legacy Cumulus 1
|-
| 1(A)
|Day-Month-Year (2 digits each)
|22/04/11
| Date as ''2 figure day [separator] 2 figure month [separator] 2 figure year'' - the separator is that set in the your locale for short date format (see [[setup]])
|-
| 2(B)
|Hour-Minute (2 digits each)
|10:25
| Current time to nearest minute (no seconds allowed). Separator must be colon in MX, but can also be set to something else in Cumulus 1
|-
| 3 to 12 (C to L)
| signed real number
| -2.4
| Ten extra temperature values
|-
| 13 to 22 (M to V)
| positive number up to 100
| 69
| Ten extra humidity values
|-
| 23 to 32 (W to AF)
| signed real number
| 12.4
| Ten extra dew point values
|-
| 33 to 36 (AG to AJ)
| signed real number
| -2.4
| Four soil temperature values
|-
| 37 to 40 (AK to AN)
|
|
| Four soil moisture values
|-
| 41 to 42 (AO to AP)
| signed real number
| -2.4
| Two leaf temperature values
|-
| 43 to 44 (AQ to AR)
|
|
| Two leaf wetness values
|-
| Colspan="4" style="background: pink;"| The next section has fields that relate to Ecowitt GW1000 (Froggit DS1500) and are available from 3.9.0
|-
| 45 to 48 (AS to AV)
|
|
| Four Air Quality values
|-
| 49 to 56 (AW to BD)
| signed real number
| -2.4
| Eight user-defined temperatures
|-
| Colspan="4" style="background: pink;"| The next section has fields that relate to Ecowitt WH45 CO₂ sensor and are available from 3.9.5
|-
|-
| 57 (BE)
|
|
| Carbon Dioxide monitor value
|-
| 58 (BF)
|
|
| Rolling CO<sub>2</sub> average
|-
| 59 (BG)
|
|
| Air Quality expressed in terms of particulate matter of 2.5 micrometres or less
|-
| 60 (BH)
|
|
| Air Quality expressed in terms of particulate matter of 2.5 μm, or less, Rolling 24 hours average
|-
| 61 (BI)
|
|
| Air Quality expressed in terms of particulate matter of 10 μm, or less
|-
| 62 (BJ)
|
|
| Air Quality expressed in terms of particulate matter of 10 μm, or less, Rolling 24 hours average
|-
| 63 (BK)
| signed real number
| -2.4
| Temperature reported by sensor
|-
| 64 (BL)
| positive number up to 100
| 69
| Relative Humidity reported by sensor
|}
Here are a few lines taken from a log file processed by MX prefixed by the default field labels:
<pre>Date (dd/mm/yy)|Time|Temperature1|Temperature2|Temperature3|Temperature4|Temperature5|Temperature6|Temperature7|Temperature8|Temperature9|Temperature10|Humidity1|Humidity2|Humidity3|Humidity4|Humidity5|Humidity6|Humidity7|Humidity8|Humidity9|Humidity10|Dew point1|Dew point2|Dew point3|Dew point4|Dew point5|Dew point6|Dew point7|Dew point8|Dew point9|Dew point10|Soil temp1|Soil temp2|Soil temp3|Soil temp4|Soil Moisture1|Soil Moisture2|Soil Moisture3|Soil Moisture4|Leaf temp1|Leaf temp2|Leaf wetness1|Leaf wetness2|Soil temp5|Soil temp6|Soil temp7|Soil temp8|Soil temp9|Soil temp10|Soil temp11|Soil temp12|Soil temp13|Soil temp14|Soil temp15|Soil temp16|Soil Moisture5|Soil Moisture6|Soil Moisture7|Soil Moisture8|Soil Moisture9|Soil Moisture10|Soil Moisture11|Soil Moisture12|Soil Moisture13|Soil Moisture14|Soil Moisture15|Soil Moisture16|Air Quality1|Air Quality2|Air Quality3|Air Quality4|Air QualityAvg1|Air QualityAvg2|Air QualityAvg3|Air QualityAvg4|User temp1|User temp2|User temp3|User temp4|User temp5|User temp6|User temp7|User temp8|CO2|CO2Avg|CO2 pm2.5|CO2 pm2.5 avg|CO2 pm10|CO2 PM10 avg|CO2 Temp|CO2 Hum
04/06/20,11:40,16.7,19.4,21.1,21.1,21.7,24.4,22.8,0.0,0.0,0.0,91,72,62,86,91,0,0,0,0,0,15.2,14.3,13.5,18.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0.0,0.0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
04/06/20,11:45,16.7,19.4,21.1,21.1,21.7,23.9,22.8,0.0,0.0,0.0,91,72,62,86,91,0,0,0,0,0,15.2,14.3,13.5,18.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0.0,0.0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
04/06/20,11:50,16.7,19.4,21.1,21.1,21.7,24.4,22.8,0.0,0.0,0.0,91,72,62,86,91,0,0,0,0,0,15.2,14.3,13.5,18.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0.0,0.0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
04/06/20,11:55,16.7,19.4,21.1,21.1,21.7,23.9,20.6,0.0,0.0,0.0,91,72,62,86,91,0,60,0,0,0,15.2,14.3,13.5,18.7,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0.0,0.0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0,0,0,0,0,0,0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0
</pre>
== List of fields in the file for Cumulus 1 ==
The field numbers shown start from zero (to be consistent with index used for arrays in programming languages like JavaScript).
The letter(s) appearing in brackets after the field number represents how the respective column would be identified in a spreadsheet like Libre Office, Open Office, Excel etc.
The fields are listed here (they also appear at start of table above):
* 00(A): Date as ''2 figure day [separator] 2 figure month [separator] 2 figure year'' - the separator is that set in the windows system short date format (see [[setup]])
* 01(B): Current time in the format ''hh:mm'' using the 24 hour clock and local time (system time)
* 02 to 11 (C to L): Ten extra temperature values
* 12 to 21 (M to V): Ten extra humidity values
* 22 to 31 (W to AF): Ten extra dew point values
* 32 to 35 (AG to AJ): Four soil temperature values
* 36 to 39 (AK to AN): Four soil moisture values
* 40 to 41 (AO to AP): Two leaf temperature values
* 42 to 43 (AQ to AR): Two leaf wetness values
= Extra Sensors=
Cumulus software has the ability to read a number of weather station types, some of these support extra optional sensors. The configuration settings determine whether the software looks for extra sensors or not:
* [[File:Badge vMx.png]] For MX, Settings Menu → Station settings → Common Options → Extra Sensors tick box
* [[File:Badge v1.png]] For the legacy Cumulus 1 software, configuration menu → '''Station settings''' screen → the settings frame → Extra Sensors tick box
For the basic sensors, common to many weather station types, Cumulus periodically logs spot values into [[Standard log files]] creating a new file each [[Monthly log files|month]], but the software is also able to [[Calculate_Missing_Values#Derived_value|derive more from those values]] and to track them so it can [[Correcting_Extremes|maintain a collection of extreme records]] for daily, monthly, yearly, and all-time periods. In turn, those enable a large collection of [[Webtags|Web tags]] to be provided.
For the extra sensors, Cumulus does the bare minimum. It can read values from the weather station for those sensors, and periodically log spot values into the files that this page documents.
All data is logged in the units which which have been selected by the Cumulus user, so a conversion is applied if the values are read from the weather station in other units, without any ability for any readings to be calibrated.
As at August 2013, Cumulus has no tracking of extra sensor readings, so it does not calculate extremes for these extra sensors (unless it is the Oregon Scientific extra temperature sensor replacing the main one, see below), and extra sensors do not feed into the [[Category:Ini Files|extreme record tracking files]].
If you did want to try to calculate extremes for yourself, or draw graphs based on these logged extra values, then you must remember that as only spot values are logged, any extremes between the times that spot values are logged are lost. But if you are skilled enough, it is possible to write [[PHP#Incorporating_Script|a script]] to read the file into either an array or database table for further processing. There are advantages in using databases, as Structured Query Language (SQL) makes reading and processing data from a database table easier than the code you would need to write to extract the same information directly from the original file. You might take inspiration from some php scripts available at [https://cumulus.hosiene.co.uk/viewtopic.php?f=18&t=18096 Forum's Create missing topic].
== General customisation ==
Cumulus has [[:Category:Configuration Files|two configuration files]], neither of which is included in any release download.
# The configuration file '''Cumulus.ini''' is created by Cumulus itself to hold the various configuration settings; it will include <code>LogExtraSensors=1</code> in its '''[Station]''' section to indicate that ''Extra sensors'' has been selected.
# The configuration file called [[Strings.ini|strings.ini]] is optional, it can be created taking section labels and attributes from a file included in the software package called [[Samplestring.ini|samplestring.ini]].
The section label in '''samplestring.ini''' that is relevant is section '''[ExtraSensorCaptions]'''. Below it in that source file are [[Strings.ini#Extra_Temperature_Captions.2C_Extra_Humidity_Captions.2C_Extra_Dew_Point_Captions.2C_Soil_Temperature_Captions.2C_Soil_Moisture_Captions.2C_Leaf_Temperature_Captions.2C_and_leaf_Wetness_Captions|a number of parameters]] each must be on a separate line and are formatted as <code>attribute=value</code>.
To associate the extra sensors with descriptive labels that make sense to us:
# Create a text file called '''strings.ini''', if it does not already exist, in the same folder as the executable
# Add a section heading '''[ExtraSensorCaptions]'', if it does not already exist
# Add parameters, copied from '''samplestring.ini''', each on a separate line, and modify the value after the equals sign into what we would like the specific sensor to be labelled.
=Extra sensors supported by Cumulus 1=
Cumulus 1 provides some support for extra sensors that can be connected to Oregon Scientific (e.g. WMR-928) and Davis (e.g.Vantage Pro 2 wireless models can have extra sensors whose readings can be interrogated via the Davis DLL) weather stations. Please note, this is nothing to do with certain [[EasyWeather_Format#Note_on_Models|Fine Offset models]] including solar and UV sensors.
'''NOTE:''' This article may need to be updated by somebody who actually has and uses such instruments. It appears from the support forum:
*the Davis DLL may be able to cope with up to 8 temperature/humidity/dewpoint sensors plus a small number of soil temperature, soil moisture, leaf temperature, and leaf wetness sensors.
For Oregon Scientific WMR-928 weather stations, the ''WMR928 extra channel captions'' frame on the '''Display Settings''' screen within the ''Configuration'' menu allows you to change the captions displayed with values for 3 extra channels (see [[Cumulus.ini_(Cumulus_1)#Section:_Display]]).
For WMR-928 stations (and other ''serial'' stations using the same protocol e.g. WMR-968) to use an 'extra' channel as your main temperature sensor, set N to the required channel by adding an extra line ''WMR928TempChannel=N'' parameter to the [[Cumulus.ini_(Cumulus_1)#Oregon_Scientific_specific:|Station section of cumulus.ini]].
For WMR-100, WMR-200 stations (and any other Oregon ''USB'' stations which work with Cumulus) you can optionally replace your main outdoor temperature with an extra sensor,
== Viewing '''current''' sensor values in legacy Cumulus 1 ==
[[File:Extra_panel.JPG]] You can add a panel to the main screen to display the Oregon Scientific extra channels - by selecting ''Display solar/extra data'' in the General block of the Display settings screen in the Configuration menu.
The same selection adds a similar panel in the same place to display Davis solar sensor output.
[[File:Cumulus 1 View menu.png]] On your computer from '''View''' menu, select ''Extra data'' screen. This lists all the possible extra sensor parameters including their units, with current values shown where they are in use. Note - the units are shown beside values and a change was made in v.1.9.4 build 1083 to ensure this screen reflects any change to units while Cumulus is running.
== Viewing historic sensor values in Extra Sensor Log files in legacy Cumulus 1 ==
In the '''View''' menu, select ''Extra data logs'', click '''Load...'''. The standard Windows File Select dialogue is displayed for the 'data' subdirectory of the Cumulus installation directory. Select the required monthly extra sensor log, click '''Open''' and you will see the contents in a neat table with column headings, rows striped, and the date/time on a lighter background. This is a text viewer, and works best when at full screen, but even then you are likely to need to scroll both horizontally and vertically to look at all the figures. If you click on an individual figure the cell will be highlighted, but you cannot edit the figures on a view screen.
If no logs seen, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Also note, this '''Extra data logs''' viewer is only for viewing monthly extra sensor log files (not any other files in the same ''data'' directory), the column headings for extra sensor files will always be shown at the top of the screen.
=== Adding extra sensor values to website pages ===
The values from extra sensors cannot be automatically included in the standard uploads to web sites without some actions by you.
Cumulus 1: On your website, use the [[Webtags#Extra_Sensors|web tags listed here]] to either display current values on your 'latest values' [[Customised_templates|template page]] (this could be the ''indexT.htm'' page) that is processed by Cumulus, or use those web tags as parameters for feeding your own graphics as on the 'gaugesT.htm' page.
== Manipulation outside Cumulus ==
Any log files for ''previous'' months can be edited (outside Cumulus) with Cumulus running. The current month log file can also be edited outside Cumulus, but you must stop Cumulus first.
'''Tips''' -- take a copy of the original log file before you work on it outside Cumulus (perhaps give the ''not to be touched copy'' a filename of "ExtraLog<Year><Month>.csv").
Edit the original file using an editor that treats all fields as text [either any text editor, a Comma Separated Value editor, or a spreadsheet program that can be instructed ''not'' to recognise special field (like date) types].
| |||