Dayfile.txt
Cumulus maintains a daily file of the highs and lows of each day, as well as a few other nuggets of information. The data contained in the file are used to build the graphs.
Dealing with rogue measurements
Cumulus provides via Configuration menu Calibration screen the ability to screen out spikes in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with some spikes. But some rogue values may be within the accepted all-time range, but just not really applicable to that particular day.
If you discover a rogue measurement (perhaps the wind affected your tiping bucket rain gauge or your weather station just reported a corrupted value), on the day it occured, see today.ini or FAQ for further advice.
If the rogue measurement is discovered some days after it occured, then in many cases it will have affected your highs and lows for the current month, month-by-month, current year, and/or all-time. As first step you should update the appropriate field in the row for the affected date in dayfile.txt. See Alltimelog.txt for current and previous values to help you know what rogue value to hunt for and know what the high/low value was before the rogue affected it. Once dayfile.txt is correct the Cumulus editors will allow you to:
- update the highs and lows in Alltime.ini by choosing all time records from the Edit menu.
- update the highs and lows in year.ini by choosing This year's records from the Edit menu.
- update the highs and lows in month.ini by choosing This month's records from the Edit menu. See Diags for current and previous values of high or low in the current month or the immediate preceding month if the rogue was recorded less than 10 days ago.
- update the highs and lows month-by-month in monthlyalltime.ini by choosing Monthly records from the Edit menu. Click the Help button for specific instructions on using Reset and the two Copy column header buttons in this Monthly Records (Highs and Lows) Editor to action all rows.
Note in each of above 4 editing screens you can:
- see the currently stored extremes, and optionally Reset (row by row) to pre-editing value and timestamp.
- load the dayfile.txt to view extremes derived from those figures (after your correction of the rogue values) and
- optionally Copy (row by row) the logged values (and associated date/time information) into the relevant .ini file.
- click the Help button for detailed instructions on using The Records (Highs and Lows) Editors.
- store your revised figures by clicking OK (or abandon all your edits by clicking Cancel).
(Each of these screens is a text editor, and works best when at full screen).
Ideally, you will subsequently try to edit the rogue data for the particular time it was logged see Monthly_log_files#Correcting_any_logged_data_problems, but correcting the daily summary in dayfile.txt must always be the priority.
Editing in Cumulus
The last command in Edit menu is dayfile.txt. This is a text editor, and works best when at full screen.
You can use this editor to correct individual values by overtyping.
The Create missing button will insert missing rows (complete days) by reading and calculating from Monthly_log_files (a technique that is useful for importing, and processing, pre-Cumulus observations). Click the Help button for detailed instructions. Cumulus Help is comprehensive and explains how a list of inserted records is produced. The procedure is explained in the Wiki topic in greater context detail than is possible in the Cumulus Help.
If just some fields are missing from a particular row (meteorological day) on day file, then there is a workaround as at all current versions (up to 1.9.4) you can only import from the Monthly_log_files if the whole date (row) is missing in dayfile.txt.
Note for version 1.9.3 only: Create missing might in some cases be affected by a bug in 1.9.3 that can cause incorrect date order for records (dayfile.txt uses dd/mm/yy or dd-mm-yy and all records should be in ascending chronological order), any bugs are believed fixed in version 1.9.4.
Manipulation outside Cumulus
The file is called dayfile.txt which can be viewed in a text editor or imported into various database systems or spreadsheets to manipulate as you wish. Just remember that Cumulus updates it as part of the rollover process, so never attempt to work on it either when Cumulus has just been restarted and is checking/updating the logs, or around the midnight/9am/10am local rollover time when Cumulus is writing a new row.
Tip: Take a copy of the file before you work on it outside Cumulus (perhaps call the copy 'dayfile.csv' so Windows can recognise its structure).
Note: Since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end. (Additions for the last few builds only are indicated below, you can explore details of earlier versions via the official product page).
Important Rules:
- The (meteorological) date format uses two digits for the year. Edit this file using an editor that treats all fields as text (a text editor or a spreadsheet program that can be instructed not to recognise special field types). - Do not edit this file using Excel or any similar tool that recognizes dates, as it is likely to change the dates to have four figure years, and then Cumulus will no longer be able to use the file.
- Times are in format hh:mm, and most value fields are in format x.y using your system decimal notation, a few (e.g. bearings, solar, humidity) are integers.
- If you insert a lowest or highest value into a day, where there was none before, insert a timestamp too, as a dayfile.txt row is only accepted by the Cumulus editor if a value and any related timestamp are either both present or both absent. (Use a timestamp of your rollovertime 00:00, 09:00 or 10:00 if you have not looked up the precise time).
- Nulls ',,' are not allowed in the line, so if you do not know the value for a particular field within the line, then type in a zero for integer values and an extreme with opposite value (e.g. -999.9 for a signed decimal maximum, and 999.9 for a decimal minimum) for decimal format (replace the full stops with your decimal separator).
- Make sure that any editing does not create any blank lines in the file.
- All rows must start with date and include some of the parameters listed in correct sequence.
- Rows can vary in length but only by missing off fields at the end.
- The row terminator normally expected is CRLF, ensure any external editor does not change the terminator.
List of fields in the file
- Alternative lists:
- A version of this list with spreadsheet column letters can be downloaded from the support forum here,
- the appropriate list of fields for your installed build is stored as dayfileheader.txt within the folder that contains your Cumulus executable.
The field number (starting from zero to be consistent with index used for arrays in programming languages like JavaScript) with description and cross-references to further explanation:
- 00: Date as day [separator] month [separator] 2 figure year - the separator is that set in the windows system short date format (see setup)
- 01: Highest wind gust
- 02: Bearing of highest wind gust (integer)
- 03: Time of highest wind gust
- 04: Minimum temperature
- 05: Time of minimum temperature
- 06: Maximum temperature
- 07: Time of maximum temperature
- 08: Minimum sea level pressure
- 09: Time of minimum pressure
- 10: Maximum sea level pressure
- 11: Time of maximum pressure
- 12: Maximum rainfall rate
- 13: Time of maximum rainfall rate
- 14: Total rainfall for the day
- 15: Average temperature for the day
- 16: Total wind run
- 17: Highest Average Wind Speed
- 18: Time of Highest Avg. Wind speed
- 19: Lowest humidity
- 20: Time of lowest humidity
- 21: Highest humidity
- 22: Time of highest humidity
- 23: Total evapotranspiration
- 24: Total hours of sunshine
- 25: High Heat index
- 26: Time of high heat index
- 27: High Apparent temperature
- 28: Time of high apparent temperature
- 29: Low apparent temperature
- 30: Time of low apparent temperature
- 31: High hourly rain
- 32: Time of high hourly rain
- 33: Low wind chill
- 34: Time of low wind chill
- 35: High dew point
- 36: Time of high dew point
- 37: Low dew point
- 38: Time of low dew point
The next 3 entries were added in version 1.9.2 Build 1004
- 39: Today's dominant/average wind direction (integer)
- 40: Heating degree days
- 41: Cooling degree days
Added in version 1.9.3 build 1036
- 42: High solar radiation
- 43: Time of high solar radiation
- 44: High UV Index
- 45: Time of high UV Index
Example of the file
An extract of a few lines of the dayfile.txt
01/08/11,19.3,61,10:22,12.5,06:58,23.8,14:49,1014.26,20:46,1018.83,09:28,0.0,00:00,0.0,17.8,21.6,4.6,10:44,36,14:14,86,01:56,3.56,8.9,23.8,14:49,23.1,14:50,12.3,06:59,0.0,00:00,12.5,06:58,11.3,00:16,6.9,14:34,354,2.0,1.5 02/08/11,16.1,20,16:55,14.7,06:45,24.2,13:54,1013.79,19:13,1015.65,11:14,0.0,00:00,0.0,18.9,13.7,8.0,15:55,42,20:42,85,06:50,2.79,4.9,24.2,13:54,24.3,13:55,15.1,06:40,0.0,00:00,14.7,06:45,14.8,11:59,7.0,21:09,57,1.0,1.7 03/08/11,14.5,36,17:23,14.9,05:50,24.6,14:46,1012.70,18:44,1015.99,08:34,0.0,00:00,0.0,19.4,17.2,4.8,16:04,50,14:38,79,07:04,3.05,5.8,24.6,14:46,25.4,14:47,15.0,05:50,0.0,00:00,14.9,05:50,14.2,20:01,8.9,00:16,32,0.8,1.9 04/08/11,17.7,16,15:43,14.1,06:20,25.3,15:06,1013.08,18:42,1015.31,08:28,0.0,00:00,0.0,20.2,19.4,8.1,14:12,52,18:20,92,06:55,3.30,9.1,25.3,15:06,26.8,14:55,14.9,06:20,0.0,00:00,14.1,06:20,15.8,14:55,12.5,06:25,36,1.0,2.9 05/08/11,16.1,32,12:52,14.2,06:12,22.2,14:07,1013.89,00:01,1016.36,09:43,0.0,00:00,0.0,18.6,21.6,5.2,13:00,62,15:57,87,06:11,3.30,8.4,22.2,14:07,23.5,14:10,14.8,07:19,0.0,00:00,14.2,06:12,15.4,10:33,12.0,06:03,34,0.9,1.3 06/08/11,16.1,309,11:15,14.3,05:29,22.4,17:12,1014.46,20:02,1016.97,10:38,0.0,00:00,0.0,18.4,19.2,5.5,16:21,55,13:33,92,05:20,2.79,7.9,22.4,17:12,23.3,18:17,15.1,06:09,0.0,00:00,14.3,05:29,14.2,18:12,10.9,10:38,32,1.1,1.3 07/08/11,17.7,342,13:24,12.9,05:47,24.1,14:53,1013.92,19:49,1016.43,09:36,0.0,00:00,0.0,18.4,19.1,6.3,14:06,48,12:45,89,05:36,3.30,9.0,24.1,14:53,24.6,15:48,13.3,05:47,0.0,00:00,12.9,05:47,14.6,15:52,10.7,11:33,11,1.6,1.7
