Dayfile.txt

Cumulus maintains a daily log file that holds the highs and lows of each day, as well as a few other nuggets of information. The figures contained in the file are used for the 'This period' display accessed from the View menu and to build any graphs based on daily values.

Viewing the daily log

To view the whole daily log use the dayfile.txt command in Edit menu. This is how you view, and optionally edit, the dayfile.txt from within Cumulus. If you upload the file to your web site, then see AnnualDataSummary for information about one of the ways to show values from this file.

To view a summary of dayfile.txt for a month, calendar year or selected period, use This month, This year or This period within the View menu. By default, these load the details for the current calendar month, the current calendar year or yesterday; i.e. in all cases they exclude the today details that are not stored on dayfile.txt until the end of day rollover.

Dealing with rogue measurements

Cumulus provides via Configuration menu Calibration screen the ability to screen out spikes (magnitude of differences between one value read and next value read) in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with some spikes.

If you discover a rogue measurement (perhaps the wind affected your tiping bucket rain gauge or your weather station just reported a corrupted value), on the day it occured, see today.ini or a FAQ for further advice.

If the rogue measurement is discovered some days after it occured, then in many cases it will have affected your highs and lows for the current month, month-by-month, current year, and/or all-time. As your first step you should update the appropriate field in the row for the affected date in dayfile.txt. Once dayfile.txt is correct the Cumulus editors will allow you to:

• update the highs and lows in Alltime.ini by choosing all time records from the Edit menu. See Alltimelog.txt for current and previous values to help you know what rogue value to hunt for and know what the high/low value was before the rogue affected it.
• update the highs and lows in year.ini by choosing This year's records from the Edit menu.
• update the highs and lows in month.ini by choosing This month's records from the Edit menu. See Diags for current and previous values of high or low in the current month or the immediate preceding month if the rogue was recorded less than 10 days ago.
• update the highs and lows month-by-month in monthlyalltime.ini by choosing Monthly records from the Edit menu. Click the Help button for specific instructions on using Reset and the two Copy column header buttons in this Monthly Records (Highs and Lows) Editor to action all rows.
• Note in each of above 4 editing screens you can:
  1. see the currently stored extremes, and optionally Reset (row by row) to pre-editing value and timestamp.
  2. load the dayfile.txt to view extremes derived from those figures (after your correction of the rogue values) and
  3. optionally Copy (row by row) the logged values (and associated date/time information) into the relevant .ini file.
  4. click the Help button for detailed instructions on using The Records (Highs and Lows) Editors.
  5. store your revised figures by clicking OK (or abandon all your edits by clicking Cancel).

 (Each of these screens is a text editor, and works best when at full screen).

• create the relevant monthly and/or annual NOAA style report by choosing NOAA Monthly Report or NOAA Annual Report from the View menu, then select the required period using the selectors. Click the Update Display button to see various statistics (including mean temperature) calculated. Generation of complete NOAA reports takes most information from dayfile.txt (based on rollover to rollover meteorological days), except average wind speed and dominant wind direction (both of these it calculates from the monthly log files) for period in question. Finally press Save button to store the new or amended report.

Ideally, you will subsequently try to edit the rogue data for the particular time it was logged; see Monthly_log_files#Correcting_any_logged_data_problems, but correcting the daily summary in dayfile.txt must always be the priority.

Editing in Cumulus

The last command in Edit menu is dayfile.txt. This is a text editor, and works best when at full screen. Click the Help button for detailed instructions. Cumulus Help is comprehensive.

If there is an error in dayfile.txt, then it is most likely to be found when you are viewing its data on one of the screens for editing the monthly, annual or all-time extremes. Cumulus will illuminate its Error light if it finds an error in such cases and tell you the line/row number of the first found error, together with some details of the error it found. For example, if a row is blank, a row is duplicated, a field is corrupted, a field does not have an acceptable value, or a field is missing so subsequent fields are to the left of where they should be.

You can use this editor to:

• correct individual values by overtyping,
• use delete key to remove an entire day (e.g. if you get a 'duplicate' error message),
• use insert key to add one or more missing rows (complete days) manually,
• use Create missing button to insert missing rows (complete days) by reading from Monthly_log_files and automatically calculating the best approximations for each field for those missing days.

If you do have a 'duplicate' error, you need to decide which row to delete, and whether to copy any values from that row into the row you are keeping to ensure the correct extremes are retained.

As an alternative to manual line insert, you may wish to use a procedure for importing, and processing, pre-Cumulus observations into Monthly_log_files. Once there is data for required days in monthly logs, Create missing can insert the new rows for those days previously missing in dayfile.txt.

For Create missing a list of inserted records is produced in dayfileeditlog.txt. If just some fields are wrong in a particular row (meteorological day) on day file, then there is a workaround as at all current versions (up to 1.9.4) you can only use 'Create missing' to read from the Monthly_log_files if the whole day (a row starting with a date) is missing in dayfile.txt.

Note for obsolete version 1.9.0 to 1.9.3: There is a bug in these versions in that 'Create missing' inserts 'heating and cooling degree day' values the wrong way round.

Note for obsolete version 1.9.3 only: Create missing might in some cases be affected by a bug in 1.9.3 that can cause incorrect date order for records (dayfile.txt uses dd/mm/yy or dd-mm-yy and all records should be in ascending chronological order)

There are no known bugs in version 1.9.4 build 1086, the current release at time of writing.

Manipulation outside Cumulus

The file is data\dayfile.txt within the directory holding the Cumulus excutable, it can be viewed in a text editor, or imported into various database systems, or imported into spreadsheets, to manipulate as you wish. Just remember that Cumulus updates it as part of the rollover process, so never attempt to work on it either when Cumulus has just been restarted and is checking/updating the logs, or around the midnight/9am/10am local rollover time when Cumulus is writing a new row.

Tip: Take a copy of the file before you work on it outside Cumulus (perhaps call the copy 'dayfile.csv' so Windows can recognise its structure).

Note: Since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end. (Additions for the last few builds only are indicated below, you can explore details of earlier versions via the official product page). You must supply at least 16 fields (i.e. up to and including 'average temperature').

Important Rules:

• The (meteorological) date format uses two digits for the year. Edit this file using an editor that treats all fields as text (a text editor or a spreadsheet program that can be instructed not to recognise special field types). For software (e.g. Excel) with default of recognising formats, ensure that such recognition is turned off, as it is likely to change the dates to either a number representing days since e.g. 31 Dec 1899, or to have four figure years, and then Cumulus will no longer be able to use the log file. (Also the month must be the middle figure, USA convention cannot apply within logfiles).
• Times are in format hh:mm
• Most value fields are in real number format x.y using your system decimal notation, a few (e.g. bearings, solar, humidity) are integers (see #List_of_fields_in_the_file). Whilst an integer can be used for a real number field, decimals are not allowed in an integer field.
• If you insert a lowest or highest value into a day, where there was none before, insert a timestamp too, as a dayfile.txt row is only accepted by the Cumulus editor if a value and any related timestamp are either both present or both absent. (Use a timestamp of your rollovertime 00:00, 09:00 or 10:00 if you have not looked up the precise time).
• Nulls ',,' are not allowed in the line, so if you do not know the value for a particular field within the line, then type in a zero for integer values and an extreme with opposite value (e.g. -999.9 for a signed decimal maximum, and 9999.9 for a decimal minimum) for decimal format (replace the full stops with your decimal separator).
• Make sure that any editing does not create any blank lines in the file.
• All rows must start with date and include some of the parameters listed in correct sequence.
• Rows can vary in length but only by missing off fields at the end.
• The row terminator normally expected is CRLF, ensure any external editor does not change the terminator.

List of fields in the file

The appropriate list of fields for your installed build is stored as dayfileheader.txt within the folder that contains your Cumulus executable.

Listed below is the field number (starting from zero to be consistent with index used for arrays in programming languages like JavaScript, plus spreadsheet column identifier) with description and cross-references to further explanation:

• 00 (A): Date as 2 figure day [separator] 2 figure month [separator] 2 figure year - the separator is that set in the windows system short date format (see setup)
• 01 (B): Highest wind gust
• 02 (C): Bearing of highest wind gust (integer)
• 03 (D): Time of highest wind gust
• 04 (E): Minimum temperature
• 05 (F): Time of minimum temperature
• 06 (G): Maximum temperature
• 07 (H): Time of maximum temperature
• 08 (I): Minimum sea level pressure
• 09 (J): Time of minimum pressure
• 10 (K): Maximum sea level pressure
• 11 (L): Time of maximum pressure
• 12 (M): Maximum rainfall rate
• 13 (N): Time of maximum rainfall rate
• 14 (O): Total rainfall for the day
• 15 (P): Average temperature for the day

(This point represents the minimum length of all records)

• 16 (Q): Total wind run
• 17 (R): Highest Average Wind Speed
• 18 (S): Time of Highest Avg. Wind speed
• 19 (T): Lowest humidity (integer)
• 20 (U): Time of lowest humidity
• 21 (V): Highest humidity (integer)
• 22 (W): Time of highest humidity
• 23 (X): Total evapotranspiration (Only valid for Davis stations, shows zero otherwise)
• 24 (Y): Total hours of sunshine (only valid if sunshine sensor connnected)
• 25 (Z): High Heat index
• 26 (AA): Time of high heat index
• 27 (AB): High Apparent temperature
• 28 (AC): Time of high apparent temperature
• 29 (AD): Low apparent temperature
• 30 (AE): Time of low apparent temperature
• 31 (AF): High hourly rain
• 32 (AG): Time of high hourly rain
• 33 (AH): Low wind chill
• 34 (AI): Time of low wind chill
• 35 (AJ): High dew point
• 36 (AK): Time of high dew point
• 37 (AL): Low dew point
• 38 (AM): Time of low dew point

The next 3 entries were added in version 1.9.2 Build 1004

• 39 (AN): Today's dominant/average wind direction (integer)
• 40 (AO): Heating degree days
• 41 (AP): Cooling degree days

Added in version 1.9.3 build 1036 (these only show valid values if appropriate sensors exist)

• 42 (AQ): High solar radiation
• 43 (AR): Time of high solar radiation
• 44 (AS): High UV Index
• 45 (AT): Time of high UV Index

Example of the file

An extract of a few lines of the dayfile.txt

01/08/11,19.3,61,10:22,12.5,06:58,23.8,14:49,1014.26,20:46,1018.83,09:28,0.0,00:00,0.0,17.8,21.6,4.6,10:44,36,14:14,86,01:56,3.56,8.9,23.8,14:49,23.1,14:50,12.3,06:59,0.0,00:00,12.5,06:58,11.3,00:16,6.9,14:34,354,2.0,1.5

02/08/11,16.1,20,16:55,14.7,06:45,24.2,13:54,1013.79,19:13,1015.65,11:14,0.0,00:00,0.0,18.9,13.7,8.0,15:55,42,20:42,85,06:50,2.79,4.9,24.2,13:54,24.3,13:55,15.1,06:40,0.0,00:00,14.7,06:45,14.8,11:59,7.0,21:09,57,1.0,1.7

03/08/11,14.5,36,17:23,14.9,05:50,24.6,14:46,1012.70,18:44,1015.99,08:34,0.0,00:00,0.0,19.4,17.2,4.8,16:04,50,14:38,79,07:04,3.05,5.8,24.6,14:46,25.4,14:47,15.0,05:50,0.0,00:00,14.9,05:50,14.2,20:01,8.9,00:16,32,0.8,1.9

04/08/11,17.7,16,15:43,14.1,06:20,25.3,15:06,1013.08,18:42,1015.31,08:28,0.0,00:00,0.0,20.2,19.4,8.1,14:12,52,18:20,92,06:55,3.30,9.1,25.3,15:06,26.8,14:55,14.9,06:20,0.0,00:00,14.1,06:20,15.8,14:55,12.5,06:25,36,1.0,2.9

05/08/11,16.1,32,12:52,14.2,06:12,22.2,14:07,1013.89,00:01,1016.36,09:43,0.0,00:00,0.0,18.6,21.6,5.2,13:00,62,15:57,87,06:11,3.30,8.4,22.2,14:07,23.5,14:10,14.8,07:19,0.0,00:00,14.2,06:12,15.4,10:33,12.0,06:03,34,0.9,1.3

06/08/11,16.1,309,11:15,14.3,05:29,22.4,17:12,1014.46,20:02,1016.97,10:38,0.0,00:00,0.0,18.4,19.2,5.5,16:21,55,13:33,92,05:20,2.79,7.9,22.4,17:12,23.3,18:17,15.1,06:09,0.0,00:00,14.3,05:29,14.2,18:12,10.9,10:38,32,1.1,1.3

07/08/11,17.7,342,13:24,12.9,05:47,24.1,14:53,1013.92,19:49,1016.43,09:36,0.0,00:00,0.0,18.4,19.1,6.3,14:06,48,12:45,89,05:36,3.30,9.0,24.1,14:53,24.6,15:48,13.3,05:47,0.0,00:00,12.9,05:47,14.6,15:52,10.7,11:33,11,1.6,1.7