Cumulus Wiki

Dayfile.txt

Revision as of 16:34, 2 July 2013 by Sfws (talk | contribs) (→‎Manipulation outside Cumulus: explain connection between low/high value and timestamp as mentioned by Steve Loft in forum reply)

Cumulus maintains a daily file of the highs and lows of each day, as well as a few other nuggets of information. The data contained in the file are used to build the graphs.

Dealing with rogue measurements

Cumulus provides via Configuration menu Calibration screen the ability to screen out spikes in data picked up from your weather station. See Cumulus help screen if you decide to use that to cope with some spikes. But some rogue values may be within the accepted all-time range, but just not really applicable to that particular day.

If you discover a rogue measurement (perhaps the wind affected your tiping bucket rain gauge or your weather station just reported a corrupted value), on the day it occured, see today.ini or FAQ for further advice.

If the rogue measurement is discovered some days after it occured, then in many cases it will have affected your highs and lows for the current month, month-by-month, current year, and/or all-time. As first step you should update the appropriate field in the row for the affected date in dayfile.txt. See Alltimelog.txt for current and previous values to help you know what rogue value to hunt for and know what the high/low value was before the rogue affected it. Once dayfile.txt is correct the Cumulus editors will allow you to:

  • update the highs and lows in Alltime.ini by choosing all time records from the Edit menu.
  • update the highs and lows in year.ini by choosing This year's records from the Edit menu.
  • update the highs and lows in month.ini by choosing This month's records from the Edit menu. See Diags for current and previous values of high or low in the current month or the immediate preceding month if the rogue was recorded less than 10 days ago.
  • update the highs and lows month-by-month in monthlyalltime.ini by choosing Monthly records from the Edit menu. Click the Help button for specific instructions on using Reset and the two Copy column header buttons in this Monthly Records (Highs and Lows) Editor to action all rows.
  Note in each of above 4 editing screens you can:
  1. see the currently stored extremes, and optionally Reset (row by row) to pre-editing value and timestamp.
  2. load the dayfile.txt to view extremes derived from those figures (after your correction of the rogue values) and
  3. optionally Copy (row by row) the logged values (and associated date/time information) into the relevant .ini file.
  4. click the Help button for detailed instructions on using The Records (Highs and Lows) Editors.
  5. store your revised figures by clicking OK (or abandon all your edits by clicking Cancel).
 (Each of these screens is a text editor, and works best when at full screen).

Ideally, you will subsequently try to edit the rogue data for the particular time it was logged see Monthly_log_files#Correcting_any_logged_data_problems, but getting the daily summary in dayfile.txt must always be the priority.

Editing in Cumulus

The last command in Edit menu is dayfile.txt. This is a text editor, and works best when at full screen.

You can use this editor to correct individual values by overtyping.

The Create missing button will insert missing rows (complete days) by reading and calculating from Monthly_log_files (a technique that is useful for importing, and processing, pre-Cumulus observations). Click the Help button for detailed instructions. Cumulus Help is comprehensive and explains how a list of inserted records is produced. The procedure is explained in the Wiki topic in greater context detail than is possible in the Cumulus Help.

If just some fields are missing from a particular row (meteorological day) on day file, then there is a workaround as at all current versions (up to 1.9.4) you can only import from the Monthly_log_files if the whole date (row) is missing in dayfile.txt.

Note for version 1.9.3 only: Create missing might in some cases be affected by a bug in 1.9.3 that can cause incorrect date order for records (dayfile.txt uses dd/mm/yy or dd-mm-yy and all records should be in ascending chronological order), any bugs are believed fixed in version 1.9.4.

Manipulation outside Cumulus

The file is called dayfile.txt which can be viewed in a text editor or imported into various database systems or spreadsheets to manipulate as you wish. Just remember that Cumulus updates it as part of the rollover process, so do not work on it when Cumulus has just been restarted or around the midnight/9am/10am local rollover time.

Tip: Take a copy of the file before you work on it outside Cumulus (perhaps call the copy 'dayfile.csv' so Windows can recognise its structure).

Rules: Each line must contain the fields in correct sequence (since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end). [Additions for the last few builds only are indicated below, you can explore details of earlier versions via the official product page].

Important:

  • The (meteorological) date format uses two digits for the year. Edit this file using an editor that treats all fields as text (a text editor or a spreadsheet program that can be instructed not to recognise special field types). - Do not edit this file using Excel or any similar tool that recognizes dates, as it is likely to change the dates to have four figure years, and then Cumulus will no longer be able to use the file.
  • Times are in format hh:mm, and most value fields are in format x.y using your system decimal notation.
  • If you insert a lowest or highest value into a day, where there was none before, insert a timestamp too, as a dayfile.txt row is only accepted by the Cumulus editor if a value and any related timestamp are either both present or both absent.
  • Also, make sure that any editing does not create any blank lines in the file, all must start with date and include some of the parameters listed in correct sequence.
  • Nulls ',,' are not allowed in the line, so if you do not know the value for a particular field within the line, then type in an extreme with opposite value (e.g. -999.9 for a signed decimal maximum, and 999.9 for a decimal minimum - assuming your system decimal format is using full stops).

Format of the file

A version of this list with spreadsheet column letters can be downloaded from the support forum here, the appropriate version of this list for your installed build (without cross-references or build information) is stored as dayfileheader.txt within the folder that contains your Cumulus executable.

  • 00: Date in the form dd/mm/yy (or sometimes dd-mm-yy, the separator can vary and is unimportant)
  • 01: Highest wind gust
  • 02: Bearing of highest wind gust (integer)
  • 03: Time of highest wind gust
  • 04: Minimum temperature
  • 05: Time of minimum temperature
  • 06: Maximum temperature
  • 07: Time of maximum temperature
  • 08: Minimum sea level pressure
  • 09: Time of minimum pressure
  • 10: Maximum sea level pressure
  • 11: Time of maximum pressure
  • 12: Maximum rainfall rate
  • 13: Time of maximum rainfall rate
  • 14: Total rainfall for the day
  • 15: Average temperature for the day
  • 16: Total wind run
  • 17: Highest Average Wind Speed
  • 18: Time of Highest Avg. Wind speed
  • 19: Lowest humidity
  • 20: Time of lowest humidity
  • 21: Highest humidity
  • 22: Time of highest humidity
  • 23: Total evapotranspiration
  • 24: Total hours of sunshine
  • 25: High Heat index
  • 26: Time of high heat index
  • 27: High Apparent temperature
  • 28: Time of high apparent temperature
  • 29: Low apparent temperature
  • 30: Time of low apparent temperature
  • 31: High hourly rain
  • 32: Time of high hourly rain
  • 33: Low wind chill
  • 34: Time of low wind chill
  • 35: High dew point
  • 36: Time of high dew point
  • 37: Low dew point
  • 38: Time of low dew point

The next 3 entries were added in version 1.9.2 Build 1004

Added in version 1.9.3 build 1036

  • 42: High solar radiation
  • 43: Time of high solar radiation
  • 44: High UV Index
  • 45: Time of high UV Index

Example of the file

An extract of a few lines of the dayfile.txt 

01/08/11,19.3,61,10:22,12.5,06:58,23.8,14:49,1014.26,20:46,1018.83,09:28,0.0,00:00,0.0,17.8,21.6,4.6,10:44,36,14:14,86,01:56,3.56,8.9,23.8,14:49,23.1,14:50,12.3,06:59,0.0,00:00,12.5,06:58,11.3,00:16,6.9,14:34,354,2.0,1.5

02/08/11,16.1,20,16:55,14.7,06:45,24.2,13:54,1013.79,19:13,1015.65,11:14,0.0,00:00,0.0,18.9,13.7,8.0,15:55,42,20:42,85,06:50,2.79,4.9,24.2,13:54,24.3,13:55,15.1,06:40,0.0,00:00,14.7,06:45,14.8,11:59,7.0,21:09,57,1.0,1.7

03/08/11,14.5,36,17:23,14.9,05:50,24.6,14:46,1012.70,18:44,1015.99,08:34,0.0,00:00,0.0,19.4,17.2,4.8,16:04,50,14:38,79,07:04,3.05,5.8,24.6,14:46,25.4,14:47,15.0,05:50,0.0,00:00,14.9,05:50,14.2,20:01,8.9,00:16,32,0.8,1.9

04/08/11,17.7,16,15:43,14.1,06:20,25.3,15:06,1013.08,18:42,1015.31,08:28,0.0,00:00,0.0,20.2,19.4,8.1,14:12,52,18:20,92,06:55,3.30,9.1,25.3,15:06,26.8,14:55,14.9,06:20,0.0,00:00,14.1,06:20,15.8,14:55,12.5,06:25,36,1.0,2.9

05/08/11,16.1,32,12:52,14.2,06:12,22.2,14:07,1013.89,00:01,1016.36,09:43,0.0,00:00,0.0,18.6,21.6,5.2,13:00,62,15:57,87,06:11,3.30,8.4,22.2,14:07,23.5,14:10,14.8,07:19,0.0,00:00,14.2,06:12,15.4,10:33,12.0,06:03,34,0.9,1.3

06/08/11,16.1,309,11:15,14.3,05:29,22.4,17:12,1014.46,20:02,1016.97,10:38,0.0,00:00,0.0,18.4,19.2,5.5,16:21,55,13:33,92,05:20,2.79,7.9,22.4,17:12,23.3,18:17,15.1,06:09,0.0,00:00,14.3,05:29,14.2,18:12,10.9,10:38,32,1.1,1.3

07/08/11,17.7,342,13:24,12.9,05:47,24.1,14:53,1013.92,19:49,1016.43,09:36,0.0,00:00,0.0,18.4,19.1,6.3,14:06,48,12:45,89,05:36,3.30,9.0,24.1,14:53,24.6,15:48,13.3,05:47,0.0,00:00,12.9,05:47,14.6,15:52,10.7,11:33,11,1.6,1.7
Retrieved from "https://www.cumuluswiki.org/index.php?title=Dayfile.txt&oldid=2810"