5,838
edits
Monthly log files: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
Explaining how to view logs, some further tidying of other parts with cross-references
m (→Correcting any data problems: Added italic note re create missing to end) |
m (Explaining how to view logs, some further tidying of other parts with cross-references) |
||
|
Files are automatically created with names in the form <Month><Year>log.txt (for example, the file for August 2009 is called <tt>Aug09log.txt</tt>); an entry is made to the current month's file every ''ten''
For ''more information'' on these files see in the '''Cumulus help''' file, in the section “Data log file format”.
==Correcting any data problems==▼
==Viewing monthly log files==
See [[FAQ#I.E2.80.99ve_just_installed_Cumulus.2C_and_it_didn.E2.80.99t_download_all_the_old_data_from_my_weather_station|FAQ: I've just installed Cumulus and it didn't download all the old data from my weather station]], [[FAQ#How_do_I_correct_my_all-time_.28or_monthly.29_records.3F|FAQ: How do I correct my all-time or monthly records]] and [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]].▼
In the '''View''' menu, select ''Data logs'', click '''Load...''' and your data files will be listed. If there is a problem see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. Select the required monthly log and display it. This is a text viewer, and works best when at full screen, but even then you are likely to need to scroll both horizontally and vertically to look at all the figures. If you click on an individual figure the cell will be highlighted, but you cannot edit the figures on a view screen.
==Viewing summary figures for a month==
In the '''View''' menu, select ''This month'', and a summary for the current month will be calculated from the monthly log. Note that many parameters are different to those available as web tags. To view for a different month, select the month and year required using the options at bottom left, click ''Update Display'' and the revised summary will be calculated. (Alternatively you can select ''This period'' and select a start date and end date then click ''Update Display'' to get the equivalent calculation for part of a month or any other period).
==What can Monthly log files be used for?==
'''After you have edited (or created any missing) Monthly log files, you can:'''
* update the [[Alltime.ini]] by choosing ''all time records'' from the '''Edit''' menu. This is a text editor, and works best when at full screen. Click the ''Help'' button for detailed instructions on how all available monthly logs are used.
* (if the created/edited monthly log
* (
* (from version 1.9.3) update the [[monthlyalltime.ini]] by choosing ''all time records'' from the '''Edit''' menu. This is a text editor, and works best when at full screen. Click the ''Help'' button for detailed instructions on how all relevant monthly logs for all years are used.
'' '''Note''' in each of above, you see the currently stored extremes, and you can load the monthly log(s) and dayfile.txt to view extremes calculated from those figures (as available) and optionally '''Copy''' the logged values (and associated date/time information) into the relevant ini file.''
* create the relevant monthly and/or annual NOAA style report by choosing ''NOAA Monthly Report'' or ''NOAA Annual Report'' from the '''View''' menu, then select the required period using the selectors, click the ''Update Display'' button to see various statistics calculated from processing the monthly log files, and finally press ''Save'' button to store the new or amended report. (Note: You can generate a NOAA report for a period without a monthly log, if data for that period is present on [[dayfile.txt]]. Generation of complete NOAA reports requires information in dayfile.txt, and the monthly logs for period in question).
* use '''create missing''' button in the ''[[dayfile.txt]]'' selection in the '''Edit''' menu; This is a text editor, and works best when at full screen. Cumulus will then look through all monthly log files and create approximate records for any ''missing dates'' in the dayfile if those dates have observations stored in the relevant monthly log. (It creates a file in the Cumulus folder called dayfileeditlog.txt which contains the entries it created). Click the ''Help'' button for more information. In past and current versions (including 1.9.3), ''create missing'' will not affect any records where some of the fields for a particular date are missing in the dayfile.txt (see workaround below).
▲==Correcting any data problems==
[[FAQ#How_do_I_correct_my_all-time_.28or_monthly.29_records.3F|See FAQ: How do I correct my all-time, annual, or monthly records]].
===Editing within Cumulus===
===Manipulation outside Cumulus===
Any log files for ''previous'' months can be edited (outside Cumulus) with Cumulus running,
'''Rules''' -- Each line must contain the fields in correct sequence (since new versions/builds can add to number of fields, Cumulus will accept lines of various lengths without the more recent fields at the end). Nulls ',,' are not allowed in the line, so if you do not know the value for a particular field within the line, then type in an extreme with opposite value (e.g. -999.9 for a signed decimal maximum, and 999.9 for a decimal minimum - assuming your system decimal format is using full stops).
'''Important''':
* Times in these files are in the form ''hh:mm'' using the 24 hour clock and local time (system time). Note that the fields related to solar data will only contain valid data if your station has the appropriate sensor(s).▼
▲'''Important''': The date format uses two digits for the year. Do not edit this file using Excel or similar, as it is likely to change the dates to have four figure years, and then Cumulus will no longer be able to use the file.
* Apart from bearings (stored as integers), most value fields are in format x.y using your system decimal notation. See [[Monthly_log_files#Example_lines_from_the_file|Monthly log files: Example lines from the file]].▼
▲Times in these files are in the form hh:mm using the 24 hour clock and local time (system time). Note that the fields related to solar data will only contain valid data if your station has the appropriate sensor(s).
▲Apart from bearings (stored as integers), most value fields are in format x.y using your system decimal notation. See [[Monthly_log_files#Example_lines_from_the_file|Monthly log files: Example lines from the file]].
===Importing pre-Cumulus data===
▲See [[FAQ#I.E2.80.99ve_just_installed_Cumulus.2C_and_it_didn.E2.80.99t_download_all_the_old_data_from_my_weather_station|FAQ: I've just installed Cumulus and it didn't download all the old data from my weather station]]
Given that monthly log files are used as input for '''updating the all-time, all-monthly (from version 1.9.3), and this year record extremes''', for '''creating missing dayfile.txt entries''', and for '''creating NOAA style reports''', you may have some information for a period before you first starting using Cumulus that you want to include.
There are some issues to consider. For example, wind directions might be reported as cardinal points (north, south etc) and need converting to bearings in degrees (spreadsheet packages provide ''look up'' functionality to do such conversions), and dates and times might need some pre-processing (spreadsheet packages usually have ability to select part of a text string and to cocatenate a number of strings) to convert them to text in the formats mentioned above.
Cumulus has various wind speed, temperature, and rainfall related fields, not directly available from weather station outputs. See, for example ''EasyWeather''
The date that Cumulus first started operating, does ''not'' need to be updated for Cumulus to recognise earlier monthly log files in its ''data'' subfolder. However, the web tag associated with that date is on the default Cumulus ''recordsT.htm'' template, and you might want to amend the associated label wording to indicate it is the date that full Cumulus parameter logging/calculation began, rather than the date the station began operating. If you do choose to edit the start date, stop Cumulus, and amend the ''StartDate='' line in [[cumulus.ini]] within the main Cumulus folder, but (in case you make a mistake) back up everything first!▼
===Using Monthly logs to deal with shorter (or incomplete) dayfile.txt records for particular dates===▼
▲The date that Cumulus first started operating, does ''not'' need to be updated for Cumulus to recognise earlier monthly log files in its ''data'' subfolder. However, the web tag associated with that date is on the default Cumulus ''recordsT.htm'' template, and you might want to amend the associated label wording to indicate it is the date that full Cumulus parameter logging/calculation began, rather than the date the station began operating.
'''WORKAROUND if dates are present in monthly log and dayfile.txt but not all fields for that date exist in dayfile.txt'''
Example: records created by earlier version of Cumulus [to help you, the versions (not builds) at which fields were added are indicated below].
Second example: records imported prior to Cumulus processing them, so some calculated parameters (apparent temperature, heat index, rain rate, wind run) may not have been available from your weather station to insert in the log files.
▲===Using Monthly logs to deal with shorter (or incomplete) dayfile.txt records===
▲'''NOTE if dates are present in monthly log and dayfile.txt but not all fields for that date exist in dayfile.txt (e.g. records created by earlier version of Cumulus:''' ''One method is (not near rollover time) to (1) take a copy of dayfile.txt, (2) delete any days with partial information (e.g. from Cumulus versions that created fewer fields), (3) use ''create missing'' in the [[dayfile.txt]] selection in the '''Edit''' menu option of Cumulus (note whilst datafile.txt normally calculates all parameters like minimums and maximums from very frequent samples of your weather station, the resolution of create missing is limited to the interval between logging records in the monthly log), (4) rename the new dayfile.txt as dayfile(generated).txt and create a new dayfile.txt and (5) use a text editor to merge the required fields from the new dayfile(generated).txt with all other fields from reading only the copy of the original file. (This method preserves the original so you can experiment with different merges and do some cross-checking).''
==List of fields in the file==
and from version 1.9.2:
Here logging is at the default interval, and the delimiters have their default settings for UK systems.
01/10/11,04:40,18.0,75,13.5,5.0,6.7,169,0.0,0.0,1021.4,885.3,24.4,61,5.6,18.0,18.0,0.0,0,0.00,0.00,17.5,0,0.0,158
| |||
