5,838
edits
Calculate Missing Values: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
no edit summary
No edit summary |
mNo edit summary |
||
|
#* Because newer releases calculate more derivatives than older releases, extra fields have been added to the standard log file
=="
This also has two meanings in a Cumulus context:
# If a particular standard log file line has fewer fields than the latest line;
#* '''
# If a particular daily summary log file, either does not have a line for a particular meteorological date, or does not have all fields defined in a line for a particular meteorological date;
#* Please see [[Amending dayfile]] page for full details.
#* '''
If you are using Cumulus MX, there is a download [[Software#Create_Missing| linked from here]] that does both of these. There are also editors within the [[MX_Administrative_Interface#The_Data_Log_Viewing_and_Editing_interface|admin interface]] for manually editing the files on a line by line basis. You can also use the PHP Hypertext Pre-processor (PHP) script specified for Cumulus 1 below, although be aware it was written for a very old PHP version.
===Accurate or Not?===
This Wiki page describes some techniques for calculating and inserting values that are missing from standard log files and from daily summary log file.
Since the derived values this page is discussing are spot values, they have to be calculated from source values measured at the same time. This means that if one of your [[:Category:Ini Files|.ini]] files is missing some fields,
However, the techniques for correcting rogue values described on the [[Correcting_Extremes]] page, can be used for inserting missing values in the daily and longer period extreme records.▼
For the standard log files, all the fields in any one line relate to the same time, therefore for derived values calculated from other fields in the same line, you should have the same value whether it was calculated when that line was originally stored, or calculated afterwards. I say '''should''' because the calculation formula is not always the same for all releases, in particular there are differences between how Cumulus 1 and how MX calculate some derivatives.
For entries in today.ini, month.ini, year.ini, alltime.ini, and monthlyalltime.ini files, you don't have access to the source values used for the original calculation afterwards.
#*Depending on your weather station, Cumulus is able to read values every minute, and consequently update today.ini each minute if an extreme happens,▼
* The original values are calculated as Cumulus is running
#*If Cumulus is set up to only log the readings every half an hour, create missing is only able to see 1/30th of the data, ▼
▲
#* Due to this mismatch, the derived values (averages, highs, lows) this approach can store are much less accurate (hence why getting missing lines from a backup is better)▼
** Obviously, dayfile.txt is updated from today.ini, so it is just as accurate
* Any "Calculate Missing" operation, done subsequently, does not have access to old data, it can only look in the spot values that have been logged.
▲
▲
=Derived spot values=
Consequently, those people who have used Cumulus for a while may have files that include some lines with fewer fields stored than their latest lines.
=Why do "
Most functionality in Cumulus is concerned with current data or extremes/sums that are derived for a hour, a day, or longer, periods. For these contexts, you might encounter an odd rogue value that needs to be corrected as described on the [[Correcting_Extremes]] page. You are unlikely to worry about missing past values.
=How to do "
As mentioned earlier, there are a number of options, here are the detailed instructions for each option.
<div style="background: LemonChiffon;padding:5px; margin:2px;">
[[File:Crystal Clear info.png|40px]] This document was written for the (legacy) Cumulus 1 software. It has been updated to cover MX, but that was for a MX release that is no longer latest!
When this section was written, the number of lines shown was fixed at a maximum of 10; later releases have given the option to display different numbers of lines, and there may be other changes still to be documented here.
</div>
In the [[MX_Administrative_Interface#The_Data_Log_Viewing_and_Editing_interface|MX admin interface]] go to the '''Data Logs''' menu tab, and select the ''Data Logs'' page.
==Lack of editor in Cumulus 1==
Cumulus 1 provides a [[Cumulus_Screenshots#View_Menu|viewer]] for the data logs
On the '''View''' menu, select ''''Data Logs''', then enter the file name you want to view and load it. You can scroll left to right through the fields, and you can scroll up and down through the lines. The viewer shows a header row so you know which field is which. You cannot do any editing.
If you find that this viewer cannot load a data log, it is probably because you ignored the '''read me''' that is part of the Cumulus 1 installation procedure, see [[FAQ#I_can.E2.80.99t_find_my_data_files.21|FAQ: I can't find my data files]]. If the displayed headings do not match the data shown, you have not read the caution on the screen, which says the viewer is only for standard data logs, not extra sensor data logs , nor the daily summary log.
* Ensure no blank lines are introduced by your editing.
*Ensure that all lines continue to have date and time information at the start of the line, and that the format of that identifier is not changed (same sequence, same character(s) separating each element of date, and a colon separating hours and minutes, and that the time does not have a seconds element added.
▲Since the derived values this page is discussing are spot values, they have to be calculated from source values measured at the same time. This means that if one of your [[:Category:Ini Files|.ini]] files is missing some fields, they cannot be calculated from other fields. This applies to any missing extreme records for today, this month, this year, monthly-all-time, or all-time.
▲However, the techniques for correcting rogue values described on the [[Correcting_Extremes]] page, can be used for inserting missing values in the daily and longer period extreme records.
| |||
