Migrating from Cumulus 1 to MX: Difference between revisions
m (Update category) |
m (→Configuration file: add links) |
||
(5 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
Like other longer pages in this Wiki, the content is divided into sections, if you are interested in a particular issue, find it in the table of contents, rather than reading whole page! |
|||
=Introduction= |
=Introduction= |
||
This page was inspired by this [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17749 update from Cumulus 1] forum topic. That post was made in January 2020, and therefore |
This page was inspired by this [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17749 update from Cumulus 1] forum topic. That post was made in January 2020, and therefore relates to Version 3.3.0, which was the MX release that was in use at the time. |
||
<div style="background: LemonChiffon;padding:5px; margin:2px;"> |
<div style="background: LemonChiffon;padding:5px; margin:2px;"> |
||
[[File:Crystal Clear info.png|40px]] This page was last partially updated for the MX release in |
[[File:Crystal Clear info.png|40px]] This page was last partially updated for the MX release available in June 2021; that is no longer latest! |
||
Appeal to contributors: Please |
Appeal to contributors: Please work out any updates needed for this page, so it is kept correct for more recent releases! |
||
</div> |
</div> |
||
Line 12: | Line 14: | ||
You might want to read the related page at [[My_migration_from_C1_to_MX]] which describes how this worked in practice for release version 3.4.5 (build 3069). |
You might want to read the related page at [[My_migration_from_C1_to_MX]] which describes how this worked in practice for release version 3.4.5 (build 3069). |
||
=Should I migrate to MX or not?= |
|||
==Should I migrate to MX or not?== |
|||
The legacy Cumulus will never be updated, and as more people migrate to MX there will be fewer people to help you if the legacy cumulus software gives you a problem. MX has a major advantage it runs on more devices, and for example a Raspberry Pi computer is far cheaper to leave on all the time to run MX (knowing it will never be rebooted by Windows updates). The current developer of MX offers support, and other people are also gaining expertise, so if you have a problem you will get help. |
The legacy Cumulus will never be updated, and as more people migrate to MX there will be fewer people to help you if the legacy cumulus software gives you a problem. MX has a major advantage it runs on more devices, and for example a Raspberry Pi computer is far cheaper to leave on all the time to run MX (knowing it will never be rebooted by Windows updates). The current developer of MX offers support, and other people are also gaining expertise, so if you have a problem you will get help. |
||
Line 45: | Line 46: | ||
=How to install MX= |
=How to install MX= |
||
The basic instructions for installing MX: |
The basic instructions for installing MX (see [[MX on Linux]], [[MX on Windows OS]], or [[Raspberry Pi Image]], as appropriate for fuller instructions): |
||
#Download the MX distribution (from [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version) |
#Download the MX distribution (from [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version) |
||
#Unzip into the location where you are going to install MX. |
#Unzip into the location where you are going to install MX. |
||
Alternatively, follow instructions at [[Raspberry Pi Image]] which describes a simpler way to install MX (assuming you have another device linked to your RPi to do downloads and micro-SD card preparation). |
|||
=Migrating from legacy Cumulus to a recent build of MX= |
|||
<big>The text that follows mostly relates to version 3.3.0, it probably needs to be updated, as later releases of MX have massively changed. The newer releases are more fussy about formats used in files, and migration from the legacy Cumulus software is harder</big> |
|||
If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and is the subject of the rest of this article. |
|||
== 'Engine' and 'Admin Interface' for MX == |
== 'Engine' and 'Admin Interface' for MX == |
||
Line 67: | Line 56: | ||
MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a [[MXdiags_folder|diagnostic log]], but many people run MX on a device that has no monitor and so the terminal output (if any) is not monitored. |
MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a [[MXdiags_folder|diagnostic log]], but many people run MX on a device that has no monitor and so the terminal output (if any) is not monitored. |
||
When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment). |
When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment). (You can also run MX as a service, that has different ways to start and stop, discussed in the links given in previous section). |
||
The separate [[MX Administrative Interface|admin interface]] is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network. |
The separate [[MX Administrative Interface|admin interface]](unfortunately this is called various names in the support forum, including "user interface", "dashboard interface", and "admin interface") is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network. |
||
If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality: |
|||
# Windows Defender has to be told to allow all for Cumulus MX. |
# Windows Defender has to be told to allow all for Cumulus MX. |
||
# Typically for other security packages, select "CumulusMX.exe" then right click and select "Change File Rating to Trusted" |
# Typically for other security packages, select "CumulusMX.exe" then right click and select "Change File Rating to Trusted" |
||
Line 75: | Line 66: | ||
# Using "command prompt as administrator" type <tt>netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users</tt>, the response "URL reservation successfully added" means it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way. |
# Using "command prompt as administrator" type <tt>netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users</tt>, the response "URL reservation successfully added" means it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way. |
||
The default URL if the browser is on the same machine as MX is http://localhost:8998/. Newer releases of MX will tell you an IPv4 address to use, such as by typing "http://192.168.1.64:8998/". |
|||
If you are using the browser on a different device on your local network to the device running MX, you cannot use that local host shortcut. Instead you specify a IPv4 address, that is listed in your router (might be called a hub) for the device running MX, this IPv4 address will look like '192.168.y.z' (where y and z are numbers that vary between implementations). For example you might acces the admin interface by typing "http://192.168.1.64:8998/". |
|||
Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device. |
Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device. |
||
Line 83: | Line 72: | ||
For security reasons, the admin interface should not be accessible via the public internet. |
For security reasons, the admin interface should not be accessible via the public internet. |
||
=Migrating existing data files from legacy Cumulus to a recent build of MX= |
|||
==Configuration file== |
|||
If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and there is quite a bit to read, taking up most of the rest of this Wiki page. The Cumulus configuration files for Cumulus 1 and for MX share the same name, but their content is very different, so these are discussed at length in subsections below. |
|||
MX's [[Cumulus.ini]] has different content to the legacy [[Cumulus.ini_(Cumulus_1)|'''cumulus.ini''']]. |
|||
Obviously, you will want to copy ALL (except weather diary which uses a different file) the files in the [[#"data" folder]], from any old installation into the new installation, but migrating these is not always easy, here is a quick summary of the potential issues (all discussed further in various subsections later): |
|||
Cumulus 1 can recognise, in some circumstances, "cumulus1.ini", and other variants, not just "Cumulus.ini". MX only recognises "Cumulus.ini". |
|||
* '''When you first run MX, the [[#Start date|Start Date]] is recorded in the configuration file''' |
|||
** In the legacy software, this date is purely used as something to appear on one of the [[Customised_templates#The_Standard_Templates_for_Cumulus_1|sample web templates]] |
|||
** In Cumulus MX, this date controls what data is read from [[Monthly log files]], such as [[Standard log files]], and [[Extra Sensor Files]], any lines in those files with earlier dates are ignored by CumulusMX.exe, ExportToMySQL.exe, and other utilities. |
|||
** You can edit this [[Cumulus.ini#Data_Logging|'''StartDate=xxxxx''']] parameter, in the admin interface select ''Station Settings → Common Options → Advanced Options → Records Began date'' |
|||
* Be aware your new installation has to use the same "locale" as the old installation, or MX will struggle as the locale affects how new lines are stored, and how MX expects old lines to have been stored the same way. |
|||
* If your old installation, is on a different operating system to the new installation, remember that Microsoft Windows uses different line terminators to all other operating systems, although MX should cope with mixed line terminators, any third party routines reading your data files will probably not accept a line terminator change. See [[MX on Linux]] page for more information on this. |
|||
* Cumulus MX is much more fussy than Cumulus 1 about every line in any data file using exactly the same locale formatting: |
|||
** Cumulus 1 is able to accept any character (other than the list separator, and space) being used to separate hour and minutes in time-stamps, MX expects a colon ":" |
|||
** [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]] |
|||
** [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files, and how MX prefers decimal points to decimal commas |
|||
==File names== |
|||
The old approach for migration, was to copy across your existing file, and let MX ignore all the parameters that do not apply to it. To add the parameters that MX does need, you would then go to the [[MX_Administrative_Interface#Changing_Settings]] pages and work steadily through ALL the options. For any parameters that were not set by the admin interface (and there were many "read-only" parameters in early releases of MX), one assumed these also existed in Cumulus 1 and so were already in your file. |
|||
Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that Wiki pages always change first letter to a capital, so do check carefully if that wiki page is describing a file name that must be all lower-case. |
|||
At releases like 3.3.0 and 3.6.0, amongst others, there were changes to "read-only" parameters for MX. These could not be adjusted through the admin interface, and needed to be entered manually. To keep this page simple, it cannot list the "read-only" parameters that you might need to add for migrating to earlier releases. |
|||
==Line terminators== |
|||
It appears that from release 3.10.1, all settings are now initialised via the admin interface (and none are "read-only"), so if you are using that release or later, it may be better to ignore your Cumulus 1 configuration file, and to let MX create a configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly. |
|||
If you are not running MX on a Microsoft Windows computer, ideally you should change the characters appearing at the end of every line in every file you move from Cumulus 1. |
|||
===Start date=== |
|||
I say ideally, because although Microsoft is fussy, and determined to be different by insisting files match its choice, most other operating systems can tolerate different line terminators. This means that MX will generally tolerate files using a mixture of line terminators. However, if you are using third-party routines, please be aware that these are often written to expect a particular line terminator, and they may give unexpected results when used with files that do not match that expectation! |
|||
When you first run any Cumulus software (whatever flavour) a parameter is added to the configuration file that documents the date Cumulus was first used. |
|||
With Cumulus 1 (or MX) on Windows, each line in every file ends with both carriage return and line feed. |
|||
For Cumulus 1, this parameter appeared in two places on the example web template for all-time records, but was otherwise ignored. Thus if you decided to import into your data logs readings from before you first ran Cumulus, the legacy software would be able to find those earlier records. |
|||
With MX on a Mac, each line should end with just a Carriage Return. |
|||
'''CumulusMX.exe''' uses this parameter to determine the first standard log file to start reading from, it will ignore any log files for earlier dates. Thus if you were to migrate your Cumulus 1 configuration file into MX, you should check the "StartDate=" line in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time. If you need to make any edit, ensure you stick to exactly the same date format. |
|||
For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed. |
|||
From release 3.10.1, MX allows you to edit this start date within the admin interface. It is "hidden" as an "advanced" setting, with a strict danger warning, but it is there! |
|||
To change end of line characters, run each file through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its '''Edit''' menu, choose '''EOL conversion'''. On a Linux system like a Raspberry Pi computer, please see [[MX_on_Linux#Changing_line_terminators]]. |
|||
===Uploading to web site=== |
|||
If you use PHP Hypertext Pre-processor anywhere on your installation, normally that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" function to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on! |
|||
Many settings have changed in this area for MX, and they vary depending on which release you are installing. No attempt is made to explain further here. |
|||
==Configuration file== |
|||
=== Station connections=== |
|||
MX's [[Cumulus.ini]] has different content to the legacy [[Cumulus.ini_(Cumulus_1)|'''cumulus.ini''']]. From release 3.12.0,many more new settings were added, more settings were removed, and some parameters in the file were renamed. The first cross reference, in listing all the parameters, makes it clear which settings were introduced in the legacy software, and which MX release added new settings, this should help you to grasp how many new settings exist! |
|||
If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in Cumulus.ini_(Cumulus_1) in the station section as a parameter in the format '''Port=n'''. |
|||
'''The [[MX_Administrative_Interface#Changing_Settings|settings pages]] in MX, work differently to the [[Cumulus_Screenshots#Configuration_Menu_Screens|settings screens]] in Cumulus 1. You need to click ''Save'' and (in many cases) stop and restart MX before changed settings take effect.''' |
|||
In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings page, but within [[Cumulus.ini]] in the station section the parameter is in the format '''Comport=tttttttt'''. |
|||
Consequently, if you are migrating from the legacy software to MX now, it is best to rename your old '''cumulus.ini'' file so it is is not seen by MX. Let MX create a new configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly. |
|||
If your old parameter had a value of '''3''', and you are still using Windows, the new setting would have value of '''COM3'''. |
|||
===Configuration file naming restrictions=== |
|||
A typical parameter value for other devices might be "/dev/ttyUSB0". |
|||
Cumulus 1 can recognise, in some circumstances, "cumulus1.ini", and other variants, not just "cumulus.ini". |
|||
===RG11 Rain gauge=== |
|||
MX only recognises "Cumulus.ini". The Windows operating system is not case sensitive for file names. All other operating systems require the first letter of the filename to be a capital, and the remainder to be lower case as shown. |
|||
If you use a RG11 rain gauge: |
|||
*Replace: '''RG11port=n''' and '''RG11port2 =n''' (Cumulus 1) where n is a number, by '''RG11portName=xxxx''' and '''RG11portName2=yyyy''' (Cumulus MX on Windows) where the value is a string with values as per previous paragraph depending on device on which Cumulus MX is running. |
|||
===Historical evolution of configuration file=== |
|||
From release 3.10.1, there appear to be other new parameters, not yet documented. |
|||
The oldest approach for migration from the legacy software, was to copy across to MX your existing configuration file, and let MX ignore all the parameters that do not apply to it. This worked because the early MX releases had few new parameters, and mostly used the majority of the old parameters. |
|||
==Strings.ini== |
|||
For any parameters that were not set by the admin interface (and there were many "read-only" parameters in early releases of MX), one assumed these also existed in Cumulus 1 (refer to [[Cumulus.ini (Cumulus 1)]]) and so were already in your file. |
|||
This is another configuration file, but it is an optional one. If you have not created a [[Strings.ini|strings.ini]] file in your (leagacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section. |
|||
To add the few new parameters that MX did need (see [[Cumulus.ini (MX 3.0.0 to 3.7.0)]] page), you would then go to the [[MX_Administrative_Interface#Changing_Settings]] pages and work steadily through ALL the options. |
|||
The contents of the [[Samplestring.ini|samplestring.ini]] file you get in your MX release distribution varies depending on the release you have downloaded. Check your existing '''strings.ini''' file against the ''samplestring.ini'' file in the MX distribution you have. If the attribute names (left hand side of the equals sign) match for the parameters you selected to include in your '''strings.ini''', then you can reuse your existing file. If your file includes attributes that are no longer in the MX ''samplesting.ini'' file, then you will need to edit your '''strings.ini''' file that is placed in the folder containing CumulusMX.exe. |
|||
Retaining your old settings as far as possible simplified any migration of data files from your legacy installation into MX, because it ensured you kept to same unit selections, and same extra sensor selections. |
|||
Please remember that the Microsoft Windows Operating System is case insensitive for file names, if you install MX on a Windows PC, then "Strings.ini", "STRINGS.INI", and "strings.ini" are all treated as the same file by MX. If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini". |
|||
At releases like 3.3.0 and 3.6.0, amongst others, there were changes to "read-only" parameters for MX. These could not be adjusted through the admin interface, and needed to be entered manually into the '''Cumulus.ini''' file, as instructed on [[Cumulus.ini (MX 3.0.0 to 3.7.0)]] page. |
|||
For those of you who are more technical: |
|||
*Note that files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems like all Linux variants including Raspberry Pi Operating System). Apple Mac are again different in using just Carriage Return. |
|||
*This should not cause any problems for your "strings.ini" file as MX does not care if there appear to be some extra blank lines (because the carriage return may be treated as end one line and the line feed as ending a separate blank line on non-Windows devices). |
|||
===Subsequent evolution of configuration file=== |
|||
==NOAA style reports== |
|||
Substantial changes to settings available were made from release 3.8.0 onwards. The content of the '''Cumulus.ini''' file changed drastically, with much deviation from the configuration file used in earlier releases (including for the legacy software), hence [[Cumulus.ini]] is a new page describing the settings, and the parameters appearing in the configuration file, applicable from that release. |
|||
The generation of reports is an optional feature, if you have never used it your (legacy) Cumulus Reports folder will be empty, then you have no files to move to your MX installation, and you should skip the rest of this sub-section. |
|||
More changes were made in 3.10.0, and subsequent releases, as settings that previously had to be made directly in file were gradually moved to be advanced settings controlled in the admin interface. In addition, these releases, have added many new settings never encountered in your legacy software. |
|||
Please see [[Reports_folder]] for full information. Cumulus software creates reports, it does not edit existing reports, so migration is simple. Just copy the contents of the '''Reports''' folder in your original Cumulus installation into the folder in the new installation. |
|||
For example the changes include the (advanced) ability to change the number of decimal places for storing derived values like daily rainfall, sea level pressure, and wind speeds. |
|||
For those of you who are more technical: |
|||
* files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems). Apple Mac are again different in using just Carriage Return. This should not cause any problems. |
|||
* files can be encoded (how individual characters are represented by binary codes) in different ways. There is more about encoding at [[Reports_folder#Encoding]], the relevance here is that if your MX settings and Cumulus 1 settings use different encodings you may find some characters (e.g. degree symbol) do not appear correctly when viewing some of your reports. |
|||
Changes to the [[New Default Web Site Information|Default Web Site]] (diverging away from the [[Customised templates]] approach in the legacy software) and implementing the option to use (as was available in the legacy software, and the earliest MX beta, but then removed) a copy action instead of a file transfer action for [[Your Own Server]], has led to many,many new settings. Thus the advice has become abandon your old legacy software cumulus.ini file when you move to MX. |
|||
=="data" folder== |
|||
The contents of your Cumulus 1 '''data''' folder (log files ending with extension '''.ini''' or '''.txt''') can NORMALLY be read by MX (but see points listed below). |
|||
===Settings=== |
|||
However, if you are not running MX on a Microsoft Windows computer, ideally you should change the characters appearing at the end of every line in any file you move from Cumulus 1. To do this, run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its '''Edit''' menu, choose '''EOL conversion'''. Do remember that with Cumulus 1 on Windows, each line in every file ends with both carriage return and line feed. If you are moving to MX on a Mac, each line should end with just a Carriage Return. For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed. |
|||
The settings pages in Cumulus 1 and MX work differently: |
|||
I say ideally, because although Microsoft is fussy, and determined to be different, most other operating systems can tolerate different line terminators. If you use PHP Hypertest Pre-processor anywhere on your installation, you will discover that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on! |
|||
* for Cumulus 1 you choose to save changes by clicking OK, |
|||
* for MX changes are only saved when you click a '''Save''' button if one is provided. |
|||
* The alarms page works slightly differently, with an "Update alarms" button. |
|||
* If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved to configuration file when you move to next field/line, and acted on when you next restart MX. |
|||
Be aware that you should restart MX after changing many settings. This is because many settings are only written to configuration file, not yet to internal code, and such settings are only read from configuration file when MX starts. The developer is changing the code, so that in more cases edits on the settings pages do update the settings used by the code, as well as the file, but at time of writing there is no list of which settings work like that, and which require a restart. |
|||
===Weather Diary=== |
|||
===Start date=== |
|||
As explained on [[Weather_Diary|weather diary]] page, the Cumulus 1 weather diary uses a different file to MX, and indeed takes a very different model for how to store the information. |
|||
When you first run any Cumulus software (whatever flavour) a parameter is added to the configuration file that documents the date Cumulus was first used. |
|||
Therefore don't copy [[Log.xml]] file into the data folder of your MX installation. You will need to manually use the [[Weather_Diary#How_to_edit_the_contents|admin interface Edit menu]] to access the new diary and add your previous entries one by one. |
|||
For Cumulus 1, this parameter appeared in two places on the example web template for all-time records, but was otherwise ignored. Thus if you decided to import into your data logs readings from before you first ran Cumulus, the legacy software would be able to find those earlier records, without you needing to change the date that appeared on the web page. |
|||
Please note the editing interface has been changed from release 3.10.1, but the page linked to above may not have been updated. A major change in the upgrade relates to handling of time-zones. |
|||
'''CumulusMX.exe''' uses this parameter to determine the first standard data file to start reading from, it will ignore any data files for earlier dates, it will also ignore any lines with an earlier date in the first data file. Thus if you were to migrate your Cumulus 1 configuration file into MX, you should check the [[Cumulus.ini#Data_Logging|'''StartDate=xxxxx''']] parameter in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time. If you need to make any edit, ensure you stick to exactly the same date format. |
|||
Please also note that the Cumulus 1 weather diary permitted multiple entries to be stored for an individual day, but the MX implementation only permits a single entry per day. Also the configuration file defines a time when the entries stop applying that may not match the rollover time for other weather measurements. |
|||
From release 3.10.1, MX allows you to edit this start date within the admin interface, just select ''Station Settings → Common Options → Advanced Options → Records Began date''. It is "hidden" as an "advanced" setting, with a strict danger warning, because of the importance of sticking to exactly the right date format when you edit it! |
|||
===dayfile.txt=== |
|||
=== Station connections=== |
|||
''You can skip this subsection if your weather station connects to MX either by USB or by wireless connection''. |
|||
====date format issue==== |
|||
This subsection is relevant if your station connects by a serial connection and MX needs to be told which port. It also applies if the serial connection is converted to USB. |
|||
Cumulus 1 allows the date that appears in the first field of each line to use any character (except space) to separate the day of the month, from the month, and from the last two digits of the year. Consequently "29/03/88", "29-03-88", and "29.03.88" are all acceptable in the legacy software (and indeed in some early MX releases). |
|||
If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in [[Cumulus.ini_(Cumulus_1)]] in the station section as a parameter in the format '''Port=n'''. |
|||
For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file. |
|||
*See [[MX_on_Windows_OS#Parameter_for_changing_Locale]] for how to specify locale if you are running on Microsoft Windows (by default the locale settings are taken from the standard Windows settings application or Control Panel). |
|||
*If you are running MX in a Linux operating system, the locale parameter can be used, but the default locale is determined by the mono-complete package, and that in turn depends on your device settings. |
|||
In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings page, but within [[Cumulus.ini]] in the station section the parameter is in the format '''Comport=tttttttt'''. |
|||
So when you migrate "dayfile.txt", run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows. That editor has a "Search" menu, with a '''Replace...''' option within it. In that option, there is a "Find what" box, a "Replace with" box, and a "Regular expression" selection. |
|||
If your old parameter had a value of '''3''', and you are still using Windows, the new setting would have value of '''COM3''', i.e. serial port 3 now requires a "COM" prefix. |
|||
Suppose your latest lines use either "." or "-" as separator, and that is what MX expects, but you have some older lines using "/" as date separator. Since you know "/" does not appear anywhere but in the dates with an older format, putting "/" in '''find what''' box and either "-" or "." (as required) in '''replace with''' box will sort you out after you hit "Replace all". |
|||
A typical parameter value for other serial connecting devices might be "/dev/ttyUSB3" where the final digit will change depending on the new connection. You can search for posts on the support forum that talk about how to find out what connection is being used, depending on what hardware you are using. |
|||
It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select '''Regular expression''' and then set '''Find what''' to "^([\d]{2})-([\d]{2})-" and '''Replace with''' to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it. |
|||
=== |
===RG11 Rain gauge=== |
||
If you use a RG11 rain gauge: |
|||
The file contains time-stamps following any value that represents a highest or lowest in the day. These must be expressed as hour and minute, and normally a colon (":") is used as a separator, but again the legacy Cumulus does not actually insist on that while MX does insist that are time-stamps are in "HH:mm" format. |
|||
*Replace: '''RG11port=n''' and '''RG11port2 =n''' (Cumulus 1) where n is a number, |
|||
* With: '''RG11portName=xxxx''' and '''RG11portName2=yyyy''' (Cumulus MX on Windows) |
|||
The new value is a string with values as per previous paragraph depending on device on which Cumulus MX is running. |
|||
=== |
===Other Cumulus.ini parameters=== |
||
Lots of parameters in [[Cumulus.ini]] are being changed as MX is developed. So much had changed by the time that MX releases reached 3.12.0, the first run of that particular release created a brand new '''Cumulus.ini''' discarding the old file. If you migrate from Cumulus 1 to MX without stepping through MX releases, you may have several issues, but without knowing which MX release you have selected, all I can suggest is that you work through all settings found in the Settings page of the MX release you choose. Although you could compare [[Cumulus.ini (Cumulus 1)]] and [[Cumulus.ini]] pages, there is no guarantee that either is totally accurate! |
|||
If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all lines for mX, and must match what is defined in the locale used. |
|||
== |
==Strings.ini== |
||
This is another configuration file, but it is an optional one. Please remember that the Microsoft Windows Operating System is case insensitive for file names, so "Strings.ini", "STRINGS.INI", and "strings.ini" are all treated as the same file by any Cumulus software. |
|||
These should migrate without any problems. You will find that MX creates at least one file that was not in the older Cumulus ([[Correcting_Extremes#How_do_I_correct_my_monthly_all-time_records.3F|monthlyalltimelog.txt]]). |
|||
If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini". |
|||
If you have not created a [[Strings.ini|strings.ini]] file in your (leagacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section. |
|||
===the .ini files=== |
|||
The contents of the [[Samplestring.ini|samplestring.ini]] file you get in your MX release distribution varies depending on the release you have downloaded. |
|||
When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. He wrote the software originally just for his own benefit and did not need to worry about time zones. Subsequently as he enhanced his software to make it usable by others, he faced many issues on how to cope with different time zones, and different weather stations having different sensors. In Cumulus 1, he basically focussed on compatibility by keeping to his original design for the data log files (both those ending in '''.ini''' and those ending in '''.txt''', and only adding extra fields to the end. |
|||
Check your existing '''strings.ini''' file against the ''samplestring.ini'' file in the MX distribution you have. If the attribute names (left hand side of the equals sign) match for the parameters you selected to include in your '''strings.ini''', then you can reuse your existing file. If your file includes attributes that are no longer in the MX ''samplesting.ini'' file, then you will need to edit your '''strings.ini''' file that is placed in the folder containing CumulusMX.exe. You may also need to add new items into your '''strings.ini''' based on new content in ''samplestring.ini''. |
|||
When Steve Loft took a new look at the data log files for Cumulus 2, he started with a new design, the principal change was that he decided to use UTC for all fields in them that reference dates and times. Steve struggled with Cumulus 2 largely because he was (at that time) not familiar with the C# language he was later to use for MX. It is fair to say that using UTC was also a major factor in his failure to get Cumulus 2 to provide the functionality he had offered in the original Cumulus. |
|||
==NOAA style reports== |
|||
When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, and he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible]. However those MX beta builds were designed to be forward compatible in that all date formats used by Cumulus 1 could be read, although MX would when rewriting any date change them to the new format. |
|||
The generation of reports is an optional feature, if you have never used it your (legacy) Cumulus Reports folder will be empty, then you have no files to move to your MX installation, and you should skip the rest of this sub-section. |
|||
For the '''.ini''' files, Steve Loft did make one change in his beta MX, all values now had to use decimal points in these files, and decimal commas were no longer valid. Thus to migrate to earlier MX releases, if you had used decimal commas, you needed to edit each ".ini" file and change those commas to full stops. Equally, if you had any times not using a ":" as separator, this had to be rectified. |
|||
Please see [[Reports_folder]] for full information. Cumulus software creates reports, it does not edit existing reports, so migration is fairly simple. Just copy the contents of the '''Reports''' folder in your original Cumulus installation into the folder in the new installation. Of course, nothing is totally simple, the encoding default changes between Cumulus 1 and MX, and once again we might need to consider end of line characters! |
|||
Apart from that, in all versions from 3.0.0 to 3.5.y, there was forward compatibility, and MX could read any file created by the original (legacy) software. '''MX releases from version 3.0.0 to 3.5.4''' can read all the date/time entries in any file whether in the Cumulus 1 format or in the MX format. MX will change any dates to the ISO format only when it needs to update that particular date/time. Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes. |
|||
For those of you who are more technical: |
|||
It appears that releases at versions 3.6.x onwards have become more fussy about existing content in these files due to changes in the source code. However, the developer has not documented whether files from Cumulus 1 can still be read, forward compatibility has only been guaranteed with earlier MX releases. That said, some people are successfully migrating from Cumulus 1 to the newer MX releases, so it works for at least some old files. |
|||
* files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems). Apple Mac are again different in using just Carriage Return. This should not cause any problems. |
|||
* files can be encoded (how individual characters are represented by binary codes) by Cumulus in two different ways. There is more about encoding at [[Reports_folder#Encoding]], the relevance here is that if your MX settings and Cumulus 1 settings use different encodings (as they will if you let this default) you may find some characters (e.g. degree symbol) do not appear correctly when viewing some of your reports. |
|||
=="data" folder== |
|||
=General points= |
|||
You should copy all files in the [[data folder]] from your legacy Cumulus installation to your MX installation if you want to be able to see past data, but as mentioned earlier there are some complications: |
|||
==File names== |
|||
* See [[#Start date]] earlier on this page for a change that you may need to make to ensure MX accepts data in any old [[Standard log files]] and [[Extra Sensor Files]] |
|||
* See [[#Weather Diary]] later on this page, and [[Weather Diary]] page; Cumulus does not provide any utility to convert the old legacy format to the new MX format. |
|||
* See [[#Extreme Record files]] later on this page, and [[:Category:Ini Files]], for information on differences between the legacy files and the MX files |
|||
* See [[#dayfile.txt]] later still on this page, and [[Amending dayfile]] page, for how you can ensure MX can read your existing dayfile.txt file |
|||
* See [[#Line terminators]] earlier on this page for some technical issues if MX is running on a different computer to your Cumulus 1 installation. |
|||
===Standard log files and other log files=== |
|||
Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that wiki pages change first letter to a capital even when a file that must be all lower-case is being described. |
|||
The [[Speciallog.txt]] file optionally used by the legacy Cumulus 1, is not used by MX. |
|||
==web server== |
|||
The [[Standard_log_files]] and any (optional) [[Extra Sensor Files]], from the legacy software can be transferred to your MX installation, but see [[#Start date]] earlier on this page, and [[Calculate Missing Values]] page to read about extra tasks you may need to do. |
|||
Steve Loft's Cumulus 1 came with a set of example template files (designed by Beth Loft) that could generate web pages to upload to your web server. It also produced a set of images representing standard graphs that could be uploaded to your web server and shown on the default "trends.htm" page, a set of images representing wind speed and direction that could be shown on the default "gauges.htm" page, and a moon image that was shown on one web page. Early MX releases included a replacement set of template files that could generate web pages that (apart from trends.htm and gauges.htm) seemed similar to the legacy pages. However, the earliest release did not generate any images at all, the generation of a moon image was added from release 3.5.0 (build 3071) onwards. |
|||
MX extends these [[Monthly log files]] by adding [[Air_Link_Log.txt|another optional set of files]]. |
|||
Mark Crossley's Steel Series is used for the MX gauges page, replacing the Cumulus 1 gauges that were based on'''Web Dashboard Components for FreeWX and FreeWX-Wi''' (no longer available) and mixed images generated by Cumulus with plots drawn using JavaScript routines (before "Canvas" functionality). |
|||
===Weather Diary=== |
|||
MX beta introduced use of a set of .json files to convey the data for drawing charts to your web server, and in release 3.10.1 further .json files were added to support default web pages on your web server. The "legacy web pages" were also available in that release modified to use use the new .json files (instead of web templates), but preserving the same look as Beth Loft's web pages. |
|||
As explained on [[Weather_Diary|weather diary]] page, the Cumulus 1 weather diary uses a different file to MX, and indeed takes a very different model for how to store the information. |
|||
Whatever release of MX you are installing, the files needed for your web server are in "CumulusMX/webfiles" and its sub-folders. The files that will be automatically uploaded to your web server are created in "CumulusMX/web" folder, depending on the release you are installing, which files are generated and which are uploaded will be determined by settings, later releases in general giving you more control over these files than earlier releases. You can't control the content of the various files, but from 3.10.1 you can individually select which are generated and which are uploaded. |
|||
Therefore don't copy [[Log.xml]] file into the data folder of your MX installation. You will need to manually use the [[Weather_Diary#How_to_edit_the_contents|admin interface Edit menu]] to access the new diary and add your previous entries one by one. |
|||
From release 3.10.1, the new default web pages have a totally new look (designed by Neil Thomas), and include responsive code allowing them to look better on wide screens and on narrow screens (such as those on smart mobile phones). |
|||
Please note the editing interface has been changed from release 3.10.1, but the page linked to above may not have been updated. A major change in the upgrade relates to handling of time-zones. |
|||
==Settings== |
|||
Please also note that the Cumulus 1 weather diary permitted multiple entries to be stored for an individual day, but the MX implementation only permits a single entry per day. Also the configuration file defines a time when the entries stop applying that may not match the rollover time for other weather measurements. |
|||
The settings in Cumulus 1 and MX work differently, for Cumulus 1 you choose to save changes by clicking OK, for MX changes are only saved when you click a '''Save''' button if one is provided. If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved when you move to next field/line. |
|||
== |
==Extreme Record files== |
||
The [[Correcting_Extremes]] page explains how extreme records are stored in the [[:Category:Ini Files]] discussed below, with the changes to all-time records being tracked in [[Alltimelog.txt]]. |
|||
Any non-technical person can skip this sub-section! |
|||
You will find that MX creates at least one file that was not in the older Cumulus ([[Correcting_Extremes#How_do_I_correct_my_monthly_all-time_records.3F|monthlyalltimelog.txt]]). |
|||
*If you have been using Cumulus 1, and you decided to customise your web site: |
|||
**you may already be using one or more of highcharts, jQuery, bootstrap, and other library software |
|||
** and you may have selected to load latest versions for maximum functionality. |
|||
*You might be enhancing what is provided with Cumulus 1, |
|||
*or you might use your web site for other purposes (not just Cumulus). |
|||
===the .ini files=== |
|||
*If so, you may find some Cumulus MX web pages do not work correctly. |
|||
**This is because Cumulus MX uses obsolete versions of library software and the way it is coded means MX is not compatible with current versions. |
|||
**For Cumulus MX to work you need to ensure the versions of such software loaded for your web pages, are not used by your browser when viewing MX admin interface or MX web pages. |
|||
Once again, the simple instruction is to copy these from your old installation to your new installation, to ensure your extreme record history is maintained. Once again, differences between Cumulus 1 and MX mean that the process may not be that simple, especially if you try to migrate straight to the latest MX release. |
|||
*Equally, ensure that the versions of library software loaded by MX, do not stay loaded when you wish to view any of your web pages that require later versions of these libraries, as your desired functionality might be limited by the library versions MX uses! |
|||
== |
====MX releases up to 3.5.4==== |
||
'''Your Cumulus 1 '''.ini''' files can be read in all release versions from 3.0.0 to 3.5.y.''' Steve Loft did make two changes in his beta MX, these mean your .ini files may look strange while they have some entries made by Cumulus 1 and some made by MX: |
|||
# all newly stored values will use decimal points in these files, (i.e. any decimal commas valid in the legacy software, are not used by MX). |
|||
# all newly stored time-stamps will use the ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times) so new dates have year first, the parts of the date are separated by hyphens, and new times use colon (:) between hour and minutes (i.e. the regional settings that the legacy software adopted have been abandoned by MX) |
|||
* '''MX releases from version 3.0.0 to 3.5.4''' can read all the date/time entries in any file whether in the Cumulus 1 format or in the MX format. |
|||
#Install MX OVER your Cumulus 1 installation (this only works if you want to run MX on same PC as you have been running Cumulus 1) |
|||
* MX will change any dates to the ISO format only when it needs to update that particular date/time. |
|||
#*This means that all your existing files will be available to MX |
|||
* Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes. |
|||
#*This is simplest approach for those who want a simple install |
|||
#*You may need to edit a few files, please see references to files that might need editing later |
|||
#*Because you have Cumulus 1 and MX executables in same location, you would expect it is easy to run either, but as already explained some files can not be read by Cumulus 1 after MX has changed their content |
|||
#Take a copy of your Cumulus 1 installation and store that copy where you can use it if you want to return to Cumulus 1 |
|||
#*In original Cumulus 1 location, delete Cumulus.exe (or whatever your Cumulus 1 executable was called) |
|||
#*Now follow the instructions for first approach, installing MX over the existing files (but ignore the point about running both executables) |
|||
#Install MX in a NEW location (this might be on your PC that was running Cumulus 1 or onto another device such as a Pi) |
|||
#*This approach requires you to manually copy various files from old folders to new location |
|||
#*MX requires all files from "data" and "Reports" folder created by Cumulus 1. |
|||
#*You also need "strings.ini" (if you use that), and "Cumulus.ini_(Cumulus_1)", plus any other tailoring set-up files, batch processes etc. you might use. |
|||
#*This approach is generally easier if you want to be able to go back to running Cumulus 1 |
|||
If you used decimal commas in your Cumulus 1 installation, you will make the migration to MX easier, by an edit of each ".ini" file to change those commas to full stops. |
|||
=== 1: SIMPLE OVERWRITE APPROACH=== |
|||
Equally, if you had any times not using a ":" as separator, you will make the migration to MX easier, by an edit of each ".ini" file to change those characters into colons. |
|||
Unzip the MX download (from [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version) so the folder '''CumulusMX''' is the same folder as that which has "Cumulus.ini" in it. The unzip ensures all the files that need to be in sub-folders go into correct sub-folders. |
|||
====MX releases at versions 3.6.x onwards==== |
|||
Your log files in the [[data folder]] and any NOAA reports you may (they are optional) have created in [[Reports folder]] are available to MX. |
|||
Migration from the legacy Cumulus to MX has become harder as MX has been developed for two reasons: |
|||
# It appears that releases at versions 3.6.x onwards have become more fussy about existing content in these files due to changes in the source code. |
|||
#* Effectively, the format of existing entries is expected to be same as any newly created entries, i.e. matching current locale settings |
|||
# The addition of Feels Like temperature and Canadian Humidity Index (Humidex) to the .ini files |
|||
#* Neither of these was stored by the legacy software or by earlier MX releases |
|||
#* The technique for adding this is described on [[Calculate Missing Values]] page |
|||
Despite this, most people migrating from Cumulus 1 to the newer MX releases, are doing so successfully, so it works for at least some old files. |
|||
====Historical background to dates/times==== |
|||
When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. He wrote the software originally just for his own benefit and did not need to worry about time zones. Subsequently as he enhanced his software to make it usable by others, he faced many issues on how to cope with different time zones, and different weather stations having different sensors. In Cumulus 1, he basically focussed on compatibility by keeping to his original design for the data logging files (both those ending in '''.ini''' and those ending in '''.txt''', and only adding extra fields to the end. |
|||
When Steve Loft took a new look at the data log files for Cumulus 2, he started with a new design, the principal change was that he decided to use UTC for all fields in them that reference dates and times. Steve struggled with Cumulus 2 largely because he was (at that time) not familiar with the C# language he was later to use for MX. It is fair to say that conversions between local time and UTC did contribute to his failure to get Cumulus 2 to provide the functionality he had offered in the original Cumulus. |
|||
When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, so he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible]. |
|||
===dayfile.txt=== |
|||
MX (except early releases) reads the whole of this file as it starts running, so all lines must use exactly the same format. |
|||
====date format issue==== |
|||
For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file. Please see [[Amending_dayfile#Date_Separator_in_MX]] for more detail of what MX expects. |
|||
Cumulus 1 allows the date that appears in the first field of each line to use any character (except space) to separate the day of the month, from the month, and from the last two digits of the year. Consequently "29/03/88", "29-03-88", and "29.03.88" are all acceptable in the legacy software (and indeed in some early MX releases). |
|||
You should read [[:Category:Ini Files]] page, as you might need to edit some items in the '''.ini''' files. |
|||
*See [[MX_on_Windows_OS#Parameter_for_changing_Locale]] for how to specify locale if you are running on Microsoft Windows (by default the locale settings are taken from the standard Windows settings application or Control Panel). |
|||
For the [[:Category:MX txt Files|'''.txt''' files]], you need to check that all lines are consistent in using the same character to separate the 3 parts of the date (see , and the same character is used throughout to separate the items in list of fields. |
|||
*If you are running MX in a Linux operating system, the locale parameter can be used, but the default locale is determined by the mono-complete package, and that in turn depends on your device settings. |
|||
So when you migrate "dayfile.txt", run it through an editor designed for programmers. Many such editors (e.g. Geary, NoteTab Light, Brackets) are available. |
|||
*Cumulus 1 will accept any character (except space, a digit, or the character used to separate fields in a list) as a separator for the date parts |
|||
* MX will only accept the character defined in your locale (for a Windows PC that is actually set in Control Panel which Windows does not make easy to access preferring you to use its Settings app. |
|||
*There is a Clock and Region section in Control Panel and in the Region window you '''define the separator MX will use in the Short date item''', most easily by clicking additional settings). |
|||
*Now ensure that is same character as used in al lines of your .txt log files. |
|||
** If you have some lines using say "/" and some using "." or "-" as date separator, then there are many editors available that offer a global replace. |
|||
***As an example, Notepad++ has a '''Replace All in All Opened Documents''', so you can open multiple documents and do a global replace of "/" into "-" very easily. Notepad++ saves the file in the same encoding as it was before, but you can also check it is UTF-8 with no Byte Order Mark via the menus. |
|||
***Note that you cannot easily replace "." or "-" globally in dates, because those appear as valid characters in fields. To edit those, you would need to open the files in a spreadsheet editor like '''Libre Office''', you need to ensure that the dates column is treated as text, then select that column and do a replace in just the date column, and then save as a CSV format file changing the extension to .txt. There is more information on this in relevant log file articles. |
|||
**The basic Microsoft Notepad that comes with Windows has a Replace option in its Edit menu, but some versions of this editor (check '''Save As''' options) don't allow you to pick "UTF-8" encoding (if that is not already shown) when saving to ensure MX can read the output. |
|||
**Also be aware when using Google editors that they may save the file in the wrong format unless you can choose a UTF-8 no BOM option. |
|||
For example "Notepad ++" is one that is popular. That editor has a "Search" menu, with a '''Replace...''' option within it. In that option, there is a "Find what" box, a "Replace with" box, and a "Regular expression" selection. |
|||
=== 2: COPY C1 AND OVERWRITE ORIGINAL WITH MX APPROACH === |
|||
Suppose your latest lines use either "." or "-" as separator, and that is what MX expects, but you have some older lines using "/" as date separator. Since you know "/" does not appear anywhere but in the dates with an older format, putting "/" in '''find what''' box and either "-" or "." (as required) in '''replace with''' box will sort you out after you hit "Replace all". |
|||
Alternatively, to run Cumulus MX with your existing Cumulus data, first take a back up copy of your existing Cumulus directory. |
|||
It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select '''Regular expression''' and then set '''Find what''' to "^([\d]{2})-([\d]{2})-" and '''Replace with''' to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it. |
|||
Next obtain the MX distribution release as a zip as per previous option. |
|||
====time-stamp format issue==== |
|||
Now unzip Cumulus MX into the original Cumulus folder. The new CumulusMX.exe should end up in same folder as existing '''Cumulus.ini''', there will be other files in this folder, and there will be some new folders you have not seen before like '''interface''' and '''MXDiags''', but MX continues to use the existing sub-folders (like '''data''' and '''backup''') without any change of name. |
|||
The dayfile.txt contains time-stamps following any value that represents a highest or lowest in the day. |
|||
This saves you from copying any of your Cumulus 1 files, they just stay where they are and get used by MX. |
|||
All such time-stamps must be expressed quoting hour and minutes, with a separator. The use of seconds is not accepted. |
|||
Where your existing [[Cumulus.ini_(Cumulus_1)]] file makes reference to local folders (in say Extra Web Files), those references stay valid. The only edit you will make to configuration that affects '''Cumulus.ini''' is dependent on your weather station type, as explained later MX uses a different parameter to refer to the port used by the weather station. |
|||
The legacy Cumulus was not fussy on the character separating the minutes from the hour. |
|||
Be aware that MX changes the date formats in some files, so that Cumulus 1 can no longer understand them, so be cautious about copying log files (.ini and .txt) back to your Cumulus 1 back-up should you want to revert to using Cumulus 1. |
|||
MX will only accept colon ":" separator, all dayfile.txt time-stamps must be in "HH:mm" format. You will need to edit your old lines if any use a different separator. |
|||
Some other issues that were described in previous sub-section will apply in this case, because for MX this approach (preserving Cumulus 1 in a new location) is no different to previous. |
|||
====value format issues===== |
|||
=== 3: NEW LOCATION APPROACH=== |
|||
If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all dayfile.txt lines for MX, and must match what is defined in the locale used. |
|||
These notes apply whether you are installing in a new location on your PC or installing MX on a different device. |
|||
=web server= |
|||
Obtain the MX download (from [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version). |
|||
Some people have been slow to migrate from the legacy Cumulus to MX because any web pages designed using a [[Cumulus template file]] that works with the legacy software will not work with MX. It is not appropriate for this page to explain how to edit your [[Customised templates]], but it can briefly cover the different approaches for default web pages. |
|||
When you do the unzip as explained below, it ensures all the files that need to be in sub-folders go into correct sub-folders. You then add various folders and files from your Cumulus 1 installation into correct places. As said before, this sub-section gives only a brief mention of possible issues with existing files, later sections will explain those in more detail. |
|||
From release 3.10.1, the new default web pages have a totally new look (designed by Neil Thomas), and include responsive code allowing them to look better on wide screens and on narrow screens (such as those on smart mobile phones). MX creates a number of [[:Category:JSON Files]] that are used to convey the data from your local MX installation to the web pages installed on your web server. Mark Crossley's [[SteelSeries Gauges|Steel Series]] is used for the MX gauges page. A single "use default web pages" setting sets up the settings required, see [[New Default Web Site Information]] page. |
|||
#First create a new directory (recommended name CumulusMX) and unzip the contents into it. |
|||
#*For a pi, the instruction might be <tt>unzip -o CumulusMXDist3'''nnn'''.zip</tt> where '''nnn''' is the last 3 digits of the build number. |
|||
Steve Loft's Cumulus 1 came with a set of example template files (designed by Beth Loft) that could generate web pages to upload to your web server. It also produced a set of images representing standard graphs that could be uploaded to your web server and shown on the default "trends.htm" page, a set of images representing wind speed and direction that could be shown on the default "gauges.htm" page these were based on'''Web Dashboard Components for FreeWX and FreeWX-Wi''' (no longer available) and mixed images generated by Cumulus with plots drawn using JavaScript routines (before "Canvas" functionality)., and a moon image that was shown on one web page. |
|||
#Then copy (or file transfer over) |
|||
#* your previous '''data''' folder contents into the new data folder created by unzipping, as before you might need to edit some log files |
|||
=Library software= |
|||
#*your previous '''Reports''' folder contents (if any) into the new Reports folder created by unzipping |
|||
#*your Cumulus.ini_(Cumulus_1) file goes into the top level folder, the one with ExportMySQL.exe, CumulusMX.exe and the .dll files, |
|||
Any non-technical person can skip this sub-section! |
|||
#** Check that destination file, it must be "Cumulus.ini" (ensure it starts with capital letter and all other letters are lower case) |
|||
#** In the new "Cumulus.ini" edit any lines that made reference to the old windows location. Remember, that only windows uses "\" in paths, and your new device will require reference to new locations. Even if you are still using windows, you may need to make changes to reflect that the files are now in a new location. |
|||
#**You can delay changing other settings (like the port used to access your weather station which uses a different parameter in MX) when you have access to the Settings pages in the MX admin interface |
|||
#* any other configuration files that you may have created (e.g. strings.ini, twitter.txt etc). |
|||
MX uses a number of library software utilities like Highstock, jQuery, boot strap, and others, see [[MX_Basic_info#Library_software_for_your_web_server]]. Be aware that MX determines the versions of these it seeks, and they may not match the versions needed for anything on your web server that is not supplied in the MX release distribution. A browser tries to reuse components that are already loaded, so there is a possibility of the wrong version being loaded. |
|||
The big advantage of this approach is that anytime you are not running MX you can go back to Cumulus 1 and let it run from where it left off (subject to availability of past data in your weather station, and possibly unplugging weather station from one device and plugging it into PC). Obviously you cannot run Cumulus 1 and MX at the same time accessing the same weather station (even if both are on same device and so no unplugging and plugging in is needed). |
Revision as of 03:09, 29 August 2021
Like other longer pages in this Wiki, the content is divided into sections, if you are interested in a particular issue, find it in the table of contents, rather than reading whole page!
Introduction
This page was inspired by this update from Cumulus 1 forum topic. That post was made in January 2020, and therefore relates to Version 3.3.0, which was the MX release that was in use at the time.
This page was last partially updated for the MX release available in June 2021; that is no longer latest!
Appeal to contributors: Please work out any updates needed for this page, so it is kept correct for more recent releases!
You might want to read the related page at My_migration_from_C1_to_MX which describes how this worked in practice for release version 3.4.5 (build 3069).
Should I migrate to MX or not?
The legacy Cumulus will never be updated, and as more people migrate to MX there will be fewer people to help you if the legacy cumulus software gives you a problem. MX has a major advantage it runs on more devices, and for example a Raspberry Pi computer is far cheaper to leave on all the time to run MX (knowing it will never be rebooted by Windows updates). The current developer of MX offers support, and other people are also gaining expertise, so if you have a problem you will get help.
Of course MX started with very little of the functionality that the original Cumulus offered, but over time MX has gained more and more functionality. Features in the original Cumulus have been gradually added to MX, but also MX offers a lot that is not in the original Cumulus. See Compare C1and MX page for more on this. Unfortunately, there is no list anywhere of all features in Cumulus 1. About_Cumulus does not cover all features. Nor is there a list of features still to be introduced into MX. Therefore it is hard to help you understand whether functionality changes between the flavours will suit you.
There are Cumulus users who tried MX, then have migrated back (from MX) to the legacy product; either on a temporary basis (to use a missing feature before they return to MX) or in a very few cases decided to stay with Cumulus 1 (because they prefer it). It is worth noting here that the legacy Cumulus is a stable product with very few bugs, but MX is constantly being developed and thus likely to contain bugs (these obviously vary depending on release as some are fixed and others appear). To put this in context, MX now has a huge amount of functionality and it is difficult for the developer to test all the different features.
Cumulus 1 creates a special log containing just temperature and humidity values, this is NOT available in MX.
Although from version 1.7.5 onwards the original Cumulus counts air frosts and gale days, this feature is not in MX.
The This Period screen that is a popular feature of the original Cumulus software as it tabulates data simply, is not available in MX.
Migrating from original Cumulus to beta MX
The beta builds (version 3.0.0) of MX were designed to make it easy to migrate. This section is based on Steve Loft's wording taken from the support forum with minimal alteration for its new context.
You can move between [Cumulus flavours] fairly easily, but you should really read all the guidance.
- In particular if you use decimal commas with Cumulus 1, then you MIGHT have issues when MX tries to read existing log files.
- [For separator characters in dates and lists] whilst Cumulus 1 only takes settings from the control panel in Windows,
- MX running in a Windows environment, takes settings from the locale you specify in a parameter when starting MX, or the Control Panel in Windows,
- MX running in a non-windows environment takes settings from the locale you specify in a parameter when starting MX, or the default locale in your Mono installation
- MX [may] take some [settings from] interaction between the two [the specified locale and the default locale], and MX may struggle to read file [lines] created by Cumulus 1 if the MX locale is not precisely same as settings when that file [line was] created.
While MX was in beta, there was limited documentation about what features were included and why, there was some fault reporting and a tracking list that showed when some of those issues were fixed, also initially the documentation on how MX had implemented features it did have was very sparse. The lack of a list of features in Cumulus 1, meant it remains difficult to track which Cumulus 1 features are or are not implemented in MX. Steve Loft said parts of MX were simply machine code level copies of parts of Cumulus 1 functionality, and parts were trying to offer better functionality, but he never said what was included in these two categories.
There was a list of enhancements requested by users for Cumulus 1, but that list was deleted before work on MX started. Despite that, it does seem that some features that were on the now lost list of enhancements for Cumulus 1 that never got implemented in Cumulus 1, have been implemented in MX, although again there is no definitive documentation.
How to install MX
The basic instructions for installing MX (see MX on Linux, MX on Windows OS, or Raspberry Pi Image, as appropriate for fuller instructions):
- Download the MX distribution (from Software for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version)
- Unzip into the location where you are going to install MX.
'Engine' and 'Admin Interface' for MX
When you run the original Cumulus software it displays a screen (see Cumulus_Screenshots#Main_Screen, and from there you navigate to other screens to view data or change settings.
MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a diagnostic log, but many people run MX on a device that has no monitor and so the terminal output (if any) is not monitored.
When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment). (You can also run MX as a service, that has different ways to start and stop, discussed in the links given in previous section).
The separate admin interface(unfortunately this is called various names in the support forum, including "user interface", "dashboard interface", and "admin interface") is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network.
If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality:
- Windows Defender has to be told to allow all for Cumulus MX.
- Typically for other security packages, select "CumulusMX.exe" then right click and select "Change File Rating to Trusted"
- In any "Firewall" package, add port 8998 as one that was permitted.
- Using "command prompt as administrator" type netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users, the response "URL reservation successfully added" means it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way.
The default URL if the browser is on the same machine as MX is http://localhost:8998/. Newer releases of MX will tell you an IPv4 address to use, such as by typing "http://192.168.1.64:8998/".
Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device.
For security reasons, the admin interface should not be accessible via the public internet.
Migrating existing data files from legacy Cumulus to a recent build of MX
If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and there is quite a bit to read, taking up most of the rest of this Wiki page. The Cumulus configuration files for Cumulus 1 and for MX share the same name, but their content is very different, so these are discussed at length in subsections below.
Obviously, you will want to copy ALL (except weather diary which uses a different file) the files in the #"data" folder, from any old installation into the new installation, but migrating these is not always easy, here is a quick summary of the potential issues (all discussed further in various subsections later):
- When you first run MX, the Start Date is recorded in the configuration file
- In the legacy software, this date is purely used as something to appear on one of the sample web templates
- In Cumulus MX, this date controls what data is read from Monthly log files, such as Standard log files, and Extra Sensor Files, any lines in those files with earlier dates are ignored by CumulusMX.exe, ExportToMySQL.exe, and other utilities.
- You can edit this StartDate=xxxxx parameter, in the admin interface select Station Settings → Common Options → Advanced Options → Records Began date
- Be aware your new installation has to use the same "locale" as the old installation, or MX will struggle as the locale affects how new lines are stored, and how MX expects old lines to have been stored the same way.
- If your old installation, is on a different operating system to the new installation, remember that Microsoft Windows uses different line terminators to all other operating systems, although MX should cope with mixed line terminators, any third party routines reading your data files will probably not accept a line terminator change. See MX on Linux page for more information on this.
- Cumulus MX is much more fussy than Cumulus 1 about every line in any data file using exactly the same locale formatting:
- Cumulus 1 is able to accept any character (other than the list separator, and space) being used to separate hour and minutes in time-stamps, MX expects a colon ":"
- Amending dayfile tells you about how MX is far more fussy about the content in dayfile.txt
- .ini files explains how time-stamps are formatted differently in the extreme tracking files, and how MX prefers decimal points to decimal commas
File names
Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that Wiki pages always change first letter to a capital, so do check carefully if that wiki page is describing a file name that must be all lower-case.
Line terminators
If you are not running MX on a Microsoft Windows computer, ideally you should change the characters appearing at the end of every line in every file you move from Cumulus 1.
I say ideally, because although Microsoft is fussy, and determined to be different by insisting files match its choice, most other operating systems can tolerate different line terminators. This means that MX will generally tolerate files using a mixture of line terminators. However, if you are using third-party routines, please be aware that these are often written to expect a particular line terminator, and they may give unexpected results when used with files that do not match that expectation!
With Cumulus 1 (or MX) on Windows, each line in every file ends with both carriage return and line feed.
With MX on a Mac, each line should end with just a Carriage Return.
For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed.
To change end of line characters, run each file through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its Edit menu, choose EOL conversion. On a Linux system like a Raspberry Pi computer, please see MX_on_Linux#Changing_line_terminators.
If you use PHP Hypertext Pre-processor anywhere on your installation, normally that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" function to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on!
Configuration file
MX's Cumulus.ini has different content to the legacy cumulus.ini. From release 3.12.0,many more new settings were added, more settings were removed, and some parameters in the file were renamed. The first cross reference, in listing all the parameters, makes it clear which settings were introduced in the legacy software, and which MX release added new settings, this should help you to grasp how many new settings exist!
The settings pages in MX, work differently to the settings screens in Cumulus 1. You need to click Save and (in many cases) stop and restart MX before changed settings take effect.
Consequently, if you are migrating from the legacy software to MX now, it is best to rename your old 'cumulus.ini file so it is is not seen by MX. Let MX create a new configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly.
Configuration file naming restrictions
Cumulus 1 can recognise, in some circumstances, "cumulus1.ini", and other variants, not just "cumulus.ini".
MX only recognises "Cumulus.ini". The Windows operating system is not case sensitive for file names. All other operating systems require the first letter of the filename to be a capital, and the remainder to be lower case as shown.
Historical evolution of configuration file
The oldest approach for migration from the legacy software, was to copy across to MX your existing configuration file, and let MX ignore all the parameters that do not apply to it. This worked because the early MX releases had few new parameters, and mostly used the majority of the old parameters.
For any parameters that were not set by the admin interface (and there were many "read-only" parameters in early releases of MX), one assumed these also existed in Cumulus 1 (refer to Cumulus.ini (Cumulus 1)) and so were already in your file.
To add the few new parameters that MX did need (see Cumulus.ini (MX 3.0.0 to 3.7.0) page), you would then go to the MX_Administrative_Interface#Changing_Settings pages and work steadily through ALL the options.
Retaining your old settings as far as possible simplified any migration of data files from your legacy installation into MX, because it ensured you kept to same unit selections, and same extra sensor selections.
At releases like 3.3.0 and 3.6.0, amongst others, there were changes to "read-only" parameters for MX. These could not be adjusted through the admin interface, and needed to be entered manually into the Cumulus.ini file, as instructed on Cumulus.ini (MX 3.0.0 to 3.7.0) page.
Subsequent evolution of configuration file
Substantial changes to settings available were made from release 3.8.0 onwards. The content of the Cumulus.ini file changed drastically, with much deviation from the configuration file used in earlier releases (including for the legacy software), hence Cumulus.ini is a new page describing the settings, and the parameters appearing in the configuration file, applicable from that release.
More changes were made in 3.10.0, and subsequent releases, as settings that previously had to be made directly in file were gradually moved to be advanced settings controlled in the admin interface. In addition, these releases, have added many new settings never encountered in your legacy software.
For example the changes include the (advanced) ability to change the number of decimal places for storing derived values like daily rainfall, sea level pressure, and wind speeds.
Changes to the Default Web Site (diverging away from the Customised templates approach in the legacy software) and implementing the option to use (as was available in the legacy software, and the earliest MX beta, but then removed) a copy action instead of a file transfer action for Your Own Server, has led to many,many new settings. Thus the advice has become abandon your old legacy software cumulus.ini file when you move to MX.
Settings
The settings pages in Cumulus 1 and MX work differently:
- for Cumulus 1 you choose to save changes by clicking OK,
- for MX changes are only saved when you click a Save button if one is provided.
- The alarms page works slightly differently, with an "Update alarms" button.
- If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved to configuration file when you move to next field/line, and acted on when you next restart MX.
Be aware that you should restart MX after changing many settings. This is because many settings are only written to configuration file, not yet to internal code, and such settings are only read from configuration file when MX starts. The developer is changing the code, so that in more cases edits on the settings pages do update the settings used by the code, as well as the file, but at time of writing there is no list of which settings work like that, and which require a restart.
Start date
When you first run any Cumulus software (whatever flavour) a parameter is added to the configuration file that documents the date Cumulus was first used.
For Cumulus 1, this parameter appeared in two places on the example web template for all-time records, but was otherwise ignored. Thus if you decided to import into your data logs readings from before you first ran Cumulus, the legacy software would be able to find those earlier records, without you needing to change the date that appeared on the web page.
CumulusMX.exe uses this parameter to determine the first standard data file to start reading from, it will ignore any data files for earlier dates, it will also ignore any lines with an earlier date in the first data file. Thus if you were to migrate your Cumulus 1 configuration file into MX, you should check the StartDate=xxxxx parameter in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time. If you need to make any edit, ensure you stick to exactly the same date format.
From release 3.10.1, MX allows you to edit this start date within the admin interface, just select Station Settings → Common Options → Advanced Options → Records Began date. It is "hidden" as an "advanced" setting, with a strict danger warning, because of the importance of sticking to exactly the right date format when you edit it!
Station connections
You can skip this subsection if your weather station connects to MX either by USB or by wireless connection.
This subsection is relevant if your station connects by a serial connection and MX needs to be told which port. It also applies if the serial connection is converted to USB.
If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in Cumulus.ini_(Cumulus_1) in the station section as a parameter in the format Port=n.
In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings page, but within Cumulus.ini in the station section the parameter is in the format Comport=tttttttt.
If your old parameter had a value of 3, and you are still using Windows, the new setting would have value of COM3, i.e. serial port 3 now requires a "COM" prefix.
A typical parameter value for other serial connecting devices might be "/dev/ttyUSB3" where the final digit will change depending on the new connection. You can search for posts on the support forum that talk about how to find out what connection is being used, depending on what hardware you are using.
RG11 Rain gauge
If you use a RG11 rain gauge:
- Replace: RG11port=n and RG11port2 =n (Cumulus 1) where n is a number,
- With: RG11portName=xxxx and RG11portName2=yyyy (Cumulus MX on Windows)
The new value is a string with values as per previous paragraph depending on device on which Cumulus MX is running.
Other Cumulus.ini parameters
Lots of parameters in Cumulus.ini are being changed as MX is developed. So much had changed by the time that MX releases reached 3.12.0, the first run of that particular release created a brand new Cumulus.ini discarding the old file. If you migrate from Cumulus 1 to MX without stepping through MX releases, you may have several issues, but without knowing which MX release you have selected, all I can suggest is that you work through all settings found in the Settings page of the MX release you choose. Although you could compare Cumulus.ini (Cumulus 1) and Cumulus.ini pages, there is no guarantee that either is totally accurate!
Strings.ini
This is another configuration file, but it is an optional one. Please remember that the Microsoft Windows Operating System is case insensitive for file names, so "Strings.ini", "STRINGS.INI", and "strings.ini" are all treated as the same file by any Cumulus software.
If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini".
If you have not created a strings.ini file in your (leagacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section.
The contents of the samplestring.ini file you get in your MX release distribution varies depending on the release you have downloaded.
Check your existing strings.ini file against the samplestring.ini file in the MX distribution you have. If the attribute names (left hand side of the equals sign) match for the parameters you selected to include in your strings.ini, then you can reuse your existing file. If your file includes attributes that are no longer in the MX samplesting.ini file, then you will need to edit your strings.ini file that is placed in the folder containing CumulusMX.exe. You may also need to add new items into your strings.ini based on new content in samplestring.ini.
NOAA style reports
The generation of reports is an optional feature, if you have never used it your (legacy) Cumulus Reports folder will be empty, then you have no files to move to your MX installation, and you should skip the rest of this sub-section.
Please see Reports_folder for full information. Cumulus software creates reports, it does not edit existing reports, so migration is fairly simple. Just copy the contents of the Reports folder in your original Cumulus installation into the folder in the new installation. Of course, nothing is totally simple, the encoding default changes between Cumulus 1 and MX, and once again we might need to consider end of line characters!
For those of you who are more technical:
- files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems). Apple Mac are again different in using just Carriage Return. This should not cause any problems.
- files can be encoded (how individual characters are represented by binary codes) by Cumulus in two different ways. There is more about encoding at Reports_folder#Encoding, the relevance here is that if your MX settings and Cumulus 1 settings use different encodings (as they will if you let this default) you may find some characters (e.g. degree symbol) do not appear correctly when viewing some of your reports.
"data" folder
You should copy all files in the data folder from your legacy Cumulus installation to your MX installation if you want to be able to see past data, but as mentioned earlier there are some complications:
- See #Start date earlier on this page for a change that you may need to make to ensure MX accepts data in any old Standard log files and Extra Sensor Files
- See #Weather Diary later on this page, and Weather Diary page; Cumulus does not provide any utility to convert the old legacy format to the new MX format.
- See #Extreme Record files later on this page, and Category:Ini Files, for information on differences between the legacy files and the MX files
- See #dayfile.txt later still on this page, and Amending dayfile page, for how you can ensure MX can read your existing dayfile.txt file
- See #Line terminators earlier on this page for some technical issues if MX is running on a different computer to your Cumulus 1 installation.
Standard log files and other log files
The Speciallog.txt file optionally used by the legacy Cumulus 1, is not used by MX.
The Standard_log_files and any (optional) Extra Sensor Files, from the legacy software can be transferred to your MX installation, but see #Start date earlier on this page, and Calculate Missing Values page to read about extra tasks you may need to do.
MX extends these Monthly log files by adding another optional set of files.
Weather Diary
As explained on weather diary page, the Cumulus 1 weather diary uses a different file to MX, and indeed takes a very different model for how to store the information.
Therefore don't copy Log.xml file into the data folder of your MX installation. You will need to manually use the admin interface Edit menu to access the new diary and add your previous entries one by one.
Please note the editing interface has been changed from release 3.10.1, but the page linked to above may not have been updated. A major change in the upgrade relates to handling of time-zones.
Please also note that the Cumulus 1 weather diary permitted multiple entries to be stored for an individual day, but the MX implementation only permits a single entry per day. Also the configuration file defines a time when the entries stop applying that may not match the rollover time for other weather measurements.
Extreme Record files
The Correcting_Extremes page explains how extreme records are stored in the Category:Ini Files discussed below, with the changes to all-time records being tracked in Alltimelog.txt.
You will find that MX creates at least one file that was not in the older Cumulus (monthlyalltimelog.txt).
the .ini files
Once again, the simple instruction is to copy these from your old installation to your new installation, to ensure your extreme record history is maintained. Once again, differences between Cumulus 1 and MX mean that the process may not be that simple, especially if you try to migrate straight to the latest MX release.
MX releases up to 3.5.4
Your Cumulus 1 .ini files can be read in all release versions from 3.0.0 to 3.5.y. Steve Loft did make two changes in his beta MX, these mean your .ini files may look strange while they have some entries made by Cumulus 1 and some made by MX:
- all newly stored values will use decimal points in these files, (i.e. any decimal commas valid in the legacy software, are not used by MX).
- all newly stored time-stamps will use the ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times) so new dates have year first, the parts of the date are separated by hyphens, and new times use colon (:) between hour and minutes (i.e. the regional settings that the legacy software adopted have been abandoned by MX)
- MX releases from version 3.0.0 to 3.5.4 can read all the date/time entries in any file whether in the Cumulus 1 format or in the MX format.
- MX will change any dates to the ISO format only when it needs to update that particular date/time.
- Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes.
If you used decimal commas in your Cumulus 1 installation, you will make the migration to MX easier, by an edit of each ".ini" file to change those commas to full stops.
Equally, if you had any times not using a ":" as separator, you will make the migration to MX easier, by an edit of each ".ini" file to change those characters into colons.
MX releases at versions 3.6.x onwards
Migration from the legacy Cumulus to MX has become harder as MX has been developed for two reasons:
- It appears that releases at versions 3.6.x onwards have become more fussy about existing content in these files due to changes in the source code.
- Effectively, the format of existing entries is expected to be same as any newly created entries, i.e. matching current locale settings
- The addition of Feels Like temperature and Canadian Humidity Index (Humidex) to the .ini files
- Neither of these was stored by the legacy software or by earlier MX releases
- The technique for adding this is described on Calculate Missing Values page
Despite this, most people migrating from Cumulus 1 to the newer MX releases, are doing so successfully, so it works for at least some old files.
Historical background to dates/times
When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. He wrote the software originally just for his own benefit and did not need to worry about time zones. Subsequently as he enhanced his software to make it usable by others, he faced many issues on how to cope with different time zones, and different weather stations having different sensors. In Cumulus 1, he basically focussed on compatibility by keeping to his original design for the data logging files (both those ending in .ini and those ending in .txt, and only adding extra fields to the end.
When Steve Loft took a new look at the data log files for Cumulus 2, he started with a new design, the principal change was that he decided to use UTC for all fields in them that reference dates and times. Steve struggled with Cumulus 2 largely because he was (at that time) not familiar with the C# language he was later to use for MX. It is fair to say that conversions between local time and UTC did contribute to his failure to get Cumulus 2 to provide the functionality he had offered in the original Cumulus.
When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, so he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not backwards compatible.
dayfile.txt
MX (except early releases) reads the whole of this file as it starts running, so all lines must use exactly the same format.
date format issue
For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file. Please see Amending_dayfile#Date_Separator_in_MX for more detail of what MX expects.
Cumulus 1 allows the date that appears in the first field of each line to use any character (except space) to separate the day of the month, from the month, and from the last two digits of the year. Consequently "29/03/88", "29-03-88", and "29.03.88" are all acceptable in the legacy software (and indeed in some early MX releases).
- See MX_on_Windows_OS#Parameter_for_changing_Locale for how to specify locale if you are running on Microsoft Windows (by default the locale settings are taken from the standard Windows settings application or Control Panel).
- If you are running MX in a Linux operating system, the locale parameter can be used, but the default locale is determined by the mono-complete package, and that in turn depends on your device settings.
So when you migrate "dayfile.txt", run it through an editor designed for programmers. Many such editors (e.g. Geary, NoteTab Light, Brackets) are available.
For example "Notepad ++" is one that is popular. That editor has a "Search" menu, with a Replace... option within it. In that option, there is a "Find what" box, a "Replace with" box, and a "Regular expression" selection.
Suppose your latest lines use either "." or "-" as separator, and that is what MX expects, but you have some older lines using "/" as date separator. Since you know "/" does not appear anywhere but in the dates with an older format, putting "/" in find what box and either "-" or "." (as required) in replace with box will sort you out after you hit "Replace all".
It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select Regular expression and then set Find what to "^([\d]{2})-([\d]{2})-" and Replace with to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it.
time-stamp format issue
The dayfile.txt contains time-stamps following any value that represents a highest or lowest in the day.
All such time-stamps must be expressed quoting hour and minutes, with a separator. The use of seconds is not accepted.
The legacy Cumulus was not fussy on the character separating the minutes from the hour.
MX will only accept colon ":" separator, all dayfile.txt time-stamps must be in "HH:mm" format. You will need to edit your old lines if any use a different separator.
value format issues=
If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all dayfile.txt lines for MX, and must match what is defined in the locale used.
web server
Some people have been slow to migrate from the legacy Cumulus to MX because any web pages designed using a Cumulus template file that works with the legacy software will not work with MX. It is not appropriate for this page to explain how to edit your Customised templates, but it can briefly cover the different approaches for default web pages.
From release 3.10.1, the new default web pages have a totally new look (designed by Neil Thomas), and include responsive code allowing them to look better on wide screens and on narrow screens (such as those on smart mobile phones). MX creates a number of Category:JSON Files that are used to convey the data from your local MX installation to the web pages installed on your web server. Mark Crossley's Steel Series is used for the MX gauges page. A single "use default web pages" setting sets up the settings required, see New Default Web Site Information page.
Steve Loft's Cumulus 1 came with a set of example template files (designed by Beth Loft) that could generate web pages to upload to your web server. It also produced a set of images representing standard graphs that could be uploaded to your web server and shown on the default "trends.htm" page, a set of images representing wind speed and direction that could be shown on the default "gauges.htm" page these were based onWeb Dashboard Components for FreeWX and FreeWX-Wi (no longer available) and mixed images generated by Cumulus with plots drawn using JavaScript routines (before "Canvas" functionality)., and a moon image that was shown on one web page.
Library software
Any non-technical person can skip this sub-section!
MX uses a number of library software utilities like Highstock, jQuery, boot strap, and others, see MX_Basic_info#Library_software_for_your_web_server. Be aware that MX determines the versions of these it seeks, and they may not match the versions needed for anything on your web server that is not supplied in the MX release distribution. A browser tries to reuse components that are already loaded, so there is a possibility of the wrong version being loaded.