Migrating from Cumulus 1 to MX: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search
(Paste back text)
(22 intermediate revisions by the same user not shown)
Line 1: Line 1:
=Introduction=


This page covers moving from Cumulus 1 to MX, and '''in particular it explains exactly what is required to run MX on the Windows Operating System'''.
This page was inspired by this [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17749 update from Cumulus 1] forum topic.  That post was made in January 2020, and therefore the bulk of the text on this page relates to Version 3.3.0, which was the MX release that was in use at the time.


It was partly inspired by this [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17749 update from Cumulus 1] forum topic.  As that post was made in January 2020, the bulk of the text on this page relates to Version 3.3.0, which was the MX release that was in use at the time. The "How I did it" section was added based on release version 3.5.0 build 3069. The "Moving Files to MX" section was updated for release version 3.7.0 when increased functionality available in MX made it attractive to Cumulus 1 users, and the support forum saw posts from a number of users who were struggling to get MX to recognise their Cumulus 1 files.
<div style="background: LemonChiffon;padding:5px; margin:2px;">
[[File:Crystal Clear info.png|40px]] This page was last partially updated for the MX release in July 2020; that is no longer latest!


Appeal to contributors: Please work through all MX release announcements and work out all the many updates needed for this page, it may even need a redesign for more recent releases!
</div>


<div style="background: LemonChiffon;padding:5px; margin:2px;">
[[File:Crystal Clear info.png|40px]] This document was written for a MX release that is no longer latest!


The text needs to be updated so that it covers those who move from Cumulus 1 to a version of MX up to 3.5.y (which is an easy transition), and those who subsequently move to the latest MX release and wish to take advantage of extra functionality that provides.
You might want to read the related page at [[My_migration_from_C1_to_MX]] which describes how this worked in practice for release version 3.4.5 (build 3069).
</div>
 
{{TOCright}}
 
==Should I migrate to MX or not?==
 
The legacy Cumulus will never be updated, and as more people migrate to MX there will be fewer people to help you if the legacy cumulus software gives you a problem. MX has a major advantage it runs on more devices, and for example a Raspberry Pi computer is far cheaper to leave on all the time to run MX (knowing it will never be rebooted by Windows updates). The current developer of MX offers support, and other people are also gaining expertise, so if you have a problem you will get help.
 
 
Of course MX started with very little of the functionality that the original Cumulus offered, but over time MX has gained more and more functionality. Features in the original Cumulus have been gradually added to MX, but also MX offers a lot that is not in the original Cumulus. See [[Compare_C1_and_MX|Compare C1and MX]] page for more on this. Unfortunately, there is no list anywhere of all features in Cumulus 1. [[About_Cumulus|About_Cumulus does not cover all features]]. Nor is there a list of features still to be introduced into MX. Therefore it is hard to help you understand whether functionality changes between the flavours will suit you.
 
There are Cumulus users who tried MX, then have migrated back (from MX) to the legacy product; either on a temporary basis (to use a missing feature before they return to MX) or in a very few cases decided to stay with Cumulus 1 (because they prefer it). It is worth noting here that the legacy Cumulus is a stable product with very few bugs, but MX is constantly being developed and thus likely to contain bugs (these obviously vary depending on release as some are fixed and others appear). To put this in context, MX now has a huge amount of functionality and it is difficult for the developer to test all the different features.
 
Cumulus 1 creates a [[Speciallog.txt|special log]] containing just temperature and humidity values, this is NOT available in MX.


Although from version 1.7.5 onwards the original Cumulus counts air frosts and gale days, this feature is not in MX.


= Considerations when moving between Cumulus flavours =
The [[Cumulus_Screenshots#View_data|This Period]] screen that is a popular feature of the original Cumulus software as it tabulates data simply, is not available in MX.


=Migrating from original Cumulus to beta MX=


NOTE: This section is based on Steve Loft's wording taken from the support forum with minimal alteration for its new context.
The beta builds (version 3.0.0) of MX were designed to make it easy to migrate.  This section is based on Steve Loft's wording taken from the support forum with minimal alteration for its new context.


You can move between [Cumulus flavours] fairly easily, but you should really read all the guidance.  
You can move between [Cumulus flavours] fairly easily, but you should really read all the guidance.  
Line 25: Line 39:
** MX [may] take some [settings from] interaction between the two [the specified locale and the default locale], and MX may struggle to read file [lines] created by Cumulus 1 if the MX locale is not precisely same as settings when that file [line was] created.
** MX [may] take some [settings from] interaction between the two [the specified locale and the default locale], and MX may struggle to read file [lines] created by Cumulus 1 if the MX locale is not precisely same as settings when that file [line was] created.


== Moving files from Cumulus 1 to MX==
While MX was in beta, there was limited documentation about what features were included and why, there was some fault reporting and a tracking list that showed when some of those issues were fixed, also initially the documentation on how MX had implemented features it did have was very sparse. The lack of a list of features in Cumulus 1, meant it remains difficult to track which Cumulus 1 features are or are not implemented in MX. Steve Loft said parts of MX were simply machine code level copies of parts of Cumulus 1 functionality, and parts were trying to offer better functionality, but he never said what was included in these two categories.
 
*Steve Loft designed MX so that in general it was able to read [[Moving_from_Cumulus_1_to_MX#Log_files|log files]] created by the original Cumulus.
There was a list of enhancements requested by users for Cumulus 1, but that list was deleted before work on MX started. Despite that, it does seem that some features that were on the now lost list of enhancements for Cumulus 1 that never got implemented in Cumulus 1, have been implemented in MX, although again there is no definitive documentation.
*See individual articles for more details on such differences.
*'''Please note, the information for MX is taken from notes by Steve Loft, and applies to MX 3.0.0.'''
* If you move from Cumulus 1 to a version of MX  up to 3.5.y, the transfer is likely to be significantly easier. This is because Steve Loft was keen to keep forward compatibility in his beta MX, and it took a while for Mark Crossley to revise the part of the MX code that reads log files.
** Cumulus 1 is tolerant of various separators for dates and times.
**Cumulus MX 3.0.0 insists all times use a colon separator (says Steve Loft), and all dates are in ISO format.  
*This means, '''MX releases from version 3.0.0 to 3.5.4''' can read the date/time entries in the Cumulus 1 format and in the MX format.
*When the MX software needs to update a particular entry (e.g. an extreme) in a particular file, it will subsequently save the file and change the content for the affected line to the new format.
*Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes.
===Update as at MX release 3.7.0===


In particular '''internal''' changes mentioned in release announcements for various 3.6.x versions, and for release 3.7.0, have made MX work significantly differently and it is now more fussy about content of log files having to be in correct MX format.
=How to install MX=
** It is difficult to move from Cumulus 1 to a release of MX that is 3.6.z or higher, mainly because you may find that Cumulus 1's .txt, and .ini, files can no longer be read.
**This is because '''for releases 3.7.0 onwards''', each log file in modern MX releases must have lines that match what is in the local settings absolutely perfectly;
**this means MX is now very fussy about the symbol that separates the 3 parts of a date, very fussy about decimal symbol, and expects the number of parameters/fields in files to be consistent with release 3.7.0.
**Thus modern releases are less tolerant of log files created by the original Cumulus software;
===General points===


* Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that wiki pages change first letter to a capital even when a file that must be all lower-case is being described.
The basic instructions for installing MX:
* Cumulus 1 is reasonably tolerant of numbers using a comma to separate the integer and decimal part of real number.
#Download the MX distribution (from [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version)
** MX 3.0.0 will accept both decimal commas and decimal full stops in the standard log files, the extra sensor log files, and the daily summary log file.
#Unzip into the location where you are going to install MX.
**In all versions of MX, the software expects periods/full stops in .ini files regardless of the locale in use, please see the [https://cumulus.hosiene.co.uk/viewtopic.php?f=27&t=12908  Cumulus MX Announcements and Download - PLEASE READ FIRST] for details, but generally if you want to transfer these files from Cumulus 1 to MX, it is best if '''you edit them so all decimal numbers have a "." between the integer and decimal parts'''.
* In individual articles, any differences in the precise content or format within individual files will be highlighted.


==Moving Files from MX to Cumulus 1==
Alternatively, follow instructions at  [[Raspberry Pi Image]] which describes a simpler way to install MX (assuming you have another device linked to your RPi to do downloads and micro-SD card preparation).


When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. When he designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, and he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible].


That said, there are some features in Cumulus 1 that are [[Moving_from_Cumulus_1_to_MX#Cumulus_1_features_not_ever_going_to_be_included_in_MX|missing in MX]], and there are Cumulus users who have migrated back either on a temporary basis (to use a missing feature before they return to MX) or in a very few cases decided to stay with Cumulus 1 (because they prefer it).


== Key differences and similarities between Cumulus 1 and MX ==
=Migrating from legacy Cumulus to a recent build of MX=


NOTE: This section is also based on Steve Loft's wording taken from the support forum with minimal alteration for its new context.


* While Cumulus 1 has a tool to generate graphs itself and then uploads them to your website, the graphs used in Cumulus MX are drawn when the end-user loads the web page, they use Highcharts routines that are free for non-commercial use only, i.e. you may not use MX with these graphs on a company web site.
<big>The text that follows mostly relates to version 3.3.0, it probably needs to be updated, as later releases of MX have massively changed. The newer releases are more fussy about formats used in files, and migration from the legacy Cumulus software is harder</big>
* While Cumulus 1 runs as an application that includes a main screen, and other screens, that appear when you start it, Cumulus MX is two separate applications, there is the "engine" that connects to your weather station and processes that data, but there is also a separate administrative interface. The latter is viewed on a browser ''on any device connected to the same local network'' as the device that runs the engine. On this admin interface you change settings, you can edit the various logs, and you can view a series of web pages that allow you to see all the weather derivatives output from MX.
* The settings for both Cumulus 1 and MX are held in [[Cumulus.ini|'''Cumulus.ini''']]. For MX the file name is case sensitive [on non-Windows devices] and must have capitals where shown.
**The case sensitivity of MX [on all devices, even Windows,] also applies to the section names within the file e.g. [FTP site] must use capitals for the FTP and must use lower case for site. Edit any section names that do not follow format in the wiki article for this file referenced above.
**All the characters used within this configuration file must be within ASCII range (represented by binary 0 to 127, basically A to Z, a to z, 0 to 9, and some punctuation), any extended characters (such as those used for accented characters, symbols and non English characters) must be removed.
**Whilst many settings are common between both flavours, some are not used by MX and MX has some new ones. In particular if you used '''Port''' in Cumulus 1, that will not be carried across to MX, and you will need to set '''ComPort''' instead. You are advised to check all '''Settings''' using the MX admin interface.
*The contents of your Cumulus 1 '''Reports''' folder (NOAA style reports) can be read by MX.
*The contents of your Cumulus 1 '''data''' folder (log files ending with extension '''.ini''' or '''.txt''') can be read by MX.
**However, if you use decimal commas in your Cumulus 1 '.ini' files then you do need to change, in each stored value, the decimal commas into periods/full stops.
**MX when it needs to update these files will change the way dates are stored, see the [[:Category:Log Files|Log File]] pages in the Wiki for more information.
** The '.txt' files in the data folder will work with both Cumulus 1 and MX - assuming you are using the same decimal and list separators in MX as you used in Cumulus 1 (i.e. the same locale).
*The Cumulus 1 web templates (files using web tags) will not work with MX (whether you use the standard files provided or have written your own replacements)
**The reason is because the content of the standard web pages is different. For any web templates you have written, you will almost certainly need to change some web tags, and you may find this difficult because certain formatting characters (e.g. H or M) have different meanings when they appear in isolation in an output format to what they mean when combined with others (e.g.H:mm or 'd M'). See the [[Webtags]] page for full information on how to change these, and ask in the support forum if you have difficulty.
*(Other file names within MX will be as supplied in the zip that you download, or as Cumulus MX decides when it creates the file).
* The settings in Cumulus 1 and MX work differently, for Cumulus 1 you choose to save changes by clicking OK, for MX changes are only saved when you click a '''Save''' button if one is provided. If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved when you move to next field/line.
Finally if you are moving from Windows to Linux, remember you need to learn a host of new commands!


If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and is the subject of the rest of this article.


=Features and functionality=


== Cumulus 1 features not ever going to be included in MX ==
== 'Engine' and 'Admin Interface' for MX ==


Unfortunately, there is no list anywhere of all features in Cumulus 1. [[About_Cumulus|About_Cumulus does not cover all features]]. Nor is there a list of features still to be introduced into MX. Thus this section is neither authoritative nor definitive.
When you run the original Cumulus software it displays a screen (see [[Cumulus_Screenshots#Main_Screen]], and from there you navigate to other screens to view data or change settings.


One big feature that Cumulus 1 had was this '''view period''' screen as illustrated here and described above, where you could specify any start and end dates, so it was easy to see a summary for any day in the past, any week in the past, any month, year or season, whatever you needed. MX does not offer anything like this.
MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a [[MXDiags_folder|diagnostic log]], but many people run MX on a device that has no monitor and so the terminal output (if any) is not monitored.


Cumulus 1 creates a [[Speciallog.txt|special log]] containing just temperature and humidity values, this is NOT available in MX.
When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment).
 
The separate [[MX Administrative Interface|admin interface]] is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network.  If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality:
#  Windows Defender has to be told to allow all for Cumulus MX.
#  Typically for other security packages, select "CumulusMX.exe" then right click and select "Change File Rating to Trusted"
#  In any "Firewall" package, add port 8998 as one that was permitted.
#  Using "command prompt as administrator" type  <tt>netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users</tt>, the response "URL reservation successfully added" means it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way.
 
The default URL if the browser is on the same machine as MX is http://localhost:8998/.
 
If you are using the browser on a different device on your local network to the device running MX, you cannot use that local host shortcut. Instead you specify a IPv4 address, that is listed in your router (might be called a hub) for the device running MX, this IPv4 address will look like '192.168.y.z' (where y and z are numbers that vary between implementations).  For example you might acces the admin interface by typing "http://192.168.1.64:8998/".
 
Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device.
 
For security reasons, the admin interface should not be accessible via the public internet.
 
==Configuration file==
 
MX's [[Cumulus.ini]] has different content to the legacy [[Cumulus.ini_(Cumulus_1)|'''cumulus.ini''']].
 
Cumulus 1 can recognise, in some circumstances, "cumulus1.ini", and other variants, not just "Cumulus.ini". MX only recognises "Cumulus.ini".
 
The old approach for migration, was to copy across your existing file, and let MX ignore all the parameters that do not apply to it.  To add the parameters that MX does need, you would then go to the [[MX_Administrative_Interface#Changing_Settings]] pages and work steadily through ALL the options. For any parameters that were not set by the admin interface (and there were many "read-only" parameters in early releases of MX), one assumed these also existed in Cumulus 1 and so were already in your file.
 
At releases like 3.3.0 and 3.6.0, amongst others, there were changes to "read-only" parameters for MX. These could not be adjusted through the admin interface, and needed to be entered manually. To keep this page simple, it cannot list the "read-only" parameters that you might need to add for migrating to earlier releases.
 
It appears that from release 3.10.1, all settings are now initialised via the admin interface (and none are "read-only"), so if you are using that release or later, it may be better to ignore your Cumulus 1 configuration file, and to let MX create a configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]].  That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly.
 
===Start date===
 
When you first run any Cumulus software (whatever flavour) a parameter is added to the configuration file that documents the date Cumulus was first used.
 
For Cumulus 1, this parameter appeared in two places on the example web template for all-time records, but was otherwise ignored.  Thus if you decided to import into your data logs readings from before you first ran Cumulus, the legacy software would be able to find those earlier records.
 
'''CumulusMX.exe''' uses this parameter to determine the first standard log file to start reading from, it will ignore any log files for earlier dates. Thus if you were to migrate your Cumulus 1 configuration file into MX, you should check the "StartDate=" line in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time.  If you need to make any edit, ensure you stick to exactly the same date format. 
 
From release 3.10.1, MX allows you to edit this start date within the admin interface. It is "hidden" as an "advanced" setting, with a strict danger warning, but it is there!
 
===Uploading to web site===
 
Many settings have changed in this area for MX, and they vary depending on which release you are installing. No attempt is made to explain further here.
 
=== Station connections===
 
If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in Cumulus.ini_(Cumulus_1) in the station section as a parameter in the format '''Port=n'''.
 
In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings page, but within [[Cumulus.ini]] in the station section the parameter is in the format  '''Comport=tttttttt'''.
 
If your old parameter had a value of '''3''', and you are still using Windows, the new setting would have value of '''COM3'''. 
 
A typical parameter value for other devices might be "/dev/ttyUSB0".
 
===RG11 Rain gauge===
 
If you use a RG11 rain gauge:
*Replace: '''RG11port=n''' and '''RG11port2 =n''' (Cumulus 1) where n is a number, by '''RG11portName=xxxx''' and '''RG11portName2=yyyy'''  (Cumulus MX on Windows) where the value is a string with values as per previous paragraph depending on device on which Cumulus MX is running.
 
From release 3.10.1, there appear to be other new parameters, not yet documented.
 
==Strings.ini==
 
This is another configuration file, but it is an optional one. If you have not created a [[Strings.ini|strings.ini]] file in your (leagacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section.
 
The contents of the [[Samplestring.ini|samplestring.ini]] file you get in your MX release distribution varies depending on the release you have downloaded.  Check your existing '''strings.ini''' file against the ''samplestring.ini'' file in the MX distribution you have.  If the attribute names (left hand side of the equals sign) match for the parameters you selected to include in your '''strings.ini''', then you can reuse your existing file. If your file includes attributes that are no longer in the MX ''samplesting.ini'' file, then you will need to edit your '''strings.ini''' file that is placed in the folder containing CumulusMX.exe.
 
Please remember that the Microsoft Windows Operating System is case insensitive for file names, if you install MX on a Windows PC, then "Strings.ini", "STRINGS.INI", and "strings.ini" are all treated as the same file by MX.  If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini".
 
For those of you who are more technical:
*Note that files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems like all Linux variants including Raspberry Pi Operating System). Apple Mac are again different in using just Carriage Return.
*This should not cause any problems for your "strings.ini" file as MX does not care if there appear to be some extra blank lines (because the carriage return may be treated as end one line and the line feed as ending a separate blank line on non-Windows devices).
 
==NOAA style reports==
 
The generation of reports is an optional feature, if you have never used it your (legacy) Cumulus Reports folder will be empty, then you have no files to move to your MX installation, and you should skip the rest of this sub-section.
 
Please see [[Reports_folder]] for full information.  Cumulus software creates reports, it does not edit existing reports, so migration is simple. Just copy the contents of the '''Reports''' folder in your original Cumulus installation into the folder in the new installation.
 
For those of you who are more technical:
* files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems). Apple Mac are again different in using just Carriage Return. This should not cause any problems.
* files can be encoded (how individual characters are represented by binary codes) in different ways. There is more about encoding at [[Reports_folder#Encoding]], the relevance here is that if your MX settings and Cumulus 1 settings use different encodings you may find some characters (e.g. degree symbol) do not appear correctly when viewing some of your reports.
 
=="data" folder==
 
The contents of your Cumulus 1 '''data''' folder (log files ending with extension '''.ini''' or '''.txt''') can NORMALLY be read by MX (but see points listed below).
 
However, if you are not running MX on a Microsoft Windows computer, ideally you should change the characters appearing at the end of every line in any file you move from Cumulus 1.  To do this, run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its '''Edit''' menu, choose '''EOL conversion'''.  Do remember that with Cumulus 1 on Windows, each line in every file ends with both carriage return and line feed. If you are moving to MX on a Mac, each line should end with just a Carriage Return. For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed.
 
I say ideally, because although Microsoft is fussy, and determined to be different, most other operating systems can tolerate different line terminators.  If you use PHP Hypertest Pre-processor anywhere on your installation, you will discover that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on!
 
===Weather Diary===
 
As explained on [[Weather_Diary|weather diary]] page, the Cumulus 1 weather diary uses a different file to MX, and indeed takes a very different model for how to store the information.
 
Therefore don't copy [[Log.xml]]  file into the data folder of your MX installation.  You will need to manually use the [[Weather_Diary#How_to_edit_the_contents|admin interface Edit menu]] to access the new diary and add your previous entries one by one.
 
Please note the editing interface has been changed from release 3.10.1, but the page linked to above may not have been updated. A major change in the upgrade relates to handling of time-zones.
 
Please also note that the Cumulus 1 weather diary permitted multiple entries to be stored for an individual day, but the MX implementation only permits a single entry per day.  Also the configuration file defines a time when the entries stop applying that may not match the rollover time for other weather measurements.
 
===dayfile.txt===


Although from version 1.7.5 onwards the original Cumulus counts air frosts and gale days, this feature is not in MX.


Another feature still missing from MX is the "Create Missing" feature that read the standard logs to work out approximate (because not accessing values for every second of day) entries to go in dayfile.txt.
====date format issue====


== Cumulus 1 Functionality Missing in beta MX ==
Cumulus 1 allows the date that appears in the first field of each line to use any character (except space) to separate the day of the month, from the month, and from the last two digits of the year. Consequently "29/03/88", "29-03-88", and "29.03.88" are all acceptable in the legacy software (and indeed in some early MX releases). 


Many other features of Cumulus 1 were missing in the beta MX, as written by Steve Loft.
For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file.
*See [[MX_on_Windows_OS#Parameter_for_changing_Locale]] for how to specify locale if you are running on Microsoft Windows (by default the locale settings are taken from the standard Windows settings application or Control Panel).
*If you are running MX in a Linux operating system, the locale parameter can be used, but the default locale is determined by the mono-complete package, and that in turn depends on your device settings.


However, subsequent developments headed by Mark Crossley have now added the majority of the missing features and the list of missing features added is below.
So when you migrate "dayfile.txt", run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows. That editor has a "Search" menu, with a '''Replace...''' option within it. In that option, there is a "Find what" box, a "Replace with" box, and a "Regular expression" selection.  


It is still difficult for a Cumulus 1 user to work out the disadvantages of leaving Cumulus 1. There are some obvious advantages in moving to MX, there are a lot of extra new features, there is ongoing support in the forum, and there is prospect of more improvements in the future. Cumulus 1 will never be enhanced.
Suppose your latest lines use either "." or "-" as separator, and that is what MX expects, but you have some older lines using "/" as date separator. Since you know "/" does not appear anywhere but in the dates with an older format, putting "/" in '''find what''' box and either "-" or "." (as required) in '''replace with''' box will sort you out after you hit "Replace all".  


== The look for web site ==
It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select '''Regular expression''' and then set '''Find what''' to "^([\d]{2})-([\d]{2})-" and '''Replace with''' to "$1/$2/".  Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it.


Cumulus 1, and later Cumulus MX versions - generate a moon image that can use FTP to place it on your web site.
====time-stamp format issue====


Cumulus 1 generated a number of chart images. MX generates a number of JSON files with data for graphs instead.
The file contains time-stamps following any value that represents a highest or lowest in the day. These must be expressed as hour and minute, and normally a colon (":") is used as a separator, but again the legacy Cumulus does not actually insist on that while MX does insist that are time-stamps are in "HH:mm" format.


The "bird" image used as a background on standard web pages is still provided with MX.


The actual web pages provided in MX fulfil same range of outputs as Cumulus 1, but these are not interchangeable.
====value format issues=====
*The Trends page of Cumulus 1 will not work as that simply displayed uploaded images, the MX Trends page uses HighStock to draw graphs from the data pairs in the uploaded JSON files.
*The old gauges page is replaced by Steel Series gauges.


== Enhancements during Beta stage for MX and after MX came out of beta ==
If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ",").  Again, these must be consistent in all lines for mX, and must match what is defined in the locale used.


While MX was in beta, there was limited documentation about what features were included and why, there was some fault reporting and a tracking list that showed when some of those issues were fixed, also initially the documentation on how MX had implemented features it did have was very sparse. The lack of a list of features in Cumulus 1, meant it remains difficult to track which Cumulus 1 features are or are not implemented in MX. Steve Loft said parts of MX were simply machine code level copies of parts of Cumulus 1 functionality, and parts were trying to offer better functionality, but he never said what was included in these two categories.
===other .txt files===


There was a list of enhancements requested by users for Cumulus 1, but that list was deleted before work on MX started. Despite that, it does seem that some features that were on the now lost list of enhancements for Cumulus 1 that never got implemented in Cumulus 1, have been implemented in MX, although again there is no definitive documentation.
These should migrate without any problems. You will find that MX creates at least one file that was not in the older Cumulus ([[Correcting_Extremes#How_do_I_correct_my_monthly_all-time_records.3F|monthlyalltimelog.txt]]).


The documentation that does exist is the release notes issued by Steve or Mark. In the next few sections, an attempt is made to prećis such announcements to track when the basic functionality from Cumulus 1 made it into MX.


Of course weather station design evolves,  and the sensors available vary. Thus another feature of MX development is adapting to new weather station dongles and new sensors (which obviously cannot be added to Cumulus 1). Once MX came out of beta, the changes in each release are being tracked in another [[Cumulus_MX_formal_release_versions|article]]. Obviously, you still need to look at the support forum for the detailed information in full release announcements about fixes, functionality changes, and actions needed to update to new major versions.  
===the .ini files===


= The difference in how a Cumulus 1 user and a MX user see current values and change settings =
When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. He wrote the software originally just for his own benefit and did not need to worry about time zones. Subsequently as he enhanced his software to make it usable by others, he faced many issues on how to cope with different time zones, and different weather stations having different sensors. In Cumulus 1, he basically focussed on compatibility by keeping to his original design for the data log files (both those ending in '''.ini''' and those ending in '''.txt''', and only adding extra fields to the end.


== Cumulus 1 ==
When Steve Loft took a new look at the data log files for Cumulus 2, he started with a new design, the principal change was that he decided to use UTC for all fields in them that reference dates and times. Steve struggled with Cumulus 2 largely because he was (at that time) not familiar with the C# language he was later to use for MX. It is fair to say that using UTC was also a major factor in his failure to get Cumulus 2 to provide the functionality he had offered in the original Cumulus.


Cumulus 1 was an all in one application, it both read the data from the weather station and provided the user interface for you to see the derived data and change the settings.
When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, and he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not [https://cumulus.hosiene.co.uk/viewtopic.php?f=4&t=15167 backwards compatible].  However those MX beta builds were designed to be forward compatible in that all date formats used by Cumulus 1 could be read, although MX would when rewriting any date change them to the new format.  


== 'Engine' and 'Admin Interface' for MX ==
For the '''.ini''' files, Steve Loft did make one change in his beta MX, all values now had to use decimal points in these files, and decimal commas were no longer valid.  Thus to migrate to earlier MX releases, if you had used decimal commas, you needed to edit each ".ini" file and change those commas to full stops.  Equally, if you had any times not using a ":" as separator, this had to be rectified.


MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a [[MXDiags|diagnostic log]], but many people run it on a device that has no monitor and so the terminal output (if any) is not monitored.
Apart from that, in all versions from 3.0.0 to 3.5.y, there was forward compatibility, and MX could read any file created by the original (legacy) software. '''MX releases from version 3.0.0 to 3.5.4''' can read all the date/time entries in any file whether in the Cumulus 1 format or in the MX format. MX will change any dates to the ISO format only when it needs to update that particular date/time. Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes.


When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment).
It appears that releases at versions 3.6.x onwards have become more fussy about existing content in these files due to changes in the source code.  However, the developer has not documented whether files from Cumulus 1 can still be read, forward compatibility has only been guaranteed with earlier MX releases.  That said, some people are successfully migrating from Cumulus 1 to the newer MX releases, so it works for at least some old files.


=General points=


The separate [[MX Administrative Interface|admin interface]] is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network. The default URL if the browser is on the same machine as MX is http://localhost:8998/.
==File names==


If you are using the browser on a different device on your local network to the device running MX, you cannot use that local host shortcut. Instead you specify a IPv4 address, that is listed in your router (might be called a hub) for the device running MX, this IPv4 address will look like '192.168.y.z' (where y and z are numbers that vary between implementations).
Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that wiki pages change first letter to a capital even when a file that must be all lower-case is being described.


Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device.
==web server==


For security reasons, the admin interface should not be accessible via the public internet.
Steve Loft's Cumulus 1 came with a set of example template files (designed by Beth Loft) that could generate web pages to upload to your web server.  It also produced a set of images representing standard graphs that could be uploaded to your web server and shown on the default "trends.htm" page, a set of images representing wind speed and direction that could be shown on the default "gauges.htm" page, and a moon image that was shown on one web page.  Early MX releases included a replacement set of template files that could generate web pages that (apart from trends.htm and gauges.htm) seemed similar to the legacy pages.  However, the earliest release did not generate any images at all, the generation of a moon image was added from release 3.5.0 (build 3071) onwards.


= Operating Systems for Cumulus 1 and MX =
Mark Crossley's Steel Series is used for the MX gauges page, replacing the Cumulus 1 gauges that were based on'''Web Dashboard Components for FreeWX and FreeWX-Wi''' (no longer available) and mixed images generated by Cumulus with plots drawn using JavaScript routines (before "Canvas" functionality).


As expressed by Steve Loft MX runs on:
MX beta introduced use of a set of .json files to convey the data for drawing charts to your web server, and in release 3.10.1 further .json files were added to support default web pages on your web server. The "legacy web pages" were also available in that release modified to use use the new .json files (instead of web templates), but preserving the same look as Beth Loft's web pages.


# Microsoft Windows operating system (Cumulus 1 only runs on this)
Whatever release of MX you are installing, the files needed for your web server are in "CumulusMX/webfiles" and its sub-folders.  The files that will be automatically uploaded to your web server are created in "CumulusMX/web" folder, depending on the release you are installing, which files are generated and which are uploaded will be determined by settings, later releases in general giving you more control over these files than earlier releases. You can't control the content of the various files, but from 3.10.1 you can individually select which are generated and which are uploaded.
#* To run MX on Windows, you need .NET installed which is included on Windows 7 upwards.
# Unix derivatives Linux and Mac OS X.
#* To run MX on the additional platforms, it requires the Mono runtime, and you will need to install this


From release 3.10.1, the new default web pages have a totally new look  (designed by Neil Thomas), and include responsive code allowing them to look better on wide screens and on narrow screens (such as those on smart mobile phones).


= Changing from Cumulus 1 to MX=
==Settings==


If you are going to run MX on  a different device to that running Cumulus 1, see [[Setting up Raspberry Pi]] article for details.  Although those notes are applicable to Linux operating systems, other Unix based systems, and those using the Mac operating system also need '''Mono''' to be installed. There is more information  about Mono in the main [[Cumulus MX]] article, and you should also read that.
The settings in Cumulus 1 and MX work differently, for Cumulus 1 you choose to save changes by clicking OK, for MX changes are only saved when you click a '''Save''' button if one is provided. If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved when you move to next field/line.


The remainder of this article was written assuming you are installing MX on the same Windows Operating System as you used for running Cumulus 1.  However, the files that you need to copy across from your Cumulus 1 installation to your MX installation remain the same whatever device you are installing MX on, although you may need to use File Transfer Process or a USB storage device to move the files to the new device.
==Library software==


== Cautions to be aware of should you already use library software prior to installing MX ==
Any non-technical person can skip this sub-section!


*If you have been using Cumulus 1, and you decided to customise your web site:
*If you have been using Cumulus 1, and you decided to customise your web site:
**you may already be using highcharts, jQuery, bootstrap, and other library software
**you may already be using one or more of highcharts, jQuery, bootstrap, and other library software
** and you may have selected to load latest versions for maximum functionality.  
** and you may have selected to load latest versions for maximum functionality.  
*You might be enhancing what is provided with Cumulus 1,  
*You might be enhancing what is provided with Cumulus 1,  
Line 174: Line 247:
**For Cumulus MX to work you need to ensure the versions of such software loaded for your web pages, are not used by your browser when viewing MX admin interface or MX web pages.
**For Cumulus MX to work you need to ensure the versions of such software loaded for your web pages, are not used by your browser when viewing MX admin interface or MX web pages.


*Equally, ensure that the versions of library software loaded by MX, do not stay loaded when you wish to view any of your web pages that use these libraries, as your desired functionality might be limited by the library versions MX uses!
*Equally, ensure that the versions of library software loaded by MX, do not stay loaded when you wish to view any of your web pages that require later versions of these libraries, as your desired functionality might be limited by the library versions MX uses!
 
== Files to copy from Cumulus 1 into where you are installing MX==
 
Subsequent sections will deal in more detail with various aspects like editing existing files, that are relevant to all the approaches below.


But first, let us focus on what to do to actually get the MX release distribution installed.  It does not come with a installer, it comes as a zip of all the files that are needed. So first we will summarise the MX install options for someone currently using Cumulus 1, then we will explain the approach in more detail both in respect of that install of MX and the incorporation of Cumulus 1 files.
==Three approaches to installing MX==


There are three approaches to installing a MX release distribution zip:


#Install MX OVER your Cumulus 1 installation (this only works if you want to run MX on same PC as you have been running Cumulus 1)
#Install MX OVER your Cumulus 1 installation (this only works if you want to run MX on same PC as you have been running Cumulus 1)
Line 195: Line 263:
#*This approach requires you to manually copy various files from old folders to new location
#*This approach requires you to manually copy various files from old folders to new location
#*MX requires all files from "data" and "Reports" folder created by Cumulus 1.  
#*MX requires all files from "data" and "Reports" folder created by Cumulus 1.  
#*You also need "strings.ini" (if you use that), and "Cumulus.ini", plus any other tailoring set-up files, batch processes etc. you might use.
#*You also need "strings.ini" (if you use that), and "Cumulus.ini_(Cumulus_1)", plus any other tailoring set-up files, batch processes etc. you might use.
#*This approach is generally easier if you want to be able to go back to running Cumulus 1
#*This approach is generally easier if you want to be able to go back to running Cumulus 1


Line 205: Line 273:


*Cumulus 1 will accept any character (except space, a digit, or the character used to separate fields in a list) as a separator for the date parts
*Cumulus 1 will accept any character (except space, a digit, or the character used to separate fields in a list) as a separator for the date parts
* MX will only accept the character defined in your locale (for a Windows pc that is actually set in Control Panel which Windows does not make easy to access preferring you to use its Settings app.
* MX will only accept the character defined in your locale (for a Windows PC that is actually set in Control Panel which Windows does not make easy to access preferring you to use its Settings app.
*There is a Clock and Region section in Control Panel and in the Region window you '''define the separator MX will use in the Short date item''', most easily by clicking additional settings).
*There is a Clock and Region section in Control Panel and in the Region window you '''define the separator MX will use in the Short date item''', most easily by clicking additional settings).
*Now ensure that is same character as used in al lines of your .txt log files.
*Now ensure that is same character as used in al lines of your .txt log files.
Line 220: Line 288:
Next obtain the MX distribution release as a zip as per previous option.
Next obtain the MX distribution release as a zip as per previous option.


Now unzip Cumulus MX into the original Cumulus folder. The new CumulusMX.exe should end up in same folder as existing Cumulus.ini, there will be other files in this folder, and there will be some new folders you have not seen before like '''interface''' and '''MXDiags''', but MX continues to use the existing sub-folders without any change of name.  
Now unzip Cumulus MX into the original Cumulus folder. The new CumulusMX.exe should end up in same folder as existing '''Cumulus.ini''', there will be other files in this folder, and there will be some new folders you have not seen before like '''interface''' and '''MXDiags''', but MX continues to use the existing sub-folders (like '''data''' and '''backup''') without any change of name.  


This saves you from copying any of your Cumulus 1 files, they just stay where they are and get used by MX.
This saves you from copying any of your Cumulus 1 files, they just stay where they are and get used by MX.


Where your existing [[Cumulus.ini]] file makes reference to local folders (in say Extra Web Files), those references stay valid.  The only edit you will make to configuration that affects Cumulus.ini is dependent on your weather station type, as explained later MX uses a different parameter to refer to the port used by the weather station.
Where your existing [[Cumulus.ini_(Cumulus_1)]] file makes reference to local folders (in say Extra Web Files), those references stay valid.  The only edit you will make to configuration that affects '''Cumulus.ini''' is dependent on your weather station type, as explained later MX uses a different parameter to refer to the port used by the weather station.


Be aware that MX changes the date formats in some files, so that Cumulus 1 can no longer understand them, so be cautious about copying log files (.ini and .txt) back to your Cumulus 1 back-up should you want to revert to using Cumulus 1.
Be aware that MX changes the date formats in some files, so that Cumulus 1 can no longer understand them, so be cautious about copying log files (.ini and .txt) back to your Cumulus 1 back-up should you want to revert to using Cumulus 1.
Line 243: Line 311:
#* your previous '''data''' folder contents into the new data folder created by unzipping, as before you might need to edit some log files
#* your previous '''data''' folder contents into the new data folder created by unzipping, as before you might need to edit some log files
#*your previous '''Reports''' folder contents (if any) into the new Reports folder created by unzipping
#*your previous '''Reports''' folder contents (if any) into the new Reports folder created by unzipping
#*your Cumulus.ini file goes into the top level folder, the one with ExportMySQL.exe, CumulusMX.exe and the .dll files,
#*your Cumulus.ini_(Cumulus_1) file goes into the top level folder, the one with ExportMySQL.exe, CumulusMX.exe and the .dll files,
#** Check that destination file, it must be  "Cumulus.ini" (ensure it starts with capital letter and all other letters are lower case)
#** Check that destination file, it must be  "Cumulus.ini" (ensure it starts with capital letter and all other letters are lower case)
#** In the new "Cumulus.ini" edit any lines that made reference to the old windows location. Remember, that only windows uses "\" in paths, and your new device will require reference to new locations. Even if you are still using windows, you may need to make changes to reflect that the files are now in a new location.  
#** In the new "Cumulus.ini" edit any lines that made reference to the old windows location. Remember, that only windows uses "\" in paths, and your new device will require reference to new locations. Even if you are still using windows, you may need to make changes to reflect that the files are now in a new location.  
Line 250: Line 318:


The big advantage of this approach is that anytime you are not running MX you can go back to Cumulus 1 and let it run from where it left off (subject to availability of past data in your weather station, and possibly unplugging weather station from one device and plugging it into PC). Obviously you cannot run Cumulus 1 and MX at the same time accessing the same weather station (even if both are on same device and so no unplugging and plugging in is needed).
The big advantage of this approach is that anytime you are not running MX you can go back to Cumulus 1 and let it run from where it left off (subject to availability of past data in your weather station, and possibly unplugging weather station from one device and plugging it into PC). Obviously you cannot run Cumulus 1 and MX at the same time accessing the same weather station (even if both are on same device and so no unplugging and plugging in is needed).
== Editing files==
Do remember that with Cumulus 1 on Windows, each line in every file ends with both carriage return and line feed. If you are moving to MX on a Mac, each line should end with just a Carriage Return. For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed. There are various editing tools that can do the necessary modification of all lines for you in just a couple of clicks; one such tool is Notepad++ (which although mostly used on PCs, can be used in other operating systems).
===Cumulus.ini===
If you are running MX on Windows, filenames are not case sensitive, so MX will recognise "Cumulus.ini" or "cumulus.ini".  But although Cumulus 1 might recognise "Cumulus1.ini" or anything else other than "Cumulus.ini", MX only recognises "Cumulus.ini".
If you are planning on running MX on another device, filenames are case sensitive, and your Cumulus 1 configuration file must be called "Cumulus.ini" with an initial capital followed by lower case, so it might need to be renamed.
If you read the [[Cumulus.ini]] article, you will see that some parameters in the configuration file are only used by Cumulus 1, some only by MX, and some apply to both. MX will ignore any Cumulus 1 parameters it does not recognise, so you don't need to edit out lines. MX will also add any parameters it needs as read/write parameters for its settings, so in general you do not need to add them manually, you can simply set them using the relevant page in the settings part of the admin interface. There are of course a number of read-only parameters (these are ones that are entered directly into the configuration file and cannot be edited using settings), and there are some of these that are Cumulus 1 only and some that need to be added manually if you want to change the default.
There are a few exceptions to the above paragraph, if you look at [[Cumulus.ini#Parameters_changed]] you will see the list of a few parameters that '''may need to be changed''' in this configuration file (it depends on what type of weather station you use). But you don't need to look them up as they are also mentioned in next 2 sections (station settings and web pages) below.
There are also some differences in how some parameters in this file are used in the 2 flavours.
One to remember now is to check the "StartDate=" line in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time, as '''CumulusMX.exe''' uses that to find the first log file to start reading from, it will ignore any log files for earlier dates; whilst Cumulus 1 and '''ExportMySQL.exe''' can process all log files in the data folder as they look for file name pattern and do not check dates.
It is advised that you work through all the MX settings screens once you have the [[MX Administrative Interface|Admin interface]] working, ensuring they do represent how you want MX to work.  Some of the settings are for functionality you did not have in Cumulus 1 versions, and for these you will need to look up the link earlier in this sub-section.
For the read-only settings in Cumulus.ini, you may need to read the documentation in the forum.  The Cumulus.ini article in this wiki has had read-only parameters added to it now, but it is impossible to know if all have been covered there.
== Station connections==
If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in Cumulus.ini in the station section as Port=n. In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings screen, but that is stored within Cumulus.ini in the station section as Comport=tttttttt. If your old number was 3, and you are still using Windows, the new setting would have value of COM3, for other devices it might be /dev/ttyUSB0.
== web pages==
There are differences in how to upload the standard files between Cumulus 1 and MX.
It is best to work through the Internet settings screen in the MX admin interface. '''INFORMATION HERE MIGHT BE MADE OBSOLETE BY MX DEVELOPMENT AFTER 3.7.0'''
Consequently, there is a difference between the entries in Cumulus.in,
* for example remove the IncludeSTDImages=1 used in Cumulus 1
*and replace it with IncludeGraphDataFiles=1 used in MX.
*whilst the former included the moon image for Cumulus 1, the latter does not affect the moon image for MX
If you have been using the Cumulus 1 supplied web pages, you will find they do not work with MX, there is a new set provided with MX that work slightly differently.
*  Cumulus 1 uploads a number of images, these include a couple of images that combined show the moon phase, and a number of graphs are also uploaded as images (used on Cumulus 1 trends page).
*Prior to release 3.6.0, Cumulus MX does not upload any images
*From release 3.6.0, Cumulus MX can upload a moon image (again derived from another base image), as an option (2 settings, one to generate, another to upload)
* Cumulus MX uploads a series of .json files that hold time and value pairs that can be used to draw graphs. (Hence the difference in settings mentioned above). Thus the trends web page works totally differently.
* The gauges page provided with MX is based on Mark's implementation of steel series, so it is different to the old "Web Dashboard Components for FreeWX and FreeWX-Wi " that Cumulus 1 used.
*  The other web pages look the same, and indeed are effectively functionally same. You might see some derivatives on MX pages that were not seen on Cumulus 1 pages.
= What I did to Install MX=
I have used Cumulus 1 for a decade or so, and been very happy with it, but I wanted to give MX a go without affecting my Cumulus 1 installation.
Here is exactly what I did on my ex NHS Windows 10 Pro PC, step by step; I am hoping this list might help some readers.
Can I just stress I downloaded version 3.4.5 (build 3069), there may be some changes that affect what I record below in more recent versions, I just noted what I had to do at that moment in time (March 2020).
Do check, as the version of MX you choose to move to may have more or less, folders and files, included in the package.
== Download and unzip==
#I downloaded the CumulusMXDist3069 zip from the Current Release section on the downloads page. Although 3069 is no longer the latest distribution, still use the same link as it will always give you latest available.
# I unzipped the contents into a partition I use just for software downloads.
#*  You don't have to have somewhere separate to the installation, but many people will want to download on a separate device to the device where they will install MX
#*  Downloading to somewhere other than where you will install it, also makes it easier if you do any customisation of particular files and wish to keep copies of originals.
# I used a package (no external software tools named here) that verifies the files it copies to duplicate the folder CumuluxMX within the zip onto the drive where I wanted to run Cumulus MX.
#  As I was not intending to use the web pages that come with MX, so I selected to exclude in this verified copying everything from the "\CumulusMX\webfiles\*.*" folder, and its sub-folders in the download.
#* If you intend to use the standard MX web pages, then you will need to:
#** Ensure you do copy these files from download into where you are installing MX.
#** You will also need to FTP all files and sub-folders within the webfiles folder (not the actual folder itself), to whatever folder on your web server you specify as the directory in the Web/FTP site part of the settings for MX.
== Copying Cumulus 1 files into MX folder==
The locale I used for Cumulus 1 is going to be the same I will use for MX (same PC!) so my copying across of my existing files was easy:
# First, I copied my \Cumulus\strings.ini to \CumulusMX\strings.ini. This preserves any tailoring I have done of terminology.
#* Remember, "\Cumulus\web" and "\CumulusMX\web" have different content, so don't do any copying between these.
# Next, I copied my existing Cumulus 1 alarm sounds in "\Cumulus" across to MX folder "\CumulusMX\interface\sounds" as these were referenced in my main Cumulus.ini file (I will edit that file next to reference the new location of these files).
#  Now, I copy my \Cumulus\Cumulus1.ini (don't worry why my file had a "1" in its name, just remember that yours probably won't) to \CumulusMX folder as "Cumulus.ini". I then edit the MX "Cumulus.ini" file:
#* In the [Alarms] section (your Cumulus.ini may have sections in a different order to mine) I edit all the parameter lines where the attribute ends in "File" to reflect their MX location in the sounds folder (there is no such folder in Cumulus 1).
#* I checked that I had a "[FTP site]" section (yes, mine was named correctly with second word all lowercase, but if your has [FTP Site] you will need to edit that section title to put second word entirely in lower-case)
# I now had to decide whether
## I would use the Extra web files settings page in the MX admin interface to make the changes, in the local file column, necessary to define which web templates I wanted MX to process
## Or I would make the changes myself by editing the appropriate lines in this [FTP site] section. (I decided on this option).
#* Although my web templates are stored in a folder "Templates for Cumulus to Process" outside the Cumulus 1 or Cumulus MX folder structure, I had to reference a new set of template files to cope with differences in the output parameters of MX web tags.
#* Next where I want a transfer done only at end of day, I added lines like "ExtraEOD19=1" (when you run MX it will add a ExtraEODnn for each of the 100 possible values of nn that you have not created) and changed the previous "ExtraRealtime19=1" to "ExtraRealtime19=0" (I previously used realtime as that was only way in Cumulus 1 to ensure a template file was processed so it held correct values as close to end of day as possible, but it inefficiently also made huge numbers of unwanted transfers during the day). I had 9 such files being copied far too often, so those 9 changes will cause a huge reduction in processing load! I know I could also do this later using the MX admin interface, but it makes sense to me to do it as I am working through the file.
#* I have a few files that are PHP scripts written as Cumulus templates; each PHP script has a number of PHP variables being set equal to a Cumulus web tag. You can find out about these scripts at Php webtags.
#* I also have a few Cumulus templates that, like the standard ones I don't use, generate web pages and embed a lot of Cumulus web tags. I have had to edit these templates as in some cases the web tags have date/time output format parameters.
#*  I mentioned some of my web pages are generated from my own Cumulus templates. Despite now having Cumulus 1 and Cumulus MX templates with different names, these still both generate remote web pages given the same names, so I don't need to alter any navigation between pages on my web server.
#*  In the "[NOAA]" section I had MonthFileFormat="NOAAMO"MMMyyyy".txt" on a line, I checked at Webtags#The_format_used_for_naming that 'MMM' and 'yyyy' were valid in Cumulus MX as well as in Cumulus 1, they were, so no edit needed either to that line or to "YearFileFormat="NOAAYR"yyyy".txt" line.
# In the download, there is a file "\CumulusMX\Reports\.gitignore" provided. In this same MX folder, I copied in the 200 or so report files I currently have in "\Cumulus\Reports". Remember you don't need to rename these files, it is only how that naming is specified in the settings that might need changing.
# Next I see there is a "MXdiags" folder, so I don't copy across the Cumulus 1 "diags" folder. Equally I don't change anything in the new MX "webfiles" folder, but I do copy across to my web server the "cumuluscharts.js" and "logoSmall.png" files I see there that I have not seen before.
#  So left to last is the "data" folder:
#* I have copied all files from "\Cumulus\data" to "\CumulusMX\data" (except log.xml as MX uses a different file and MX does not provide a way to read the XML file into its diary.db).
#* Some of my .ini contain date-time entries like "12/03/2019 14:50:45". I'm assured MX can read these, although it changes a date time like that to "2019-03-12T14:50:45", but I am waiting to see if that is true. I use the full stop character for all my decimal points, but if you use decimal commas, you might find you need to do some editing of your .ini files in this data folder before MX can read them.
== Aside on .ini files in data folder==
I find after running the CumulusMX engine, that it edits those .ini files it needs to, and the new versions contain date-time entries in the "2019-03-16T12:45:00" style, but despite what I worried about from reading on the forum, I found you don't need to edit beforehand these entries in log files like month.ini as Cumulus MX can read the old formats like "12/03/2019 14:50:45" as well as the new year first formats.
== Creating the shortcut to run MX==
I right clicked on the "CumulusMX.exe" entry in the top level folder and selected Send to ... desktop (create shortcut).
*    Ensure that your shortcut has a Start in: defined as the folder where your executable is stored. This is the most vital setting.
*    You can optionally change other shortcut properties. I selected to use different settings in the colours tab, so when I have that terminal/console window open I don't confuse it with any other command window I might open. I selected to change options in the layout tab to position the terminal/console window on my second (smaller) screen. I also selected the Start minimised option, as I don't need the window for this MX engine taking up screen space all the time.
*    I have renamed my shortcut to "MX_run" so I can recognise it as different to the folder name, as I have also created a shortcut for the folder.
*    On the same right click menu I also selected Pin to taskbar.
*    When I am happy with MX, I will copy the shortcut into C:\Users\Personal\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\ and then MX will start automatically after I (or Windows Update) reboot my PC.
== One-off extra installation steps==
#  I clicked on one of my new shortcuts. Windows Defender popped up, so I told it allow all for Cumulus MX.
#    I opened my Internet Security premium package, in the "unrecognised files" section I selected CumulusMX.exe then right clicked and selected Change File Rating to Trusted
#    In the "Firewall" section of the Internet Security package I added port 8998 as one that was permitted.
#    I viewed my hub (router) to see the IPv4 address allocated to my Cumulus MX computer (192.168.1.64), that told me that I would find the user interface by typing "http://192.168.1.64:8998/" into my browser while the MX engine command window remains open (so MX is actually running), so I typed that and I saw the user interface and navigated around it.
#    I right clicked on my desktop (you may need to right click on the windows symbol at bottom left), selected command prompt as administrator that opened a command window where I typed  <tt>netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users</tt>
and I got the response "URL reservation successfully added", so I know it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way.
IMPORTANT NOTE: I don't use "localhost:8998" for two reasons, first I already have a web server on my PC at IPv4 "127.0.0.1" using "localhost" as an alternative name (and port 81, selected because something called 'Skype' that I don't use had reserved port 80 that I had expected to use), and second using the IPv4 exact "http://192.168.1.64:8998/" address as a bookmark, I can view the Cumulus Admin Interface on my mobile phone which shares bookmarks with my PC and connects to my LAN via wifi.
== End of my test of MX actions==
When I am happy to stop using MX, I type Control + C into that MX command window on my PC and MX closes.
== Testing MX while still using Cumulus 1==
Obviously, you cannot have Cumulus 1 and Cumulus MX running at same time, accessing same weather station.
If you (like I was to begin with) are just experimenting with MX you may sometimes run one flavour (say MX) and sometimes the other (Cumulus 1).
Each time you swap, you must copy all the updated log files from the just used data folder to the data folder you are about to use when you close one favour before you start the other flavour, or you must have both executables in same top level folder to force them to share the data folder.
Please note, today.ini is a special case, the time-stamp line has a different format in Cumulus 1 (C1) and MX; while MX can read the format that C1 uses, C1 cannot understand the format that MX uses. Remember today.ini determines which stored entries in the weather station console need to be read to "catch-up" and it also holds the various figures that will inform what gets stored in dayfile.txt at the end of the day. So to have this file being read and updated correctly is vital.
If you don't do ensure both flavours use the same log files (.txt and .ini), various derivatives (e.g. Chill Hours) will become wrong and you may have conflicting rows in dayfile.txt (because its content is generated from what that flavour saw when it was running the previous day) and generally this will be particularly evident in any weather parameter that varies a lot like the wind vector (speed and direction). It also affects what is stored for any derivatives that rely on averaging (temperature, wind run, rain rate) as these are calculated biased towards the actual times when that flavour of Cumulus was actually running, so you can have issues if you run the 2 flavours in different folders/devices as if the other does not exist.
I have some batch scripts that Cumulus initiates, and a number of Cumulus templates, and in my case I had to be happy these were working before I stopped using Cumulus 1, and got MX as the flavour that auto-started on switching on my PC.
I also use output modifiers on a lot of the web tags I use in my custom web pages and it took me a long time to work out the necessary changes to get these templates correctly edited so that MX could process them and produce the web page content I wanted. I am not going to explain all the problems nor give the solutions, because you probably don't have web pages as complex as mine.
2.4.8 My migration from Cumulus 1 to MX
My installation of Cumulus MX has succeeded, and as the experiment did help me find a mistake in one of my Cumulus templates where I had not defined input parameters for "Recent History", it has been useful. Initially, I am continuing to use Cumulus 1 for the moment, until I am absolutely sure MX can do everything I want.
I believe MX will do some tasks better, but there is a lot more to learn about how to use MX. MX does lack some features that I used in Cumulus 1. I found the View Period screen in Cumulus 1, where you could look at any day, week, month, season, or year in the past extremely useful. MX does not have such functionality yet.
While I was using Cumulus 1, I found with my Chas Olsen Fine Offset I had to define EWpressureoffset=x.y otherwise Cumulus 1 frequently failed to read the correct pressure. In implementing MX, I decided to try without that line in the new Cumulus.ini file; I have not had any problems either when I first ran MX or indeed 2 months later when it is more than a month since I last ran Cumulus 1. Consequently, if you used to have pressure reading problems, you might find you don't with MX.
Some days after I first started trying MX, I have tried out more MX features, been happy with those, and as MX is now doing all I want I have stopped using Cumulus 1.
I still use my own PHP script to update my database tables, I tried the custom SQL and it does not do all I want.
I have done a little editing of the user interface, partly to discover how easy it is to edit, partly to understand better how it works. For each file I have edited (HTML or CSS) I have kept copy of original and made a second copy of my edited version, so I cam easily go back to original and if I download a new release I won't lose the copies of my edited versions of files.
You can judge my progress with trying features, because elsewhere in this Wiki I have expanded the text for those features I have tried and therefore understand.

Revision as of 09:07, 6 April 2021

Introduction

This page was inspired by this update from Cumulus 1 forum topic. That post was made in January 2020, and therefore the bulk of the text on this page relates to Version 3.3.0, which was the MX release that was in use at the time.

Crystal Clear info.png This page was last partially updated for the MX release in July 2020; that is no longer latest!

Appeal to contributors: Please work through all MX release announcements and work out all the many updates needed for this page, it may even need a redesign for more recent releases!


You might want to read the related page at My_migration_from_C1_to_MX which describes how this worked in practice for release version 3.4.5 (build 3069).


Should I migrate to MX or not?

The legacy Cumulus will never be updated, and as more people migrate to MX there will be fewer people to help you if the legacy cumulus software gives you a problem. MX has a major advantage it runs on more devices, and for example a Raspberry Pi computer is far cheaper to leave on all the time to run MX (knowing it will never be rebooted by Windows updates). The current developer of MX offers support, and other people are also gaining expertise, so if you have a problem you will get help.


Of course MX started with very little of the functionality that the original Cumulus offered, but over time MX has gained more and more functionality. Features in the original Cumulus have been gradually added to MX, but also MX offers a lot that is not in the original Cumulus. See Compare C1and MX page for more on this. Unfortunately, there is no list anywhere of all features in Cumulus 1. About_Cumulus does not cover all features. Nor is there a list of features still to be introduced into MX. Therefore it is hard to help you understand whether functionality changes between the flavours will suit you.

There are Cumulus users who tried MX, then have migrated back (from MX) to the legacy product; either on a temporary basis (to use a missing feature before they return to MX) or in a very few cases decided to stay with Cumulus 1 (because they prefer it). It is worth noting here that the legacy Cumulus is a stable product with very few bugs, but MX is constantly being developed and thus likely to contain bugs (these obviously vary depending on release as some are fixed and others appear). To put this in context, MX now has a huge amount of functionality and it is difficult for the developer to test all the different features.

Cumulus 1 creates a special log containing just temperature and humidity values, this is NOT available in MX.

Although from version 1.7.5 onwards the original Cumulus counts air frosts and gale days, this feature is not in MX.

The This Period screen that is a popular feature of the original Cumulus software as it tabulates data simply, is not available in MX.

Migrating from original Cumulus to beta MX

The beta builds (version 3.0.0) of MX were designed to make it easy to migrate. This section is based on Steve Loft's wording taken from the support forum with minimal alteration for its new context.

You can move between [Cumulus flavours] fairly easily, but you should really read all the guidance.

  • In particular if you use decimal commas with Cumulus 1, then you MIGHT have issues when MX tries to read existing log files.
  • [For separator characters in dates and lists] whilst Cumulus 1 only takes settings from the control panel in Windows,
    • MX running in a Windows environment, takes settings from the locale you specify in a parameter when starting MX, or the Control Panel in Windows,
    • MX running in a non-windows environment takes settings from the locale you specify in a parameter when starting MX, or the default locale in your Mono installation
    • MX [may] take some [settings from] interaction between the two [the specified locale and the default locale], and MX may struggle to read file [lines] created by Cumulus 1 if the MX locale is not precisely same as settings when that file [line was] created.

While MX was in beta, there was limited documentation about what features were included and why, there was some fault reporting and a tracking list that showed when some of those issues were fixed, also initially the documentation on how MX had implemented features it did have was very sparse. The lack of a list of features in Cumulus 1, meant it remains difficult to track which Cumulus 1 features are or are not implemented in MX. Steve Loft said parts of MX were simply machine code level copies of parts of Cumulus 1 functionality, and parts were trying to offer better functionality, but he never said what was included in these two categories.

There was a list of enhancements requested by users for Cumulus 1, but that list was deleted before work on MX started. Despite that, it does seem that some features that were on the now lost list of enhancements for Cumulus 1 that never got implemented in Cumulus 1, have been implemented in MX, although again there is no definitive documentation.

How to install MX

The basic instructions for installing MX:

  1. Download the MX distribution (from Software for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version)
  2. Unzip into the location where you are going to install MX.

Alternatively, follow instructions at Raspberry Pi Image which describes a simpler way to install MX (assuming you have another device linked to your RPi to do downloads and micro-SD card preparation).


Migrating from legacy Cumulus to a recent build of MX

The text that follows mostly relates to version 3.3.0, it probably needs to be updated, as later releases of MX have massively changed. The newer releases are more fussy about formats used in files, and migration from the legacy Cumulus software is harder

If you want to continue using the same weather station, and you are not moving to a different home, you will want to maintain the data you collected using Cumulus 1 in your MX installation. This is what is meant by migration, and is the subject of the rest of this article.


'Engine' and 'Admin Interface' for MX

When you run the original Cumulus software it displays a screen (see Cumulus_Screenshots#Main_Screen, and from there you navigate to other screens to view data or change settings.

MX is different, it consists of a stand-alone 'engine' which performs the reading and logging of data, uploading to a web site etc. This 'engine' is a command-line/terminal/console application which has no user interface. It does write diagnostic information to a diagnostic log, but many people run MX on a device that has no monitor and so the terminal output (if any) is not monitored.

When you successfully start MX, the engine is running, and it continues, until it is terminated by control C (or its equivalent in a Mac environment).

The separate admin interface is provided by virtue of the engine acting as a web server. You can view the admin interface by typing the URL of the built-in web server into your browser, either on the same machine, or on a separate machine sharing the same local network. If you install MX on Microsoft Windows, then a few extra one-off steps are needed to allow this web server functionality:

  1. Windows Defender has to be told to allow all for Cumulus MX.
  2. Typically for other security packages, select "CumulusMX.exe" then right click and select "Change File Rating to Trusted"
  3. In any "Firewall" package, add port 8998 as one that was permitted.
  4. Using "command prompt as administrator" type netsh http add urlacl url=http://http://192.168.1.64:8998/ user=\users, the response "URL reservation successfully added" means it worked. This command is apparently to allow all users to bind to port 8998 (i.e. that used for the Cumulus interface). This also means you don't have to run the engine (CumulusMX.exe) in an administrator user, nor select "Run as administrator" from right click menu on the shortcut, nor set the properties for any shortcut to run in any special way.
The default URL if the browser is on the same machine as MX is http://localhost:8998/. 

If you are using the browser on a different device on your local network to the device running MX, you cannot use that local host shortcut. Instead you specify a IPv4 address, that is listed in your router (might be called a hub) for the device running MX, this IPv4 address will look like '192.168.y.z' (where y and z are numbers that vary between implementations). For example you might acces the admin interface by typing "http://192.168.1.64:8998/".

Equally, if "localhost" is already in use for another web server (that you already run on your device), unless you can differentiate purely by port number you may need to use the correct IPv4 address as above, even on the same device.

For security reasons, the admin interface should not be accessible via the public internet.

Configuration file

MX's Cumulus.ini has different content to the legacy cumulus.ini.

Cumulus 1 can recognise, in some circumstances, "cumulus1.ini", and other variants, not just "Cumulus.ini". MX only recognises "Cumulus.ini".

The old approach for migration, was to copy across your existing file, and let MX ignore all the parameters that do not apply to it. To add the parameters that MX does need, you would then go to the MX_Administrative_Interface#Changing_Settings pages and work steadily through ALL the options. For any parameters that were not set by the admin interface (and there were many "read-only" parameters in early releases of MX), one assumed these also existed in Cumulus 1 and so were already in your file.

At releases like 3.3.0 and 3.6.0, amongst others, there were changes to "read-only" parameters for MX. These could not be adjusted through the admin interface, and needed to be entered manually. To keep this page simple, it cannot list the "read-only" parameters that you might need to add for migrating to earlier releases.

It appears that from release 3.10.1, all settings are now initialised via the admin interface (and none are "read-only"), so if you are using that release or later, it may be better to ignore your Cumulus 1 configuration file, and to let MX create a configuration file with just the parameters it needs using the various [MX_Administrative_Interface#Changing_Settings|settings pages in the admin interface]]. That will ensure you don't get muddled by parameters used for Cumulus 1 (but not for MX); and you do have all the parameters you do need, set correctly.

Start date

When you first run any Cumulus software (whatever flavour) a parameter is added to the configuration file that documents the date Cumulus was first used.

For Cumulus 1, this parameter appeared in two places on the example web template for all-time records, but was otherwise ignored. Thus if you decided to import into your data logs readings from before you first ran Cumulus, the legacy software would be able to find those earlier records.

CumulusMX.exe uses this parameter to determine the first standard log file to start reading from, it will ignore any log files for earlier dates. Thus if you were to migrate your Cumulus 1 configuration file into MX, you should check the "StartDate=" line in the '[Station]' section is correct for your earliest data before you let MX read this configuration file for first time. If you need to make any edit, ensure you stick to exactly the same date format.

From release 3.10.1, MX allows you to edit this start date within the admin interface. It is "hidden" as an "advanced" setting, with a strict danger warning, but it is there!

Uploading to web site

Many settings have changed in this area for MX, and they vary depending on which release you are installing. No attempt is made to explain further here.

Station connections

If your weather station used a port to connect to Cumulus 1, that port was set on the settings screen as a number and stored in Cumulus.ini_(Cumulus_1) in the station section as a parameter in the format Port=n.

In Cumulus MX, as it runs on various operating systems, the port is specified using text (instead of a number), again you select it within settings, on Station settings page, but within Cumulus.ini in the station section the parameter is in the format Comport=tttttttt.

If your old parameter had a value of 3, and you are still using Windows, the new setting would have value of COM3.

A typical parameter value for other devices might be "/dev/ttyUSB0".

RG11 Rain gauge

If you use a RG11 rain gauge:

  • Replace: RG11port=n and RG11port2 =n (Cumulus 1) where n is a number, by RG11portName=xxxx and RG11portName2=yyyy (Cumulus MX on Windows) where the value is a string with values as per previous paragraph depending on device on which Cumulus MX is running.

From release 3.10.1, there appear to be other new parameters, not yet documented.

Strings.ini

This is another configuration file, but it is an optional one. If you have not created a strings.ini file in your (leagacy) Cumulus top level folder, then you have no file to move to your MX installation, and you should skip the rest of this sub-section.

The contents of the samplestring.ini file you get in your MX release distribution varies depending on the release you have downloaded. Check your existing strings.ini file against the samplestring.ini file in the MX distribution you have. If the attribute names (left hand side of the equals sign) match for the parameters you selected to include in your strings.ini, then you can reuse your existing file. If your file includes attributes that are no longer in the MX samplesting.ini file, then you will need to edit your strings.ini file that is placed in the folder containing CumulusMX.exe.

Please remember that the Microsoft Windows Operating System is case insensitive for file names, if you install MX on a Windows PC, then "Strings.ini", "STRINGS.INI", and "strings.ini" are all treated as the same file by MX. If you install MX on another operating system, then the file system is case sensitive, in this case MX will only recognise "strings.ini".

For those of you who are more technical:

  • Note that files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems like all Linux variants including Raspberry Pi Operating System). Apple Mac are again different in using just Carriage Return.
  • This should not cause any problems for your "strings.ini" file as MX does not care if there appear to be some extra blank lines (because the carriage return may be treated as end one line and the line feed as ending a separate blank line on non-Windows devices).

NOAA style reports

The generation of reports is an optional feature, if you have never used it your (legacy) Cumulus Reports folder will be empty, then you have no files to move to your MX installation, and you should skip the rest of this sub-section.

Please see Reports_folder for full information. Cumulus software creates reports, it does not edit existing reports, so migration is simple. Just copy the contents of the Reports folder in your original Cumulus installation into the folder in the new installation.

For those of you who are more technical:

  • files created in Microsoft's Windows Operating System use two characters (carriage return and line feed) to end each line, while all other operating systems use a single character (line feed in most Unix derived systems). Apple Mac are again different in using just Carriage Return. This should not cause any problems.
  • files can be encoded (how individual characters are represented by binary codes) in different ways. There is more about encoding at Reports_folder#Encoding, the relevance here is that if your MX settings and Cumulus 1 settings use different encodings you may find some characters (e.g. degree symbol) do not appear correctly when viewing some of your reports.

"data" folder

The contents of your Cumulus 1 data folder (log files ending with extension .ini or .txt) can NORMALLY be read by MX (but see points listed below).

However, if you are not running MX on a Microsoft Windows computer, ideally you should change the characters appearing at the end of every line in any file you move from Cumulus 1. To do this, run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows, but can run on other operating systems. In its Edit menu, choose EOL conversion. Do remember that with Cumulus 1 on Windows, each line in every file ends with both carriage return and line feed. If you are moving to MX on a Mac, each line should end with just a Carriage Return. For all Unix-based operating systems (Linux, Raspberry Pi OS, and other variants), each line should end with just a Line Feed.

I say ideally, because although Microsoft is fussy, and determined to be different, most other operating systems can tolerate different line terminators. If you use PHP Hypertest Pre-processor anywhere on your installation, you will discover that is written for "Line-feed" as line terminator and any "carriage return" in your files may mess up the content unless you code in a "trim" to remove it. Hopefully, if you use PHP, you are technical enough to realise you may need to edit the code depending on which device it is being run on!

Weather Diary

As explained on weather diary page, the Cumulus 1 weather diary uses a different file to MX, and indeed takes a very different model for how to store the information.

Therefore don't copy Log.xml file into the data folder of your MX installation. You will need to manually use the admin interface Edit menu to access the new diary and add your previous entries one by one.

Please note the editing interface has been changed from release 3.10.1, but the page linked to above may not have been updated. A major change in the upgrade relates to handling of time-zones.

Please also note that the Cumulus 1 weather diary permitted multiple entries to be stored for an individual day, but the MX implementation only permits a single entry per day. Also the configuration file defines a time when the entries stop applying that may not match the rollover time for other weather measurements.

dayfile.txt

date format issue

Cumulus 1 allows the date that appears in the first field of each line to use any character (except space) to separate the day of the month, from the month, and from the last two digits of the year. Consequently "29/03/88", "29-03-88", and "29.03.88" are all acceptable in the legacy software (and indeed in some early MX releases).

For MX (from release 3.5.4 onwards), the date separator specified for the locale when you run MX must be used in all lines of this file.

  • See MX_on_Windows_OS#Parameter_for_changing_Locale for how to specify locale if you are running on Microsoft Windows (by default the locale settings are taken from the standard Windows settings application or Control Panel).
  • If you are running MX in a Linux operating system, the locale parameter can be used, but the default locale is determined by the mono-complete package, and that in turn depends on your device settings.

So when you migrate "dayfile.txt", run it through an editor designed for programmers. Various editors are available but "Notepad ++" is one that is popular on Windows. That editor has a "Search" menu, with a Replace... option within it. In that option, there is a "Find what" box, a "Replace with" box, and a "Regular expression" selection.

Suppose your latest lines use either "." or "-" as separator, and that is what MX expects, but you have some older lines using "/" as date separator. Since you know "/" does not appear anywhere but in the dates with an older format, putting "/" in find what box and either "-" or "." (as required) in replace with box will sort you out after you hit "Replace all".

It is all a bit technical, if MX expects "/", but you have "-" in some older lines. The complication comes because "-" may be used in value fields too, so you need to find a way of specifying a minus with two digits before it and two digits after it. For this correction you need to select Regular expression and then set Find what to "^([\d]{2})-([\d]{2})-" and Replace with to "$1/$2/". Please check this, as this was copied from a forum post by Mark Crossley and I have not verified it.

time-stamp format issue

The file contains time-stamps following any value that represents a highest or lowest in the day. These must be expressed as hour and minute, and normally a colon (":") is used as a separator, but again the legacy Cumulus does not actually insist on that while MX does insist that are time-stamps are in "HH:mm" format.


value format issues=

If you moved your Cumulus 1 installation from one windows pc to another, it is just possible that you might have a mix of "decimal comma" and "decimal point" in your values, or you might have changed the field separator (normally ";" or ","). Again, these must be consistent in all lines for mX, and must match what is defined in the locale used.

other .txt files

These should migrate without any problems. You will find that MX creates at least one file that was not in the older Cumulus (monthlyalltimelog.txt).


the .ini files

When Steve Loft designed his original Cumulus (1), he had no experience to draw upon as to the best way to treat items like dates. He wrote the software originally just for his own benefit and did not need to worry about time zones. Subsequently as he enhanced his software to make it usable by others, he faced many issues on how to cope with different time zones, and different weather stations having different sensors. In Cumulus 1, he basically focussed on compatibility by keeping to his original design for the data log files (both those ending in .ini and those ending in .txt, and only adding extra fields to the end.

When Steve Loft took a new look at the data log files for Cumulus 2, he started with a new design, the principal change was that he decided to use UTC for all fields in them that reference dates and times. Steve struggled with Cumulus 2 largely because he was (at that time) not familiar with the C# language he was later to use for MX. It is fair to say that using UTC was also a major factor in his failure to get Cumulus 2 to provide the functionality he had offered in the original Cumulus.

When Steve Loft designed Cumulus MX, he was able to learn from his experiences with both Cumulus 1 and Cumulus 2, and he decided to use dates to an ISO specification (ISO 8601 Data elements and interchange formats – Information interchange – Representation of dates and times), but in the local time zone of the particular user, and therefore log files are not backwards compatible. However those MX beta builds were designed to be forward compatible in that all date formats used by Cumulus 1 could be read, although MX would when rewriting any date change them to the new format.

For the .ini files, Steve Loft did make one change in his beta MX, all values now had to use decimal points in these files, and decimal commas were no longer valid. Thus to migrate to earlier MX releases, if you had used decimal commas, you needed to edit each ".ini" file and change those commas to full stops. Equally, if you had any times not using a ":" as separator, this had to be rectified.

Apart from that, in all versions from 3.0.0 to 3.5.y, there was forward compatibility, and MX could read any file created by the original (legacy) software. MX releases from version 3.0.0 to 3.5.4 can read all the date/time entries in any file whether in the Cumulus 1 format or in the MX format. MX will change any dates to the ISO format only when it needs to update that particular date/time. Consequently, within a single file both formats will co-exist, you may see lines using Cumulus 1 format for the extremes that have existed for a while, and Cumulus MX format for any new extremes.

It appears that releases at versions 3.6.x onwards have become more fussy about existing content in these files due to changes in the source code. However, the developer has not documented whether files from Cumulus 1 can still be read, forward compatibility has only been guaranteed with earlier MX releases. That said, some people are successfully migrating from Cumulus 1 to the newer MX releases, so it works for at least some old files.

General points

File names

Note that if you run MX on a UNIX based operating system (e.g. Linux or Raspberry Pi OS) all file names are case-sensitive, please read documentation to see where capital letters are required in those file names. Be aware that wiki pages change first letter to a capital even when a file that must be all lower-case is being described.

web server

Steve Loft's Cumulus 1 came with a set of example template files (designed by Beth Loft) that could generate web pages to upload to your web server. It also produced a set of images representing standard graphs that could be uploaded to your web server and shown on the default "trends.htm" page, a set of images representing wind speed and direction that could be shown on the default "gauges.htm" page, and a moon image that was shown on one web page. Early MX releases included a replacement set of template files that could generate web pages that (apart from trends.htm and gauges.htm) seemed similar to the legacy pages. However, the earliest release did not generate any images at all, the generation of a moon image was added from release 3.5.0 (build 3071) onwards.

Mark Crossley's Steel Series is used for the MX gauges page, replacing the Cumulus 1 gauges that were based onWeb Dashboard Components for FreeWX and FreeWX-Wi (no longer available) and mixed images generated by Cumulus with plots drawn using JavaScript routines (before "Canvas" functionality).

MX beta introduced use of a set of .json files to convey the data for drawing charts to your web server, and in release 3.10.1 further .json files were added to support default web pages on your web server. The "legacy web pages" were also available in that release modified to use use the new .json files (instead of web templates), but preserving the same look as Beth Loft's web pages.

Whatever release of MX you are installing, the files needed for your web server are in "CumulusMX/webfiles" and its sub-folders. The files that will be automatically uploaded to your web server are created in "CumulusMX/web" folder, depending on the release you are installing, which files are generated and which are uploaded will be determined by settings, later releases in general giving you more control over these files than earlier releases. You can't control the content of the various files, but from 3.10.1 you can individually select which are generated and which are uploaded.

From release 3.10.1, the new default web pages have a totally new look (designed by Neil Thomas), and include responsive code allowing them to look better on wide screens and on narrow screens (such as those on smart mobile phones).

Settings

The settings in Cumulus 1 and MX work differently, for Cumulus 1 you choose to save changes by clicking OK, for MX changes are only saved when you click a Save button if one is provided. If there is no Save button anywhere on the screen (as in Extra Web Files) then the setting is saved when you move to next field/line.

Library software

Any non-technical person can skip this sub-section!

  • If you have been using Cumulus 1, and you decided to customise your web site:
    • you may already be using one or more of highcharts, jQuery, bootstrap, and other library software
    • and you may have selected to load latest versions for maximum functionality.
  • You might be enhancing what is provided with Cumulus 1,
  • or you might use your web site for other purposes (not just Cumulus).
  • If so, you may find some Cumulus MX web pages do not work correctly.
    • This is because Cumulus MX uses obsolete versions of library software and the way it is coded means MX is not compatible with current versions.
    • For Cumulus MX to work you need to ensure the versions of such software loaded for your web pages, are not used by your browser when viewing MX admin interface or MX web pages.
  • Equally, ensure that the versions of library software loaded by MX, do not stay loaded when you wish to view any of your web pages that require later versions of these libraries, as your desired functionality might be limited by the library versions MX uses!

Three approaches to installing MX

  1. Install MX OVER your Cumulus 1 installation (this only works if you want to run MX on same PC as you have been running Cumulus 1)
    • This means that all your existing files will be available to MX
    • This is simplest approach for those who want a simple install
    • You may need to edit a few files, please see references to files that might need editing later
    • Because you have Cumulus 1 and MX executables in same location, you would expect it is easy to run either, but as already explained some files can not be read by Cumulus 1 after MX has changed their content
  2. Take a copy of your Cumulus 1 installation and store that copy where you can use it if you want to return to Cumulus 1
    • In original Cumulus 1 location, delete Cumulus.exe (or whatever your Cumulus 1 executable was called)
    • Now follow the instructions for first approach, installing MX over the existing files (but ignore the point about running both executables)
  3. Install MX in a NEW location (this might be on your PC that was running Cumulus 1 or onto another device such as a Pi)
    • This approach requires you to manually copy various files from old folders to new location
    • MX requires all files from "data" and "Reports" folder created by Cumulus 1.
    • You also need "strings.ini" (if you use that), and "Cumulus.ini_(Cumulus_1)", plus any other tailoring set-up files, batch processes etc. you might use.
    • This approach is generally easier if you want to be able to go back to running Cumulus 1

1: SIMPLE OVERWRITE APPROACH

Unzip the MX download (from Software for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version) so the folder CumulusMX is the same folder as that which has "Cumulus.ini" in it. The unzip ensures all the files that need to be in sub-folders go into correct sub-folders.

Your in the data folder and any NOAA reports you may (they are optional) have created in Reports folder are available to MX. You should read the page in that log file link, as you might need to edit some items in the .ini files. For the .txt files, you need to check that all lines are consistent in using the same character to separate the 3 parts of the date, and the same character is used throughout to separate the items in list of fields.

  • Cumulus 1 will accept any character (except space, a digit, or the character used to separate fields in a list) as a separator for the date parts
  • MX will only accept the character defined in your locale (for a Windows PC that is actually set in Control Panel which Windows does not make easy to access preferring you to use its Settings app.
  • There is a Clock and Region section in Control Panel and in the Region window you define the separator MX will use in the Short date item, most easily by clicking additional settings).
  • Now ensure that is same character as used in al lines of your .txt log files.
    • If you have some lines using say "/" and some using "." or "-" as date separator, then there are many editors available that offer a global replace.
      • As an example, Notepad++ has a Replace All in All Opened Documents, so you can open multiple documents and do a global replace of "/" into "-" very easily. Notepad++ saves the file in the same encoding as it was before, but you can also check it is UTF-8 with no Byte Order Mark via the menus.
      • Note that you cannot easily replace "." or "-" globally in dates, because those appear as valid characters in fields. To edit those, you would need to open the files in a spreadsheet editor like Libre Office, you need to ensure that the dates column is treated as text, then select that column and do a replace in just the date column, and then save as a CSV format file changing the extension to .txt. There is more information on this in relevant log file articles.
    • The basic Microsoft Notepad that comes with Windows has a Replace option in its Edit menu, but some versions of this editor (check Save As options) don't allow you to pick "UTF-8" encoding (if that is not already shown) when saving to ensure MX can read the output.
    • Also be aware when using Google editors that they may save the file in the wrong format unless you can choose a UTF-8 no BOM option.

2: COPY C1 AND OVERWRITE ORIGINAL WITH MX APPROACH

Alternatively, to run Cumulus MX with your existing Cumulus data, first take a back up copy of your existing Cumulus directory.

Next obtain the MX distribution release as a zip as per previous option.

Now unzip Cumulus MX into the original Cumulus folder. The new CumulusMX.exe should end up in same folder as existing Cumulus.ini, there will be other files in this folder, and there will be some new folders you have not seen before like interface and MXDiags, but MX continues to use the existing sub-folders (like data and backup) without any change of name.

This saves you from copying any of your Cumulus 1 files, they just stay where they are and get used by MX.

Where your existing Cumulus.ini_(Cumulus_1) file makes reference to local folders (in say Extra Web Files), those references stay valid. The only edit you will make to configuration that affects Cumulus.ini is dependent on your weather station type, as explained later MX uses a different parameter to refer to the port used by the weather station.

Be aware that MX changes the date formats in some files, so that Cumulus 1 can no longer understand them, so be cautious about copying log files (.ini and .txt) back to your Cumulus 1 back-up should you want to revert to using Cumulus 1.

Some other issues that were described in previous sub-section will apply in this case, because for MX this approach (preserving Cumulus 1 in a new location) is no different to previous.

3: NEW LOCATION APPROACH

These notes apply whether you are installing in a new location on your PC or installing MX on a different device.

Obtain the MX download (from Software for latest version or from https://github.com/cumulusmx/CumulusMX/releases if you want an earlier release version).

When you do the unzip as explained below, it ensures all the files that need to be in sub-folders go into correct sub-folders. You then add various folders and files from your Cumulus 1 installation into correct places. As said before, this sub-section gives only a brief mention of possible issues with existing files, later sections will explain those in more detail.

  1. First create a new directory (recommended name CumulusMX) and unzip the contents into it.
    • For a pi, the instruction might be unzip -o CumulusMXDist3nnn.zip where nnn is the last 3 digits of the build number.
  2. Then copy (or file transfer over)
    • your previous data folder contents into the new data folder created by unzipping, as before you might need to edit some log files
    • your previous Reports folder contents (if any) into the new Reports folder created by unzipping
    • your Cumulus.ini_(Cumulus_1) file goes into the top level folder, the one with ExportMySQL.exe, CumulusMX.exe and the .dll files,
      • Check that destination file, it must be "Cumulus.ini" (ensure it starts with capital letter and all other letters are lower case)
      • In the new "Cumulus.ini" edit any lines that made reference to the old windows location. Remember, that only windows uses "\" in paths, and your new device will require reference to new locations. Even if you are still using windows, you may need to make changes to reflect that the files are now in a new location.
      • You can delay changing other settings (like the port used to access your weather station which uses a different parameter in MX) when you have access to the Settings pages in the MX admin interface
    • any other configuration files that you may have created (e.g. strings.ini, twitter.txt etc).

The big advantage of this approach is that anytime you are not running MX you can go back to Cumulus 1 and let it run from where it left off (subject to availability of past data in your weather station, and possibly unplugging weather station from one device and plugging it into PC). Obviously you cannot run Cumulus 1 and MX at the same time accessing the same weather station (even if both are on same device and so no unplugging and plugging in is needed).