Updating MX to new version: Difference between revisions

From Cumulus Wiki
Jump to navigationJump to search
 
(59 intermediate revisions by 3 users not shown)
Line 1: Line 1:
=Who this article is for=
=Who this article is for=


Please be aware that if you want to move from Cumulus 1 to MX, you should read [[Moving_from_Cumulus_1_to_MX]] article instead.
Please be aware that if you want to move from Cumulus 1 to MX, you should read [[Migrating_from_Cumulus_1_to_MX]] article instead.


This article is for those who already use MX, and so are comfortable with basic MX installation and running.
This article is for those who already use MX, and so are comfortable with basic MX installation and running.
Line 7: Line 7:
=Who is not intended reader=
=Who is not intended reader=


Cumulus MX has been updated so frequently in 2020, that you may be used to updating to a new build, and for you this article is not useful.
Cumulus MX has been updated so frequently in 2020, that you may be used to upgrading to a new build, and for you this article is not useful.


=Introduction to updating MX=


==Installer Option==
=Where to download a release distribution=
 
==Latest release==
 
Download release distribution zip from  [[Software|the Software article in this Wiki]] for latest version.
 
Be aware that the developer needs to remember to update this link each time there is a new release.
 
An image that contains a '''Raspberry Pi lite operating system''' and a recent CumulusMX release already included, can be downloaded from the same wiki article. Not every CumulusMX release is provided as a Raspberry Pi image, so you may need to perform an update to get the latest release. The Raspberry Pi image places CumulusMX in /opt/CumulusMX.
 
==Any release developed by Mark Crossley==
 
Download release distribution zip from https://github.com/cumulusmx/CumulusMX/releases for earlier versions.
 
 
=Introduction to upgrading MX=
 
<div style="background: LemonChiffon;padding:5px; margin:2px;">
[[File:Crystal Clear info.png|40px]] This document is 'Work In Progress' so content may not be complete or accurate!
</div>
[[Category:Cumulus MX]]
 
You can download and unzip in advance, but cannot replace existing files while they are being used, so how you stop MX is in next two sub-sections.
 
My preference, after download, and unzip into a holding area, is to copy (it would be file transfer if you download on another device) over my existing installation all files except:
*CumulusMX.exe
*CumulusMX.exe.config
while MX is still running. I then stop MX for minimal time needed to replace just those two files before I restart MX.  That way I minimise downtime, especially useful for a patch release when few files have changed, as it takes some time to replace all the files in the installation.
 
For simplicity, in guidance below, MX is stopped before any files are replaced.
 
==Upgrading on a Raspberry Pi (RPi) computer by commands from a Microsoft Windows computer (PC)==
 
I recommend you download WinSCP (this and PuTTy share some settings). This will allow you to connect your PC to your rPi and copy files back and forth.  You can use it to copy the MX distribution, you download and unzip on your PC, onto the RPi. You can also use it to take a back-up of your RPI files onto your PC, and to pick certain files off your RPi. e.g. copy log files  from [[MXdiags folder]] to your PC to read diagnostics etc.
 
You can download PuTTy (shares some settings with WinSCP) to enable you to open a terminal session and send commands to your RPi computer.
 
If you have Cumulus MX setup to run as a service then you will need to do one of these, should you wish to add debugging when the upgraded MX begins...
# Edit the CumulusMX service file (see [[MX_on_Linux#Running_as_a_service]]) to add the -debug parameter (do this if always want debugging on, not usually needed)
# Just set it from Program Settings | Logging Options (easiest for turning debug on when you need it and off when you don't need it)
# Edit the Cumulus.ini file and enable it in there (no longer recommended)
 
==Upgrading if you run MX as a service==
 
Don't forget to stop the service, before you do the upgrade.
 
For  a Linux Operating system: <code>sudo systemctl stop cumulusmx</code>.
 
For Microsoft Windows Operating System: <br>
 
Open Windows Administration Tools from Windows Task Bar, select Component Services, click on Services (local)
In middle window select CumulusMX service from the list; click on Stop the service option. (click on Start the service after upgrade)


HansR on support forum is developing an installer as this is being typed, see [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=18893 this topic]. He proposes to start a new topic when his installer is ready for general use, and I hope he will update this section appropriately.
==Upgrading if you run MX interactively==


==Updating if you are running MX on a Linux computer==
If you do NOT run as a service, you will want to use '''Control and C''' in the terminal (or command) window running MX to make the software close tidily.


You might want to read galfert's post on the support forum [https://cumulus.hosiene.co.uk/viewtopic.php?p=148851#p148851 here] for the relevant Linux instructions in a concise format.


== Updating to the next MX release if you have not updated before ==
== Updating to the next MX release if you have not upgraded before ==


The simplest update is from the immediate preceding build, and the steps required are summarised as follows:
The simplest upgrade is from the immediate preceding build, and the steps required are summarised as follows:
# Download the release distribution zip for the next build (see later for where from)
# Download the release distribution zip for the next build (see later for where from)
#* My advice is to have a separate download location away from the location where you are installing/running the software; it can retain older releases to make regression simple if the new release proves to have a bug.
# Use '''Control-C''' to stop Cumulus MX (see later if running as a service)
# Use '''Control-C''' to stop Cumulus MX (see later if running as a service)
# Take a backup of your complete existing MX installation (as it is not running, no files will be locked)
# Take a backup of your complete existing MX installation (as it is not running, no files will be locked)
# Unzip the new distribution overwriting the previous installation (the release announcement might ask you to delete obsolete files)
# Unzip the new distribution, overwriting the previous installation (the release announcement might ask you to delete obsolete files)
# Run any one-off batch scripts needed to prepare for upgrade (see later for examples)
# Run any one-off batch scripts needed to prepare for upgrade
#* These might be needed if log files in the new release contain fields that were not in earlier release
#* These might be needed if you use optional database functionality (see later for examples)
# Do any actions that require you to use [[MX Administrative Interface|Admin Interface]] to change settings
# Do any one-off actions in relation to any files provided by MX for you to run a web site with the default pages
# Restart your Cumulus MX (consider running with '''-debug''' parameter if you are not sure the new build is bug free)
# Restart your Cumulus MX (consider running with '''-debug''' parameter if you are not sure the new build is bug free)


The remainder of this article explains all options for updating, and is thus less simple than above.
The remainder of this article explains all options for updating, and is thus less simple than above.


==Considerations that determine when to update==
==Installer Option==
 
[https://cumulus.hosiene.co.uk/memberlist.php?mode=viewprofile&u=9016 HansR] on support forum has developed an multi-platform installer, see [https://cumulus.hosiene.co.uk/viewtopic.php?f=44&t=18916 Multiplatform Installer for CumulusMX] which makes life a lot easier if you do not wish to dive into the (file level contents) internals of CumulusMX.


Cumulus MX will nag you, in various places, to make you aware if you are not running the latest build. Some people will choose to update as soon after a new release as they can. However, each upgrade does involve a period when MX is not running, and that causes some loss of data:
#Copy the '''InstallCMX.exe''' to any directory you want, on the drive where you wish to install (or have previously installed) MX.
*for some weather station types readings taken every minute, or more frequently, are replaced by whatever period you station data logger records at;
#Copy the CumulusMX release distribution zip(s) to that same directory, if you have not already done so. You may have more than one distribution in the same directory.
*for other stations, without their own logging, all data is lost for the period when MX is not running.
#*The install procedure gives you the possibility to select, or define, the Archive to install, and the location where to install.
#Stop CumulusMX
#Run '''InstallCMX''' and confirm / fill in (on the console) where you wish to install (or update) CMX. The default for Windows is C:\CumulusMX\ and for Linux it is : /home/CumulusMX. The Installation directory can be modified.
#* You can give the '''build number''', for the release distribution zip, to install as command line '''argument'''.
#Start CumulusMX
 
'''NOTE:''' On Windows you run the installer as any other command line executable and it is best to open a command window in which you start the installer. On Linux you run it on the command line as "mono ./InstallCMX.exe", the mono command can be omitted if mono is already active (e.g. if you run CumulusMX as a service, and stop it, mono remains active).
'''NOTE:''' In an existing installation with modified files, make sure those are in a different, either filename or folder (a safe place). If they have the same name as files in the distribution, they will be overwritten.
'''NOTE:''' Check in the <code>webfiles</code> directory to see if any files there have been modified, since your last upload to your web server. If so, move those new files to the website.
 
After the installation, there is a '''log file'''. Check the log file to see everything has gone well.
There is an '''ini file''' where you can control:
* NormalMessageToConsole=true or (default) NormalMessageToConsole=false
* TraceInfoLevel=Warning (out of: Error, Warning, Info, Verbose, None)


Any reactions (thank you, issues and questions) please post in the [https://cumulus.hosiene.co.uk/viewtopic.php?f=44&t=18916 download thread of the installer].


Other, more cautious, people (like the present writer) will not update each time a new release becomes available (and there are a lot of new releases in 2020), here are some of the reasons:
Modifications and additions on user request can be discussed (e.g. think about automated start/stop, upload of webfiles to the directory etc....), use the talk for this page.
* you will see in the Cumulus Support Forum that many builds have bugs, and so you realise that it is better to stick with your fully working release, than install one with bugs;
* you may wish to avoid the loss of data mentioned above, by minimising the number of times that you stop MX;
* you may run MX on a computer that you rarely visit, perhaps even in a remote location not often visited, so you prefer to leave it untouched, rather than risk possibility of being unable to restart remotely;
* you may just have more important ways to use your time than updating your MX software, and some new builds might not give you any benefits that make it worthwhile to change your priorities.


It is perfectly possible to update from old versions of MX to the latest, skipping intermediate versions, but there are some key versions that you should not skip over. This article also includes suggestions for which releases to install and get running before moving onto newer releases. This will be especially useful for those people who do not immediately update to new releases, as per above suggestions.
==Upgrading if you are running MX on a Linux computer==


===Advice about skipping versions===
You might want to read galfert's post on the support forum [https://cumulus.hosiene.co.uk/viewtopic.php?p=148851#p148851 here] for the relevant Linux instructions in a concise format.


Early builds of MX were all assigned to the same version number, recently the version number seems to change with every new build. Thus the following simple advice about skipping versions might not be applicable in 2021 onwards.
If you want more detailed information, on any Linux (including Raspberry Pi operating system) aspects of the upgrade instructions on this page, see [[MX on Linux]] page.


One would assume that, skipping minor releases in an update (from 3.x.y to 3.x.z, i.e. only final digit of version number changing is a minor release) would always be simple, even if several intermediate builds are skipped. Most developers would not introduce a new feature in a minor release that requires a one-off extra action for a successful upgrade. Unfortunately, some minor releases of MX have required you to do special actions.
== Updating if you use the start/stop management script ==


One might assume that the ease of doing a major update (i.e. where middle part of version number changes) involves greater difficulty.  Most developers would only classify a new release as a major update if there was new functionality being introduced that, depending on what functionality you use, might involve a one-off extra action to prepare for upgrade to that version. It is harder to understand why some MX releases are classified as major version number change, and some as minor version number change.
This section contributed by ''Jank'' on support forum.  Note the version on the forum might have been updated from the original included here.


You may be using an old version of MX that is actually several versions behind, or even still using a beta version released by Steve Loft, don't worry because this article covers upgrading however old a version you are using. You can update skipping some major updates, but as that can be more complex, there is a separate section later with advice on how to update in stages from an early build to a recent build.
1. look on [[Software|Software download page]], find the link to latest version, and fill out the '...' below appropriately as you run these 2 commands on your device where you do downloads:
<pre>cd /tmp
wget https://github.com/cumulusmx/CumulusMX/ ... .zip</pre>


= Knowing when a new release is available =
2. Once that download is complete, start cumulusmx.sh with option -u
<pre>/home/pi/CumulusMX/cumulusmx.sh -u</pre>


== CREDIT ==
3.When asked for the zip file, enter
<pre>/tmp/CumulusMXDist</pre> and hit the TAB Button


Thanks to ''Billy'' on support forum for suggesting text for this section.
4.Choose the zip file with the CumulusMX upgrade and hit return.


== Using forum notifications ==
5. Follow the on screen instructions


As new releases are announced in [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 Cumulus MX Announcements and Download - PLEASE READ FIRST] topic, you can use the spanner tool to '''subscribe''' to this topic to receive notifications of a new post announcing a new release (or any other release-related announcement).
6. With each update component .....you can choose: [y]es, [n]o, [A]ll, [N]one, [r]ename


== MX terminal message ==
I would recommend select '''A''' as that will simply replace all files without further action.


If ...
#...you have a monitor to see the terminal output from the Cumulus MX engine (Windows calls this either a command window, powershell window, or a terminal window depending on how you invoke it, for Unix-based implementations this is the output window when using the terminal functionality), AND
#...your device running MX is connected to internet, AND
#...your MONO (if not Windows) is not obsolete (SSL certificate out of date), AND
#...you restart MX
... then you will see a prompt when a new version of MX is available.


It is worth stressing that if you leave MX running, then this feature will leave you blissfully unaware that an update is available; it only checks when MX is restarted.
CumulusMX will be restarted after upgrade completes.


== MXDiags ==
You can check if the upgrade was successful by using option -s:
<pre> /home/pi/CumulusMX/cumulusmx.sh -s</pre>


In addition ...
=Considerations that determine when to upgrade=
#...if you can view the MXdiags file for the current session of MX, AND
#... MX is running with connection to the internet, AND
#...you restart MX
... if a new version of MX is available, the MXDiags file will say so (the message is not easy to spot as there is a lot of output before it, and variation in what output appears before it). Anyway, here is one example, just one example as in my experience '''the message has appeared at different places for each of the recent updates'''):
<pre>
2020-05-27 04:18:48.326 Calculating sunrise and sunset times
2020-05-27 04:18:48.326 Sunrise: 04:58:11
2020-05-27 04:18:48.326 Sunset : 21:19:54
2020-05-27 04:18:48.326 Tomorrow sunrise: 04:57:08
2020-05-27 04:18:48.326 Tomorrow sunset : 21:21:11
2020-05-27 04:18:48.388 You are not running the latest version of CumulusMX, build 3080 is available.
2020-05-27 04:18:48.763 Station type:</pre>


There is no message in whatever place it can appear ...
Cumulus MX will nag you, in various places, to make you aware if you are not running the latest build. Some people will choose to upgrade as soon after a new release as they can. However, each upgrade does involve a period when MX is not running, and that causes some loss of data:
*... if you are using latest version, OR
*for some weather station types readings taken every minute, or more frequently, are replaced by whatever period you station data logger records at;
* ... if you are not connected to the internet, OR
*for other stations, without their own logging, all data is lost for the period when MX is not running.
*... if you keep MX running all the time!


== Start/Stop script ==


When the '''Status''' option of this script is used, whenever a new release of MX is available, it will output a message.
Other, more cautious, people (like the present writer) will not upgrade each time a new release becomes available (and there are a lot of new releases in 2020), here are some of the reasons:
* you will see in the Cumulus Support Forum that many builds have bugs, and so you realise that it is better to stick with your fully working release, than install one with bugs;
* you may wish to avoid the loss of data mentioned above, by minimising the number of times that you stop MX;
* you may run MX on a computer that you rarely visit, perhaps even in a remote location not often visited, so you prefer to leave it untouched, rather than risk possibility of being unable to restart remotely;
* you may just have more important ways to use your time than updating your MX software, and some new builds might not give you any benefits that make it worthwhile to change your priorities.


== Dashboard in MX Admin Interface ==
It is perfectly possible to '''upgrade from rather old versions of MX, to the latest''', skipping intermediate versions, '''but there are some key versions that you should not skip over'''. This page, in subsequent sections, includes suggestions for which releases to install (and get running) before moving onto newer releases. This will be especially useful for those people who do not immediately upgrade to new releases, as per above suggestions.


In recent releases, one of the alarm indicators shown at the bottom of the dashboard page of the administrative interface, will flash red when you are not running latest build, and glow green when there is no newer release compared to the one you are running.
===Advice about skipping versions===


== Web Tags ==
Please see [[Updating_MX_to_new_version#Updating_from_a_very_old_version]] sub-section later on this page (and preceding sub-sections where relevant)


Since 3.7.0 release of MX, two relevant web tags have become available:
The important point to make here is that when you do an upgrade, you should copy in '''ALL''' files from the release distribution you want to run next. This is because there are dependencies between files within a distribution, so missing out any particular file may stop other files from working. However, if you are skipping versions, read the in-between release announcements to see if there are any one-off actions.
* <#NewBuildAvailable> Returns a boolean value
**  0 - MX running the latest build available
**  1 - MX is running an earlier build than the latest public release
* <#NewBuildNumber>  Displays the latest public release build number


= What to read (and when) before updating =
= What to read (and when) before upgrading =


== Updating from immediately preceding build ==
== Upgrading from immediately preceding build ==


=== The release announcement ===
=== The release announcement ===
Line 136: Line 187:
***That may mean you are advised to regress to an earlier version and use that
***That may mean you are advised to regress to an earlier version and use that
***It might mean that some supporting files in current version are wrong, and you only need to regress those named files
***It might mean that some supporting files in current version are wrong, and you only need to regress those named files
***There might be an emergency release to fix the bugs, and you need to update to that emergency release
***There might be an emergency release to fix the bugs, and you need to upgrade to that emergency release
***Finally you might be given advice to avoid using certain parts of the functionality or take some other action until the next release is available.
***Finally you might be given advice to avoid using certain parts of the functionality or take some other action until the next release is available.


Line 145: Line 196:
You can also view the latest [https://github.com/cumulusmx/CumulusMX/blob/master/Updates.txt Updates.txt].
You can also view the latest [https://github.com/cumulusmx/CumulusMX/blob/master/Updates.txt Updates.txt].


===Deciding whether to update to new release ===
===Deciding whether to upgrade to new release ===


This has been covered earlier in this article, but I repeat here 2 critical considerations:
This has been covered earlier in this article, but I repeat here 2 critical considerations:
Line 157: Line 208:
===Reading multiple release announcements===
===Reading multiple release announcements===


If you are skipping some intermediate builds, then you will need to read each of the formal release announcements for builds after the build you currently use. You need to apply the cumulative actions recommended. Otherwise, most of the advice above for updating from immediately preceding build still applies.  The other sources of release information mentioned above may be consulted as an alternative.
If you are skipping some intermediate builds, then you will need to read each of the formal release announcements for builds after the build you currently use.
 
The other sources of release information mentioned above may be consulted as an alternative and may highlight something useful, but the formal release announcement contains what the developer wants you to know.
 
===Action to take when skipping minor intermediate builds===
 
You need to apply the cumulative actions recommended (i.e. apply any actions in sequence as listed for each intermediate build). Unfortunately, for Cumulus MX, one cannot assume that minor builds have no one-off actions associated with them.  Luckily, most of the advice above for updating from immediately preceding build still applies.


==Updating to a new major version==
==Updating to a new major version==
Line 165: Line 222:
=== Examples of what might be classified as a major change===
=== Examples of what might be classified as a major change===


*Additions to fields in log files
*Additions to fields in log files (whilst older lines can continue to be read, you might want to consider [https://cumulus.hosiene.co.uk/viewtopic.php?f=18&t=18096 amending your standard data log] by populating the additional derivatives for older lines)
*Additions or removals in configuration file
*Schema changes for database tables (if you don't use the standard tables, this might be irrelevant to you), the standard insert operations by MX can cope with extra columns in a table, but will fail if any column they try to update is missing
*New pages in Administrative Interface
*Additions, changes, or removals, in configuration file (in general MX will ignore unrecognised parameters, but it does need correct values for those attributes it does recognise), see release announcements to discover if new lines in configuration file need you to add them manually or you need to use the admin interface to select the right setting for you
*Catering for new weather station sensors, or new ways of communicating
*New pages in Administrative Interface (additional pages require menu on every other page to be amended)
*Any interface functionality changes
*Changes to existing pages in admin interface that involve changes to associated files (it is vital that all associated files are from same build)
*Additions to files being generated for web server
*Catering for new weather station sensors, or new ways of communicating (this might not seem relevant to you)
*Schema changes for database tables
*Any interface functionality changes (it may look the same, but what it does has changed)
*Additions to files being generated for web server (i.e. changes to template files, or additions to ''web'' or ''webfiles'' folders), even if you use your own versions of such files you may want to check the revisions


=== What you need to read ===
=== What you need to read ===
Line 177: Line 235:
*Basically, check the corresponding release announcements for every version since the one you have been using before planning your upgrade.  
*Basically, check the corresponding release announcements for every version since the one you have been using before planning your upgrade.  
**Make a note of any one-off actions required at particular in-between versions, remember these actions are only in forum release announcements.
**Make a note of any one-off actions required at particular in-between versions, remember these actions are only in forum release announcements.
**Although one-off actions will not be described in the Wiki (whether on the [[Software]] page or the [[Cumulus_MX_formal_release_versions]] page), the Wiki can give you an idea of what functionality has been improved to help you decide whether to update.
**Although one-off actions will not be described in the Wiki (whether on the [[Software]] page or the [[Cumulus_MX_formal_release_versions]] page), the Wiki can give you an idea of what functionality has been improved to help you decide whether to upgrade.
*It is still worth reading all the points made above for updating from immediately preceding build.
*It is still worth reading all the points made above for updating from immediately preceding build.


=== Doing the upgrade===
# Download the release distribution
# Use '''control-c''' (or equivalent for MX running as service) to stop your existing MX software
# Backup your existing installation
# Do the on-off actions identified (e.g. changing schema of database tables, adding new configuration
# Unzip the distribution over existing one (alternative installation options listed later)
# Restart MX, considering whether to use '''-debug''' parameter in case the new release does not work for you
# Check the latest file in the MXDiags folder after an hour, to see if there have been any problems with reading from station, real-time uploads, standard uploads, or hourly uploads. (You may wish to recheck after an End of Day action has been done).


==Updating from a very old version==
==Updating from a very old version==
Line 191: Line 258:
===Recommendations for staged updating===
===Recommendations for staged updating===


====If you use 3.0.0 (the MX original beta) now====
<big>'''This section will need to be updated, new contributors are needed to keep this advice current.'''</big>
 
Do read all release announcements, to see if the developer has made a major change since those described below that means you need to stage your updates if you are currently running an old release.


Update to 3.5.1 by downloading it at https://github.com/cumulusmx/CumulusMX/releases/tag/b3072.


This gives you essential new functionality in the admin interface like editors for the log files and extreme records. But it also fixes multiple bugs in the beta you were using and adds some useful validation missing in the beta.


It does not involve any updates to the fields in the log files nor to the columns in any database tables you use.


It skips you past the problems in 3.5.0. It gives you benefits introduced in 3.1.x, 3.2.y, 3.3.z, and 3.4.w releases.


====If you use 3.1.x, 3.2.y, 3.3.z, and 3.4.w releases====


As above, update to 3.5.1 by downloading it at https://github.com/cumulusmx/CumulusMX/releases/tag/b3072.


====If you use a 3.5.x release ====


My advice is to update directly to 3.7.0 or any subsequent release available at [[Software]].  This is because several 3.6.y releases have bugs in them and you want to avoid those problems (the bugs vary in severity between mistakes in calculations done by MX to particular functionality not working).
 
 
 
====If using a release up to 3.11.4 and want to use any later release====
 
Upgrade exactly to release 3.12.0 (no later) by using zip at [https://github.com/cumulusmx/CumulusMX/releases/download/b3141/CumulusMXDist3141.zip download/b3141/CumulusMXDist3141.zip].
 
Run that release, it will rename your [[Cumulus.ini]] file, and create a new file with same name, but different settings.
 
You need to work through all the settings pages in the interface, and ensure all settings are correct, then close MX so it writes away the new settings according to your preferences.
 
Now [[Software#Latest_build_distribution_download|download the latest release]], and copy in all the files from the zip, before restarting MX at the release that is supported by developer.
 
 
 
 
 
 
====Currently running 3.10.x or 3.11.y====
 
Treat like any other major version upgrade as described above.
 
The changes between any 3.10.x release and any 3.11.y release were minor, but ''you can't go from 3.10.x to the latest release'', because you must use 3.12.0 to convert [[Cumulus.ini]].
 
 
 
 
 
 
====Currently using one of the 3.9.y releases ====
 
If you are using any release in the 3.9.y series  (note release announcement 3.9.1 warning about Mono if you use that), and you are using web pages that were provided by MX...
 
Then you need to be aware that 3.10.1 (3.10.0 was withdrawn) got rid of the HTML templates that you have been using, and introduced a lot of new settings.
 
The good news is that you can now jump straight to 3.12.0 as that release will rename your existing [[Cumulus.ini]] and create a new file with all the new settings:
* Changes to settings
** You need to open the [[MX Administrative Interface]] and work through all pages in the '''Settings''' menu.
*Changes to web files included in two folders in distribution: <code>[[Web folder|CumulusMX/web]]</code>, <code>[[Webfiles folder|CumulusMX/webfiles]]</code>
**If you used the provided templates to produce web pages in earlier releases, or you had directly customised provided [[Customised templates|web templates]], these will no longer work
**Please see [[New Default Web Site Information|New Default Web Site Information page]] for further advice
 
Note:  The folder <code>CumulusMX/webfiles-legacy</code> mentioned in the new default web site information page, with some alternative web pages that have no ongoing support, is only available from a 3.10.0 or 3.10.1 download (find from [https://github.com/cumulusmx/CumulusMX/releases this Github page]).
 
 
 
 
 
====if using 3.7.0 release, any 3.8.x  release, or 3.9.y release====
 
See sub-section directly above, that describes similar process,
* EITHER '''upgrade directly to 3.9.6, or any later build, within 3.9.x, without any major change,'''
* or to 3.12.0 with major change
*  (find either release from [https://github.com/cumulusmx/CumulusMX/releases this Github page]).
 
If you are using  a release earlier that 3.7.0, you can note 3.7.0 was only build in 3.7.y series, but because 3.7.0 introduced a lot of changes these staged upgrades recommend that 3.7.0 is implemented, and run for a while, before continuing to upgrade.
 
Be aware that 3.8.0 was a major release, as it introduced the ability to run Cumulus MX as a service, but there is no reason to install it as the ability to run either interactively, or as a service, continues to be available in all subsequent releases.
 
It is optional to install release 3.9.6 build 3101, because that is a safe release to use while there were bugs in the builds in all 3.8.z versions, and in other 3.9.x releases, however please action the one-off changes noted as IMPORTANT here:
 
'''IMPORTANT''' one-off actions needed:
* There is a one-off change described in [https://cumulus.hosiene.co.uk/viewtopic.php?p=146957#p146957  v3.9.0 - b3095 release announcement] for those using RG-11 rain sensor.
* There is a further on-off change described in [https://cumulus.hosiene.co.uk/viewtopic.php?p=147329#p147329 release announcement for Patch release 3.9.1 - b3096] for those who use '''Mono''' to enable the executables to run.
 
Be aware that 3.10.0 was withdrawn, but it was a major release that totally changed the files in [[Web folder]] and [[Webfiles folder]] within the release download. However, you can skip directly from 3.9.6 (if you have implemented that) to 3.12.0 as soon as you are happy to change your web server contents (if you are using web pages that MX provides), see sub-section above for further advice re web pages.  You should continue in stages because it is mandatory to install 3.12.0 as only that release can rewrite your [[Cumulus.ini]] file ready for subsequent release, so ensure that 3.12.0 is working before installing current release as instructed in sub-sections for those using subsequent releases.
 
====If using a 3.5.x release ====
 
My advice is to upgrade directly to 3.7.0 available at [https://github.com/cumulusmx/CumulusMX/releases Mark's Github repository].   
 
''You should skip the intermediate releases because several 3.6.y releases have bugs in them'', and you want to avoid those problems (the bugs vary in severity between mistakes in calculations done by MX to particular functionality not working).


But there may be additional actions you need to do when moving from 3.5.x to 3.7.y, depending on what features you use in MX:
But there may be additional actions you need to do when moving from 3.5.x to 3.7.y, depending on what features you use in MX:
*there are no additional actions if '''you use standard web pages and you do not use database tables''', just enjoy the bug fixes and extra features after your update!
*there are no additional actions if '''you use standard web pages and you do not use database tables''', just enjoy the bug fixes and extra features after your update!
*'''If you use database tables''', be aware that the schema (Column names that must be in a table) varies between 3.5.x versions and 3.7.y versions.
*'''If you use database tables''', be aware that the <big>schema (Column names that must be in a table) varies</big> between 3.5.x versions and 3.7.y versions.
**The updating database table features in MX will only work if all the columns named in each such upload update ''already exist'' in the database table it is trying to update.
**The '''updating database table features in MX will only work''' if all the columns named in each such upload update ''already exist'' in the database table it is trying to update.
**Look at individual release announcements for both 3.6.0 and 3.7.0, to see the SQL provided for adding the columns added at particular versions.
**Look at individual release announcements for both 3.6.0 and 3.7.0, to see the SQL provided for adding the columns added at particular versions.
***At 3.6.0 the SQL was provided as a separate attachment to the release announcement (UpdateMYSQL-b3076.zip)
***At 3.6.0 the SQL was provided as a separate attachment to the release announcement (read [https://cumulus.hosiene.co.uk/viewtopic.php?p=142085#p142085 UpdateMYSQL-b3076.zip])
***At 3.7.0 the SQL was provided as part of the main zip (AlterSQLTables3098.sql)
***At 3.7.0 the SQL was provided as part of the main zip (read [https://cumulus.hosiene.co.uk/viewtopic.php?p=145048#p145048 AlterSQLTables3098.sql])
***In both cases, the SQL provided assumes you are using the default names for database tables, it does not read [[Cumulus.ini]] to see whether you have selected different names for the tables (unlike ExportMySQL.exe and standard CumulusMX.exe uploads which check what tables names you have selected for updates).
***In both cases, the SQL provided assumes you are using the default names for database tables, it does not read [[Cumulus.ini]] to see whether you have selected different names for the tables (unlike ExportMySQL.exe which used to be provided in the MX release zip with standard CumulusMX.exe;  both of those (at MX release 3.7.0) check what tables names you have selected for updates).
*'''If you have your own customised web pages''', then there are changes to web tags that might lead to you needing to edit your web pages.
*'''If you have your own customised web pages''', then there are changes to web tags that might lead to you needing to edit your web pages.
*'''If you use commas to separate integer and decimal parts of real numbers''', then various releases from 3.6.0 to 3.7.0 add "rc=y" to various web tags, that option will replace the decimal commas you use by decimal points that are required for some script languages (like the JavaScript used by HighCharts), and that makes it easier if you want to customise your web site.
*'''If you use commas to separate integer and decimal parts of real numbers''', then various releases from 3.6.0 to 3.7.0 add "rc=y" to various web tags, that option will replace the decimal commas you use by decimal points that are required for some script languages (like the JavaScript used by HighCharts), and that makes it easier if you want to customise your web site.
When you are happy with running 3.7.0, then you should continue to upgrade, but in steps (optionally try 3.9.6 because that will test some new features, then mandatory upgrade to exactly 3.12.0 which will rewrite your Cumulus.ini file, then you can continue your upgrade to latest) as described in earlier sub-sections for later releases.
====If using either 3.1.x, 3.2.y, 3.3.z, or 3.4.w releases====
First, upgrade to 3.5.1 by downloading it at [[https://github.com/cumulusmx/CumulusMX/releases/tag/b3072 Mark's Github respository]]. The actual installation is done using the instructions early in this Wiki page for simple next build upgrades. You can safely skip reading the intermediate release announcements, as there are no special one-off actions.  There are one-off actions at 3.5.1, see release announcement.  It does not involve any updates to the fields in the log files nor to the columns in any database tables you use.
When you are happy running 3.5.1, then you should continue to upgrade, initially follow instructions given for later releases
====If using 3.0.0 (the MX original beta)====
Upgrade from the beta directly to 3.5.1 by downloading it at https://github.com/cumulusmx/CumulusMX/releases/tag/b3072.  There are one-off actions at 3.5.1, see release announcement. This skips you past the problems in 3.5.0. It gives you benefits introduced in 3.1.x, 3.2.y, 3.3.z, and 3.4.w releases.
This gives you essential new functionality in the admin interface like editors for the log files and extreme records. But it also fixes multiple bugs in the beta you were using and adds some useful validation missing in the beta.
It does not involve any updates to the fields in the log files nor to the columns in any database tables you use.
Now follow instructions in other sub-sections, to upgrade in stages to where there are significant actions to do, until you reach latest release and get support from developer.
= Knowing when a new release is available =
== CREDIT ==
Thanks to ''Billy'' on support forum for suggesting text for this section.
== Using forum notifications ==
As new releases are announced in [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 Cumulus MX Announcements and Download - PLEASE READ FIRST] topic, you can use the spanner tool to '''subscribe''' to this topic to receive notifications of a new post announcing a new release (or any other release-related announcement).
== MX terminal message ==
If ...
#...you have a monitor to see the terminal output from the Cumulus MX engine (Windows calls this either a command window, powershell window, or a terminal window depending on how you invoke it, for Unix-based implementations this is the output window when using the terminal functionality), AND
#...your device running MX is connected to internet, AND
#...your MONO (if not Windows) is not obsolete (SSL certificate out of date), AND
#...you restart MX
... then you will see a prompt when a new version of MX is available.
It is worth stressing that if you leave MX running, then this feature will leave you blissfully unaware that an update is available; it only checks when MX is restarted.
== MXDiags ==
In addition ...
#...if you can view the MXdiags file for the current session of MX, AND
#... MX is running with connection to the internet, AND
#...you restart MX
... if a new version of MX is available, the MXDiags file will say so (the message is not easy to spot as there is a lot of output before it, and variation in what output appears before it). Anyway, here is one example, just one example as in my experience '''the message has appeared at different places for each of the recent upgrades'''):
<pre>
2020-05-27 04:18:48.326 Calculating sunrise and sunset times
2020-05-27 04:18:48.326 Sunrise: 04:58:11
2020-05-27 04:18:48.326 Sunset : 21:19:54
2020-05-27 04:18:48.326 Tomorrow sunrise: 04:57:08
2020-05-27 04:18:48.326 Tomorrow sunset : 21:21:11
2020-05-27 04:18:48.388 You are not running the latest version of CumulusMX, build 3080 is available.
2020-05-27 04:18:48.763 Station type:</pre>
There is no message in whatever place it can appear ...
*... if you are using latest version, OR
* ... if you are not connected to the internet, OR
*... if you keep MX running all the time!
== Start/Stop script ==
When the '''Status''' option of this script is used, whenever a new release of MX is available, it will output a message.
== Dashboard in MX Admin Interface ==
In recent releases, one of the alarm indicators shown at the bottom of the dashboard page of the administrative interface, will flash red when you are not running latest build, and glow green when there is no newer release compared to the one you are running.
== Web Tags ==
Since 3.7.0 release of MX, two relevant web tags have become available:
* <#NewBuildAvailable> Returns a boolean value
**  0 - MX running the latest build available
**  1 - MX is running an earlier build than the latest public release
* <#NewBuildNumber>  Displays the latest public release build number


= Recommended Actions when Updating =
= Recommended Actions when Updating =
Line 224: Line 445:
== Back-ups ==
== Back-ups ==


It is always best to take a backup of your existing MX installation before you do an update, this allows you to regress back to the earlier version if either you mess up installing the new version, or the new version has a issue that prevents it working with the versions of other software (like MONO) that your installation uses.
It is always best to take a backup of your existing MX installation before you do an upgrade, this allows you to regress back to the earlier version if either you mess up installing the new version, or the new version has a issue that prevents it working with the versions of other software (like MONO) that your installation uses.


== The two approaches ==
== The two approaches ==
Line 230: Line 451:
Contributors to this section are named in text.
Contributors to this section are named in text.


Some people upgrade by just copying in the files that the release announcement says have changed, others copy in all files from the downloaded zip. The first should only be used with caution, files like '''CumulusMX.exe.config''' can change between versions, but not be mentioned in a release announcement, and the developer will have been making edits to files since the previous release, and might forget exactly which files have been edited between releases. Also you may be upgrading from an earlier version and therefore be skipping several intermediate releases. You may be able to see the dates when files were changed within the zip and therefore be able to decide for yourself if you compare those dates with the previous release you were using if you have kept the download for the version you were using.
Some people upgrade by just copying in the files that the release announcement says have changed, others copy in all files from the downloaded zip. The first should only be used with caution, files like '''CumulusMX.exe.config''' can change between versions, but not be mentioned in a release announcement, and the developer will have been making edits to files since the previous release, and might forget exactly which files have been edited between releases. Also you may be upgrading from an earlier version and therefore be skipping several intermediate releases.  
 
You may be able to see the dates when files were changed within the zip and therefore be able to decide for yourself if you compare those dates with the previous release you were using if you have kept the download for the version you were using. However, do not assume that only files with later date are needed; in some releases the new build regresses one or more files and so a file with an older date is the correct one to retain.


===The rename old approach===
===The rename old approach===


#The popular approach recommended by many forum contributors in many different posts (including at [https://cumulus.hosiene.co.uk/viewtopic.php?p=141763#p141763 this post by Mark Crossley for example] is to rename your current install directory, then unzip the new release, letting it create a new '''CumulusMX''' folder (or whatever name you prefer and specify in unzip options).  
#The popular approach, recommended by many forum contributors, in many different posts (including at [https://cumulus.hosiene.co.uk/viewtopic.php?p=141763#p141763 this post by Mark Crossley for example] is to rename your current install directory, then unzip the new release, letting it create a new '''CumulusMX''' folder (or whatever name you prefer and specify in unzip options).
#Copy across '''Cumulus.ini''' and '''string.ini''' into that new directory, and then copy the contents of the '''data''' and '''Reports''' directories from your ''current install'' to the new install.   
#However, you will find that this option is not being recommended any more, because it relies on you knowing which files (see next 2 points) you need to copy back.
#Copy across '''Cumulus.ini''' and (if you use it) '''string.ini''' into that new directory, and then copy the contents of the '''data''' and '''Reports''' directories from your ''current install'' to the new install.   
#Don't forget to copy any other set-up files across too.  
#Don't forget to copy any other set-up files across too.  
#*The advantage (as Mark says) is that you ensure you do use all the files in the new release, and don't miss out any he may have forgotten to mention in his release announcement.
#*The advantage (as Mark says) is that you ensure you do use all the files in the new release, and don't miss out any he may have forgotten to mention in his release announcement.
Line 241: Line 465:
#*Another advantage (as PaulMy says [https://cumulus.hosiene.co.uk/viewtopic.php?p=140262#p140262 here for example]) is that you retain your old set-up intact and can easily restore it should you have a problem with new release.
#*Another advantage (as PaulMy says [https://cumulus.hosiene.co.uk/viewtopic.php?p=140262#p140262 here for example]) is that you retain your old set-up intact and can easily restore it should you have a problem with new release.
#Various people advise that you don't delete your old installation for a week or so; as you may notice something from the older version that you haven't copied across!  
#Various people advise that you don't delete your old installation for a week or so; as you may notice something from the older version that you haven't copied across!  
#Check again that you copied across strings.ini, twitter.txt, Cumulus.ini, and similar files in the same folder level as CumulusMX.exe, as well as all the files in the '''data''' and '''Reports''' sub-folders. Then, provided you are happy, you can delete the old release when you want to reduce clutter on your storage discs.
#Check again that you copied across (you may not have all of these files, but if you have them in old installation then you need them in new installation) '''strings.ini''', '''twitter.txt''', '''Cumulus.ini''', and similar files in the same folder level as CumulusMX.exe, as well as all the files in the '''data''' and '''Reports''' sub-folders. Then, provided you are happy, you can delete the old release when you want to reduce clutter on your storage discs.


===The overwrite approach===
===The overwrite approach===


#* if you have a lot of set-up files, or other custom files, (i.e. files not part of release), or
The general advice is that if you want custom pages for the administrative interface, or custom templates to process  to produce your web pages, then these custom files should be given different names (and ideally be placed in different folders) to the standard files included in the release distributions. If you follow this advice, the approach described in this sub-section is the best for you, if you don't follow this advice, see later section on dealing with customised files.
#*if you are downloading on a different device, or on a different disc to where you are running MX,  
 
#then David [https://cumulus.hosiene.co.uk/viewtopic.php?p=140355#p140355 (see this post)] recommends this different approach. After downloading a new release unzip it on the device/disc where you down load it. Next simply copy the files (optionally only those that have newer dates because they have changed) into the existing MX directory on the device where you run MX.  Then you know all your existing files are there, and you can choose to only copy in files that have been updated.
#* if you have a lot of set-up files, or other custom files, (i.e. files with names that are not part of release), these won't be overwritten by files in the release distribution, but you do want to keep them;
#'''The problem with this approach is that in some releases files are removed from the distribution''', yet unless you check carefully, this approach may ''leave files that are no longer required'' in your installation, which you can continue to use instead of the intended alternative. This can cause you problems if those files are then still used by you, giving you a non-standard installation that it is difficult for others to support. So do check through file by file to ensure you only have files in the distribution and files that either configure your system or are log files.
#*or if you are downloading on a different computer, or on a different disc, to where you are running MX, it is easiest to keep your MX installation in the same place
#*The updated files can be tracked by their modification date, or by seeing which are specified in the release notes (find them on [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 this forum] or in [[Cumulus_MX_formal_release_versions|Wiki here]]).
#Note the forum administrator, David, [https://cumulus.hosiene.co.uk/viewtopic.php?p=140355#p140355 (see this post)] recommends this particular approach.  
#*In general, especially if updating a major release, it is best to overwrite with all files in zip, whether they are listed as changed or not.  This ensures you have a consistent set of files.
#After downloading a new release unzip it on the device/disc where you down load it. Next stop your existing Cumulus MX software and take a backup of your existing installation. After that, simply copy the files (optionally only those that have newer dates because they have changed) into the existing MX directory on the device where you run MX.  Then you know all your existing files are there, and your MX can be run as before.
#The great advantage of this approach, and the reason that it is widely recommended as best, is that you can't lose any data or configuration files (i.e. files that are not part of the release distribution).
#Provided you did the backup, '''the only problem with this approach is that in some releases files are removed from the distribution''', yet unless you check carefully, this approach may ''leave files that are no longer required'' in your installation, and it is just possible you might continue to use the obsolete file instead of the intended alternative. Such a non-standard installation is difficult for others to support. So do check through file by file to ensure you only have files in the distribution you have just installed, not any from earlier distributions. The only files that you still want (that are not in distribution) will be those that either configure your system or contain your data in log files.
#*The updated files can be tracked by their modification date, some will be specified in the release notes (find them on [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 this forum]). There is a list indicating what files vary in [[Cumulus_MX_formal_release_versions|this Wiki article]].
#*In general, especially if updating a major release, it is best to overwrite with all files in zip, whether they are listed as changed or not.  This ensures you have a consistent set of files, and avoids any issues when the developer does not correctly list all the files that have changed, or a file might be regressed to an earlier version and not have a newer date than the file in your previous installation.


====Example downloading on PC and installing on pi ====
====Example: downloading on PC and installing on pi ====


#On your pc: Download zip either from  [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases for earlier versions.
#On your pc: Download zip either from  [[Software]] for latest version or from https://github.com/cumulusmx/CumulusMX/releases for earlier versions.
#On your pc: Use Filezilla (or similar file transfer program, or it might even be a copy over network depending on way set up) to transfer zip to pi  e.g. <tt>FileZilla CumulusMXDist3089.zip to /home/pi</tt>
#On your pc: Use Filezilla (or similar file transfer program, or it might even be a copy over network depending on way set up) to transfer zip to pi  (e.g. <tt>FileZilla CumulusMXDist3089.zip to /home/pi</tt>), but remember you might want to install to a different location (perhaps on an external USB drive, or SSD)
#Control C on pi to stop MX
#Control C on pi to stop MX
#On pi go to directory where zip now is <tt>cd /home/pi</tt>
#On pi go to directory where zip now is, for example it might be <tt>cd /home/pi</tt>
# Unzip the installation over your existing installation <tt>unzip -o CumulusMXDist3089.zip</tt>
# Unzip the installation over your existing installation, by running on your Raspberry Pi this instruction: <tt>unzip -o CumulusMXDist3089.zip</tt>
# Restart MX
# Restart MX
# Remove any files that were needed by earlier releases but release announcement says are not needed by the release you are installing
# Remove any files that were needed by earlier releases but release announcement says are not needed by the release you are installing


== After an update if you use the standard web template files ==
===The customised approach===
 
If you edit a file that is included in the release distribution, you should give it a different name (e.g. I have placed an underline in front of original file name for some of the files I edited in the admin interface), and/or place it in a different folder (e.g. I place all the cumulus templates I have created in a folder called '''cumulus_Templates'''). If you follow this advice, then you won't lose your files by following the advice above, regardless of whether you overwrite or not.


=== web folder from zip ===
If you won't listen to best practice advice, and have decided to for example modify the provided Cumulus web templates without allocating new names, then installing the whole release distribution will overwrite your edited file(s) with the standard file content provided by the developer. Therefore you need to do a customised approach where you select which files from the distribution to copy into your installation. If you have changed one or more web templates, you might think it safe just to omit the relevant files in the zip within folder '''web''', but some builds change the files in the '''webfiles''' folder too, and you need to retain consistency.


The new web template files will be in your '''web''' folder and MX will process the new files and upload the web pages produced as before with no further action by you.
==== Updating when files within release might overwrite your own edits ====


=== webfiles folder from zip ===
To further explain the customised approach, here are some more examples.


You do need a further action, if any files in this folder have been updated.  The updated files must be uploaded to your web site.
===== web site files =====


This may be easy to forget if you use the standard web templates, because you thought that upload was a one-off and maybe you cannot remember what you did.  
If you have edited any files that sit on your '''web site''', then these can remain on your web site. However, if those web pages are generated from templates that you expect MX to process and upload for you, you do need to take special action. Maybe you edited a template to give the look you desire, or the content you want (e.g. adding rain this month to this month page, or combining this month and this year page).


Certain third-party web-pages still use files from that folder, if one of the files they use has been updated, then ensure you have a backup of your web site before you upload the updated file, it should work, but it might not, and you might need to regress.
The first step is to see if the release notice says that file has been revised, if it has not then it is easy to keep your edited file by not copying in the replacement file from within the zip.  If the release revises any file you previously edited, take a backup of your edited file, before you copy the new file into your folder. You can then use a file comparing tool to see what has changed in the release and what you changed and hopefully manage to merge to a new file that keeps any functionality change in a new release and keeps your customisation. Do think about changing name of the template file, or placing it in a different folder, and using the '''Extra Files''' settings. That will make it easier for you next time you do an upgrade to a new MX build.


Let me go through the files currently in webfiles folder:
If you have done major customisation to the standard website then you probably have followed the guidance and stored your new web page templates in a different directory and you use '''Extra Files''' to specify where they are, so they cannot be overwritten, and the new MX will still find them. It might be that new web tags have been added since whatever build you were running before, and that you decide to edit your customised template file, but that can be done any time after you install the new MX build.
*'''weatherstyle.css''' has not been updated since its last update by Steve Loft in 2009.
*subfolder '''images''' contains 2 pictures that have stayed same since MX launch
*subfolder '''js''' contains one file. The file '''\CumulusMX\webfiles\js\cumuluscharts.js''' has changed in a lot of releases.  The standard web page '''trends.htm''' will only work if the correct version of that file has been uploaded to your web site and placed in a sub-folder "js" of where "trends.htm" is installed.  BCJKiwi's user interface '''charts.php''' also requires the same JavaScript file, so upload the file anytime it is updated in a MX release.
*subfolder '''lib''' currently contains 3 folders:
**'''highstock''' - this folder is no longer needed on your web site
**'''jquery''' - this folder is needed for "trends,htm" to work, but it has not been updated since 2014. Both the files included are obsolete packages, no longer officially available, and there are no plans to replace them.
**'''steelseries''' - this folder has 3 sub-folders, and I can only think of 2 MX releases when anything was changed here, and I don't anticipate any further changes.


== Updating when files within release might overwrite your own edits ==
===== admin interface files =====


=== web site files ===
If you have done any customisation to the '''interface''' (perhaps you don't like seeing solar in the tables when you don't measure that) then again you can skip over these files when copying. Hopefully you will have copies of the files that you have customised of the ''interface folder'' so you have ability to copy them back into installation if you do overwrite with the release version.   
If you have edited any files that sit on your '''web site''', to give the look you desire, or the content you want (e.g. adding rain this month to this month page, or combining this month and this year page), see if the release notice says that file has been revised, if it has not then it is easy to keep your edited file by not copying in the replacement file from within the zip. If the release revises any file you previously edited, take a backup of your edited file, before you copy the new file into your folder. You can then use a file comparing tool to see what has changed in the release and what you changed and hopefully manage to merge to a new file that keeps any functionality change in a new release and keeps your customisation.   


If you have done major customisation to the standard website then you probably have followed the guidance and stored your new web page templates in a different directory and you use '''Extra Files''' to specify where they are, so they cannot be overwritten and the new MX will still find them.  
'''You do have to be careful''', as many releases change the interface in some way, even if the change is not in the file you have customised, it might be in an asociated file, and all the various components of the interface have to work together as a coherent unit. For instance when feels like was added to MX, the api used for sending data from the MX engine to the admin interface was updated. In some releases, a new web page is added to the admin interface, with the consequence that all the navigation menus were adjusted in all web pages. Any HTML page in the admin interface may be edited to refer to a different styling page (so it, and the .css file, must be updated together) or to a different script (so the .HTML, and the .js file, must be updated together).  The settings pages in the admin interface are dependent on the correct pair of json files to define the options or values that appear on that web page, so again all must be consistent.


=== admin interface files ===
To sum up, admin interface files are interdependent, and you cannot always update some, but not all, of the admin interface pages.


If you have done any customisation to the '''interface''' (perhaps you don't like seeing solar in the tables when you don't measure that) then again you can skip over these files when copying. Hopefully you will have copies of the files that you have customised of the ''interface folder'' so you have ability to copy them back into installation if you do overwrite with the release version.
Be prepared to go back to the standard file for whatever you customised if something it depends upon has changed, after all you must not lose any vital functionality.


You do have to be careful, as many releases change the interface in some way and all the various components of the interface have to work together as a coherent unit. For instance when feels like was added to MX, the api used for sending data from the MX engine to the admin interface was updated. In another release, adding a new web page, all the navigation menus were adjusted in all web pages. A HTML page may be edited to refer to a different styling page (so it and the .css file must be updated together) or to a different script (so the .HTML and the .js file must be updated togetherMany web pages are dependent on the correct pair of json files to define the options or values that appear on that web page.  So files are interdependent, and you cannot always update some but not all of the admin interface pages.  
On my site, my own versions of interface files have a "_" (underline character) added to the start of the standard MX file name. This applies to both HTML pages, and JavaScript files that I have edited. I edit the menu items within my edited pages so those all go to my versions where I have a HTML customised page, leaving unchanged the menu items that can still go to a standard MX web page where I don't have my own version of the .HTML page.  This makes it easy for me to navigate between my pages, as all of them link to my other pages.  If I am on a standard MX page and want to go to one of my customised pages, I select the equivalent standard page, then edit the URL to add the underline and get easily to my pageThis naming means I can always use a standard page instead of my customised page when I need to, and I never miss out on any new features.


Be prepared to go back to the standard file for whatever you customised if something it depends upon has changed, after all you must not lose any vital functionality.
== After an upgrade if you use the standard web template files ==


On my site, my own versions of interface files have a "_" (underline character) added to the standard MX file name (before the "." and the relevant extension). This applies to both HTML pages, and JavaScript files that I have edited. I edit the menu items within my edited pages so those all go to my versions where I have a HTML customised page, leaving unchanged the menu items that can still go to a standard MX web page where I don't have my own version of the .HTML page.  This makes it easy for me to navigate between my pages, as all of them link to my other pages.  If I am on a standard MX page and want to go to one of my customised pages, I select the equivalent standard page, then edit the URL to add the underline and get easily to my page.  This naming means I can always use a standard page instead of my customised page when I need to and I never miss out on any new features.
If you do not use the example web template files, provided as standard, with a MX release, the remainder of this section can be ignored.


== After update - checking for bugs in MX or mistakes in your installation ==
If you use the third party [[CumulusMX and Cumulus1 UI style Multilingual Websites]] web pages, as maintained by BCJKiwi, follow any instructions in his package as to which of the files in this section are required and any announcements in his forum topic about when such files need to be upgraded.


Start the new installation of MX and watch out for any errors - If the device you run MX on has a monitor, then look in the terminal/command window. In all cases look at the latest file in the MXdiags folder to see if any errors are reported.
=== web folder from zip ===


Also keep an eye on the support forum for first few days, some releases do have bugs, as developer cannot test all possible configurations.
If you do use the example web templates to produce web pages for your web server, and you have copied all files from the release distribution ....


== Updating if you use a virus Checker ==
....MX will find any new web template files in your '''web''' folder,  MX will process those template files, and upload the web pages produced, ''as before with no further action by you''.
You may find that virus checkers such as Windows Defender reject your new version of MX. They need to be told it is safe.


== Updating if you use the start/stop management script ==  
=== webfiles folder from zip ===


This section contributed by ''Jank'' on support forum.  Note the version on the forum might have been updated from the original included here.
<br/>
<big>PLEASE COULD SOMEONE USING LATEST MX RELEASE ENSURE THIS IS KEPT UP TO DATE</big>


1. look on [[Software|Software download page]], find the link to latest version, and fill out the '...' below appropriately as you run these 2 commands on your device where you do downloads:
<br/>
<pre>cd /tmp
wget https://github.com/cumulusmx/CumulusMX/ ... .zip</pre>


2. Once that download is complete, start cumulusmx.sh with option -u
You do need a further action, if any files in this folder have been updated. The updated files must be uploaded to your web site.
<pre>/home/pi/CumulusMX/cumulusmx.sh -u</pre>


3.When asked for the zip file, enter
This may be easy to forget if you use the standard web templates, because you thought that upload was a one-off and maybe you cannot remember what you did.
<pre>/tmp/CumulusMXDist</pre> and hit the TAB Button


4.Choose the zip file with the CumulusMX update and hit return.
Certain third-party web-pages still use files from that folder, if one of the files they use has been updated, then ensure you have a backup of your web site before you upload the updated file, it should work, but it might not, and you might need to regress.


5. Follow the on screen instructions
Let me go through the files currently in webfiles folder:
*'''weatherstyle.css''' has not been updated since its last update by Steve Loft in 2009.
*subfolder '''images''' contains 2 pictures that have stayed same since MX launch
*subfolder '''js''' contains one file or two files (depending on which MX release you are using:
**The standard web page '''trends.htm''' will only work if the correct version of any file here has been uploaded to your web site, and placed in a sub-folder "js" of where "trends.htm" is installed. .
*** BCJKiwi's user interface '''charts.php''' also requires the same JavaScript file (or files?), so upload the file(s0 anytime there is a change in a MX release.
** The file '''\CumulusMX\webfiles\js\cumuluscharts.js''' has changed in a lot of releases, it is updated whenever there is a change in the JSON files used for providing data to the charts, and you must use the right upgrade.
** The file '''historiccharts.js'' (it was introduced from release 3.8.7) is similar to previous script, but drives the historic chart functionality on your web server
*subfolder '''lib''' currently contains 3 folders:
**'''highstock''' - this folder is no longer needed on your web site (and the most recent MX releases have emptied it)
**'''jquery''' - this folder is needed for "trends,htm" to work, but it has not been updated since 2014. Both the files included are obsolete packages, no longer officially available, and there are no plans to replace them.
**'''steelseries''' - this folder has 3 sub-folders, I can only think of 2 MX releases when anything was changed here, and I don't anticipate any further changes.


6. With each update component .....you can choose: [y]es, [n]o, [A]ll, [N]one, [r]ename
= After upgrade - checking for bugs in MX or mistakes in your installation =


I would recommend select '''A''' as that will simply replace all files without further action.
Start the new installation of MX and watch out for any errors - If the device you run MX on has a monitor, then look in the terminal/command window. In all cases look at the latest file in the [[MXdiags_folder|MXdiags folder]] to see if any errors are reported.  


In newer releases of MX, also see [[ServiceConsoleLog.txt]]


CumulusMX will be restarted after update completes.
Also keep an eye on the support forum for first few days, some releases do have bugs, as developer cannot test all possible configurations.


You can check if the update was successful by using option -s:
== Updating if you use a virus Checker ==
<pre> /home/pi/CumulusMX/cumulusmx.sh -s</pre>
You may find that virus checkers such as Windows Defender reject your new version of MX. They need to be told it is safe.

Latest revision as of 09:32, 16 March 2023

Who this article is for

Please be aware that if you want to move from Cumulus 1 to MX, you should read Migrating_from_Cumulus_1_to_MX article instead.

This article is for those who already use MX, and so are comfortable with basic MX installation and running.

Who is not intended reader

Cumulus MX has been updated so frequently in 2020, that you may be used to upgrading to a new build, and for you this article is not useful.


Where to download a release distribution

Latest release

Download release distribution zip from the Software article in this Wiki for latest version.

Be aware that the developer needs to remember to update this link each time there is a new release.

An image that contains a Raspberry Pi lite operating system and a recent CumulusMX release already included, can be downloaded from the same wiki article. Not every CumulusMX release is provided as a Raspberry Pi image, so you may need to perform an update to get the latest release. The Raspberry Pi image places CumulusMX in /opt/CumulusMX.

Any release developed by Mark Crossley

Download release distribution zip from https://github.com/cumulusmx/CumulusMX/releases for earlier versions.


Introduction to upgrading MX

Crystal Clear info.png This document is 'Work In Progress' so content may not be complete or accurate!

You can download and unzip in advance, but cannot replace existing files while they are being used, so how you stop MX is in next two sub-sections.

My preference, after download, and unzip into a holding area, is to copy (it would be file transfer if you download on another device) over my existing installation all files except:

  • CumulusMX.exe
  • CumulusMX.exe.config

while MX is still running. I then stop MX for minimal time needed to replace just those two files before I restart MX. That way I minimise downtime, especially useful for a patch release when few files have changed, as it takes some time to replace all the files in the installation.

For simplicity, in guidance below, MX is stopped before any files are replaced.

Upgrading on a Raspberry Pi (RPi) computer by commands from a Microsoft Windows computer (PC)

I recommend you download WinSCP (this and PuTTy share some settings). This will allow you to connect your PC to your rPi and copy files back and forth. You can use it to copy the MX distribution, you download and unzip on your PC, onto the RPi. You can also use it to take a back-up of your RPI files onto your PC, and to pick certain files off your RPi. e.g. copy log files from MXdiags folder to your PC to read diagnostics etc.

You can download PuTTy (shares some settings with WinSCP) to enable you to open a terminal session and send commands to your RPi computer.

If you have Cumulus MX setup to run as a service then you will need to do one of these, should you wish to add debugging when the upgraded MX begins...

  1. Edit the CumulusMX service file (see MX_on_Linux#Running_as_a_service) to add the -debug parameter (do this if always want debugging on, not usually needed)
  2. Just set it from Program Settings | Logging Options (easiest for turning debug on when you need it and off when you don't need it)
  3. Edit the Cumulus.ini file and enable it in there (no longer recommended)

Upgrading if you run MX as a service

Don't forget to stop the service, before you do the upgrade.

For a Linux Operating system: sudo systemctl stop cumulusmx.

For Microsoft Windows Operating System:

Open Windows Administration Tools from Windows Task Bar, select Component Services, click on Services (local) In middle window select CumulusMX service from the list; click on Stop the service option. (click on Start the service after upgrade)

Upgrading if you run MX interactively

If you do NOT run as a service, you will want to use Control and C in the terminal (or command) window running MX to make the software close tidily.


Updating to the next MX release if you have not upgraded before

The simplest upgrade is from the immediate preceding build, and the steps required are summarised as follows:

  1. Download the release distribution zip for the next build (see later for where from)
    • My advice is to have a separate download location away from the location where you are installing/running the software; it can retain older releases to make regression simple if the new release proves to have a bug.
  2. Use Control-C to stop Cumulus MX (see later if running as a service)
  3. Take a backup of your complete existing MX installation (as it is not running, no files will be locked)
  4. Unzip the new distribution, overwriting the previous installation (the release announcement might ask you to delete obsolete files)
  5. Run any one-off batch scripts needed to prepare for upgrade
    • These might be needed if log files in the new release contain fields that were not in earlier release
    • These might be needed if you use optional database functionality (see later for examples)
  6. Do any actions that require you to use Admin Interface to change settings
  7. Do any one-off actions in relation to any files provided by MX for you to run a web site with the default pages
  8. Restart your Cumulus MX (consider running with -debug parameter if you are not sure the new build is bug free)

The remainder of this article explains all options for updating, and is thus less simple than above.

Installer Option

HansR on support forum has developed an multi-platform installer, see Multiplatform Installer for CumulusMX which makes life a lot easier if you do not wish to dive into the (file level contents) internals of CumulusMX.

  1. Copy the InstallCMX.exe to any directory you want, on the drive where you wish to install (or have previously installed) MX.
  2. Copy the CumulusMX release distribution zip(s) to that same directory, if you have not already done so. You may have more than one distribution in the same directory.
    • The install procedure gives you the possibility to select, or define, the Archive to install, and the location where to install.
  3. Stop CumulusMX
  4. Run InstallCMX and confirm / fill in (on the console) where you wish to install (or update) CMX. The default for Windows is C:\CumulusMX\ and for Linux it is : /home/CumulusMX. The Installation directory can be modified.
    • You can give the build number, for the release distribution zip, to install as command line argument.
  5. Start CumulusMX
NOTE: On Windows you run the installer as any other command line executable and it is best to open a command window in which you start the installer. On Linux you run it on the command line as "mono ./InstallCMX.exe", the mono command can be omitted if mono is already active (e.g. if you run CumulusMX as a service, and stop it, mono remains active).
NOTE: In an existing installation with modified files, make sure those are in a different, either filename or folder (a safe place). If they have the same name as files in the distribution, they will be overwritten.
NOTE: Check in the webfiles directory to see if any files there have been modified, since your last upload to your web server. If so, move those new files to the website.

After the installation, there is a log file. Check the log file to see everything has gone well. There is an ini file where you can control:

  • NormalMessageToConsole=true or (default) NormalMessageToConsole=false
  • TraceInfoLevel=Warning (out of: Error, Warning, Info, Verbose, None)

Any reactions (thank you, issues and questions) please post in the download thread of the installer.

Modifications and additions on user request can be discussed (e.g. think about automated start/stop, upload of webfiles to the directory etc....), use the talk for this page.

Upgrading if you are running MX on a Linux computer

You might want to read galfert's post on the support forum here for the relevant Linux instructions in a concise format.

If you want more detailed information, on any Linux (including Raspberry Pi operating system) aspects of the upgrade instructions on this page, see MX on Linux page.

Updating if you use the start/stop management script

This section contributed by Jank on support forum. Note the version on the forum might have been updated from the original included here.

1. look on Software download page, find the link to latest version, and fill out the '...' below appropriately as you run these 2 commands on your device where you do downloads:

cd /tmp
wget https://github.com/cumulusmx/CumulusMX/ ... .zip

2. Once that download is complete, start cumulusmx.sh with option -u

/home/pi/CumulusMX/cumulusmx.sh -u

3.When asked for the zip file, enter

/tmp/CumulusMXDist

and hit the TAB Button

4.Choose the zip file with the CumulusMX upgrade and hit return.

5. Follow the on screen instructions

6. With each update component .....you can choose: [y]es, [n]o, [A]ll, [N]one, [r]ename

I would recommend select A as that will simply replace all files without further action.


CumulusMX will be restarted after upgrade completes.

You can check if the upgrade was successful by using option -s:

 /home/pi/CumulusMX/cumulusmx.sh -s

Considerations that determine when to upgrade

Cumulus MX will nag you, in various places, to make you aware if you are not running the latest build. Some people will choose to upgrade as soon after a new release as they can. However, each upgrade does involve a period when MX is not running, and that causes some loss of data:

  • for some weather station types readings taken every minute, or more frequently, are replaced by whatever period you station data logger records at;
  • for other stations, without their own logging, all data is lost for the period when MX is not running.


Other, more cautious, people (like the present writer) will not upgrade each time a new release becomes available (and there are a lot of new releases in 2020), here are some of the reasons:

  • you will see in the Cumulus Support Forum that many builds have bugs, and so you realise that it is better to stick with your fully working release, than install one with bugs;
  • you may wish to avoid the loss of data mentioned above, by minimising the number of times that you stop MX;
  • you may run MX on a computer that you rarely visit, perhaps even in a remote location not often visited, so you prefer to leave it untouched, rather than risk possibility of being unable to restart remotely;
  • you may just have more important ways to use your time than updating your MX software, and some new builds might not give you any benefits that make it worthwhile to change your priorities.

It is perfectly possible to upgrade from rather old versions of MX, to the latest, skipping intermediate versions, but there are some key versions that you should not skip over. This page, in subsequent sections, includes suggestions for which releases to install (and get running) before moving onto newer releases. This will be especially useful for those people who do not immediately upgrade to new releases, as per above suggestions.

Advice about skipping versions

Please see Updating_MX_to_new_version#Updating_from_a_very_old_version sub-section later on this page (and preceding sub-sections where relevant)

The important point to make here is that when you do an upgrade, you should copy in ALL files from the release distribution you want to run next. This is because there are dependencies between files within a distribution, so missing out any particular file may stop other files from working. However, if you are skipping versions, read the in-between release announcements to see if there are any one-off actions.

What to read (and when) before upgrading

Upgrading from immediately preceding build

The release announcement

The release announcement is found in Cumulus MX Announcements and Download - PLEASE READ FIRST topic of the support forum.

  • Generally, a release announcement WILL contain
    1. An overall purpose for that particular release (e.g. fixing bugs or adding functionality)
    2. Details of what functionality has been added
    3. Details of what bugs have been fixed, or where the processing has been improved
    4. (For older releases only) A list of the main files that have been added or amended in the release (excluding "updates.txt" which is amended in each release)
      • (Publishing a list of changed files has been discontinued, because people were updating selected files, and often missed crucial ones, like CumulusMX.cfg)
  • Sometimes, a release announcement MAY contain
    • Descriptions of one-off actions (like changing the schema if you use a database, or editing your web pages to take advantage of new web tags).
    • Actual scripts to download and run to perform the necessary one-off actions.
  • Be aware that it is worth while checking back on the release announcement for a few days after a release.
    • It may have been edited because the original announcement forgot to mention something.
    • It may have been edited to mention that some bugs have now been found
      • That may mean you are advised to regress to an earlier version and use that
      • It might mean that some supporting files in current version are wrong, and you only need to regress those named files
      • There might be an emergency release to fix the bugs, and you need to upgrade to that emergency release
      • Finally you might be given advice to avoid using certain parts of the functionality or take some other action until the next release is available.

Other places where you can find information about release content

Although it is not always kept in step, a concise summary of all formal MX releases is available at Cumulus_MX_formal_release_versions.

You can also view the latest Updates.txt.

Deciding whether to upgrade to new release

This has been covered earlier in this article, but I repeat here 2 critical considerations:

  • Any new development or change in a new version of MX might cause problems for some users. You might want to stick with the version you are already using unless you really need any new functionality or the fixes gained by upgrading.
  • Also remember that there are bugs in (almost) all versions of MX, this is a large and complicated package, and the current developer has not been able to test all the code with all possible settings and all possible weather stations.

Updating to a new minor build, skipping in-between minor builds

For a minor version build either the associated version number does not change or only the final section changes (3.x.y to 3.x.z).

Reading multiple release announcements

If you are skipping some intermediate builds, then you will need to read each of the formal release announcements for builds after the build you currently use.

The other sources of release information mentioned above may be consulted as an alternative and may highlight something useful, but the formal release announcement contains what the developer wants you to know.

Action to take when skipping minor intermediate builds

You need to apply the cumulative actions recommended (i.e. apply any actions in sequence as listed for each intermediate build). Unfortunately, for Cumulus MX, one cannot assume that minor builds have no one-off actions associated with them. Luckily, most of the advice above for updating from immediately preceding build still applies.

Updating to a new major version

Generally, if the developer decides a new build warrants classification as a major version (i.e. 3.w.0) then the change being implemented is significant enough that updating might be more complex.

Examples of what might be classified as a major change

  • Additions to fields in log files (whilst older lines can continue to be read, you might want to consider amending your standard data log by populating the additional derivatives for older lines)
  • Schema changes for database tables (if you don't use the standard tables, this might be irrelevant to you), the standard insert operations by MX can cope with extra columns in a table, but will fail if any column they try to update is missing
  • Additions, changes, or removals, in configuration file (in general MX will ignore unrecognised parameters, but it does need correct values for those attributes it does recognise), see release announcements to discover if new lines in configuration file need you to add them manually or you need to use the admin interface to select the right setting for you
  • New pages in Administrative Interface (additional pages require menu on every other page to be amended)
  • Changes to existing pages in admin interface that involve changes to associated files (it is vital that all associated files are from same build)
  • Catering for new weather station sensors, or new ways of communicating (this might not seem relevant to you)
  • Any interface functionality changes (it may look the same, but what it does has changed)
  • Additions to files being generated for web server (i.e. changes to template files, or additions to web or webfiles folders), even if you use your own versions of such files you may want to check the revisions

What you need to read

  • Basically, check the corresponding release announcements for every version since the one you have been using before planning your upgrade.
    • Make a note of any one-off actions required at particular in-between versions, remember these actions are only in forum release announcements.
    • Although one-off actions will not be described in the Wiki (whether on the Software page or the Cumulus_MX_formal_release_versions page), the Wiki can give you an idea of what functionality has been improved to help you decide whether to upgrade.
  • It is still worth reading all the points made above for updating from immediately preceding build.

Doing the upgrade

  1. Download the release distribution
  2. Use control-c (or equivalent for MX running as service) to stop your existing MX software
  3. Backup your existing installation
  4. Do the on-off actions identified (e.g. changing schema of database tables, adding new configuration
  5. Unzip the distribution over existing one (alternative installation options listed later)
  6. Restart MX, considering whether to use -debug parameter in case the new release does not work for you
  7. Check the latest file in the MXDiags folder after an hour, to see if there have been any problems with reading from station, real-time uploads, standard uploads, or hourly uploads. (You may wish to recheck after an End of Day action has been done).

Updating from a very old version

Many people believe if it works, it don't need fixing. So they install whatever MX version is available when they start using MX and ignore all the bug fixing and new functionality being added, and stick with their existing installation.

Suddenly they realise that perhaps their version does lack functionality that would be useful to them. Maybe their version does not have an editor for correcting rogue extreme records and they realise correcting log files is something that is hard for them to do.

This section is therefore included to give advice on why you should not jump from an old version to the latest directly, but how it can be safely done in stages.

Recommendations for staged updating

This section will need to be updated, new contributors are needed to keep this advice current.

Do read all release announcements, to see if the developer has made a major change since those described below that means you need to stage your updates if you are currently running an old release.






If using a release up to 3.11.4 and want to use any later release

Upgrade exactly to release 3.12.0 (no later) by using zip at download/b3141/CumulusMXDist3141.zip.

Run that release, it will rename your Cumulus.ini file, and create a new file with same name, but different settings.

You need to work through all the settings pages in the interface, and ensure all settings are correct, then close MX so it writes away the new settings according to your preferences.

Now download the latest release, and copy in all the files from the zip, before restarting MX at the release that is supported by developer.




Currently running 3.10.x or 3.11.y

Treat like any other major version upgrade as described above.

The changes between any 3.10.x release and any 3.11.y release were minor, but you can't go from 3.10.x to the latest release, because you must use 3.12.0 to convert Cumulus.ini.




Currently using one of the 3.9.y releases

If you are using any release in the 3.9.y series (note release announcement 3.9.1 warning about Mono if you use that), and you are using web pages that were provided by MX...

Then you need to be aware that 3.10.1 (3.10.0 was withdrawn) got rid of the HTML templates that you have been using, and introduced a lot of new settings.

The good news is that you can now jump straight to 3.12.0 as that release will rename your existing Cumulus.ini and create a new file with all the new settings:

Note: The folder CumulusMX/webfiles-legacy mentioned in the new default web site information page, with some alternative web pages that have no ongoing support, is only available from a 3.10.0 or 3.10.1 download (find from this Github page).



if using 3.7.0 release, any 3.8.x release, or 3.9.y release

See sub-section directly above, that describes similar process,

  • EITHER upgrade directly to 3.9.6, or any later build, within 3.9.x, without any major change,
  • or to 3.12.0 with major change
  • (find either release from this Github page).

If you are using a release earlier that 3.7.0, you can note 3.7.0 was only build in 3.7.y series, but because 3.7.0 introduced a lot of changes these staged upgrades recommend that 3.7.0 is implemented, and run for a while, before continuing to upgrade.

Be aware that 3.8.0 was a major release, as it introduced the ability to run Cumulus MX as a service, but there is no reason to install it as the ability to run either interactively, or as a service, continues to be available in all subsequent releases.

It is optional to install release 3.9.6 build 3101, because that is a safe release to use while there were bugs in the builds in all 3.8.z versions, and in other 3.9.x releases, however please action the one-off changes noted as IMPORTANT here:

IMPORTANT one-off actions needed:

Be aware that 3.10.0 was withdrawn, but it was a major release that totally changed the files in Web folder and Webfiles folder within the release download. However, you can skip directly from 3.9.6 (if you have implemented that) to 3.12.0 as soon as you are happy to change your web server contents (if you are using web pages that MX provides), see sub-section above for further advice re web pages. You should continue in stages because it is mandatory to install 3.12.0 as only that release can rewrite your Cumulus.ini file ready for subsequent release, so ensure that 3.12.0 is working before installing current release as instructed in sub-sections for those using subsequent releases.

If using a 3.5.x release

My advice is to upgrade directly to 3.7.0 available at Mark's Github repository.

You should skip the intermediate releases because several 3.6.y releases have bugs in them, and you want to avoid those problems (the bugs vary in severity between mistakes in calculations done by MX to particular functionality not working).

But there may be additional actions you need to do when moving from 3.5.x to 3.7.y, depending on what features you use in MX:

  • there are no additional actions if you use standard web pages and you do not use database tables, just enjoy the bug fixes and extra features after your update!
  • If you use database tables, be aware that the schema (Column names that must be in a table) varies between 3.5.x versions and 3.7.y versions.
    • The updating database table features in MX will only work if all the columns named in each such upload update already exist in the database table it is trying to update.
    • Look at individual release announcements for both 3.6.0 and 3.7.0, to see the SQL provided for adding the columns added at particular versions.
      • At 3.6.0 the SQL was provided as a separate attachment to the release announcement (read UpdateMYSQL-b3076.zip)
      • At 3.7.0 the SQL was provided as part of the main zip (read AlterSQLTables3098.sql)
      • In both cases, the SQL provided assumes you are using the default names for database tables, it does not read Cumulus.ini to see whether you have selected different names for the tables (unlike ExportMySQL.exe which used to be provided in the MX release zip with standard CumulusMX.exe; both of those (at MX release 3.7.0) check what tables names you have selected for updates).
  • If you have your own customised web pages, then there are changes to web tags that might lead to you needing to edit your web pages.
  • If you use commas to separate integer and decimal parts of real numbers, then various releases from 3.6.0 to 3.7.0 add "rc=y" to various web tags, that option will replace the decimal commas you use by decimal points that are required for some script languages (like the JavaScript used by HighCharts), and that makes it easier if you want to customise your web site.

When you are happy with running 3.7.0, then you should continue to upgrade, but in steps (optionally try 3.9.6 because that will test some new features, then mandatory upgrade to exactly 3.12.0 which will rewrite your Cumulus.ini file, then you can continue your upgrade to latest) as described in earlier sub-sections for later releases.



If using either 3.1.x, 3.2.y, 3.3.z, or 3.4.w releases

First, upgrade to 3.5.1 by downloading it at [Mark's Github respository]. The actual installation is done using the instructions early in this Wiki page for simple next build upgrades. You can safely skip reading the intermediate release announcements, as there are no special one-off actions. There are one-off actions at 3.5.1, see release announcement. It does not involve any updates to the fields in the log files nor to the columns in any database tables you use.


When you are happy running 3.5.1, then you should continue to upgrade, initially follow instructions given for later releases




If using 3.0.0 (the MX original beta)

Upgrade from the beta directly to 3.5.1 by downloading it at https://github.com/cumulusmx/CumulusMX/releases/tag/b3072. There are one-off actions at 3.5.1, see release announcement. This skips you past the problems in 3.5.0. It gives you benefits introduced in 3.1.x, 3.2.y, 3.3.z, and 3.4.w releases.

This gives you essential new functionality in the admin interface like editors for the log files and extreme records. But it also fixes multiple bugs in the beta you were using and adds some useful validation missing in the beta.

It does not involve any updates to the fields in the log files nor to the columns in any database tables you use.


Now follow instructions in other sub-sections, to upgrade in stages to where there are significant actions to do, until you reach latest release and get support from developer.

Knowing when a new release is available

CREDIT

Thanks to Billy on support forum for suggesting text for this section.

Using forum notifications

As new releases are announced in Cumulus MX Announcements and Download - PLEASE READ FIRST topic, you can use the spanner tool to subscribe to this topic to receive notifications of a new post announcing a new release (or any other release-related announcement).

MX terminal message

If ...

  1. ...you have a monitor to see the terminal output from the Cumulus MX engine (Windows calls this either a command window, powershell window, or a terminal window depending on how you invoke it, for Unix-based implementations this is the output window when using the terminal functionality), AND
  2. ...your device running MX is connected to internet, AND
  3. ...your MONO (if not Windows) is not obsolete (SSL certificate out of date), AND
  4. ...you restart MX

... then you will see a prompt when a new version of MX is available.

It is worth stressing that if you leave MX running, then this feature will leave you blissfully unaware that an update is available; it only checks when MX is restarted.

MXDiags

In addition ...

  1. ...if you can view the MXdiags file for the current session of MX, AND
  2. ... MX is running with connection to the internet, AND
  3. ...you restart MX

... if a new version of MX is available, the MXDiags file will say so (the message is not easy to spot as there is a lot of output before it, and variation in what output appears before it). Anyway, here is one example, just one example as in my experience the message has appeared at different places for each of the recent upgrades):

2020-05-27 04:18:48.326 Calculating sunrise and sunset times
2020-05-27 04:18:48.326 Sunrise: 04:58:11
2020-05-27 04:18:48.326 Sunset : 21:19:54
2020-05-27 04:18:48.326 Tomorrow sunrise: 04:57:08
2020-05-27 04:18:48.326 Tomorrow sunset : 21:21:11
2020-05-27 04:18:48.388 You are not running the latest version of CumulusMX, build 3080 is available.
2020-05-27 04:18:48.763 Station type:

There is no message in whatever place it can appear ...

  • ... if you are using latest version, OR
  • ... if you are not connected to the internet, OR
  • ... if you keep MX running all the time!

Start/Stop script

When the Status option of this script is used, whenever a new release of MX is available, it will output a message.

Dashboard in MX Admin Interface

In recent releases, one of the alarm indicators shown at the bottom of the dashboard page of the administrative interface, will flash red when you are not running latest build, and glow green when there is no newer release compared to the one you are running.

Web Tags

Since 3.7.0 release of MX, two relevant web tags have become available:

  • <#NewBuildAvailable> Returns a boolean value
    • 0 - MX running the latest build available
    • 1 - MX is running an earlier build than the latest public release
  • <#NewBuildNumber> Displays the latest public release build number


Recommended Actions when Updating

Back-ups

It is always best to take a backup of your existing MX installation before you do an upgrade, this allows you to regress back to the earlier version if either you mess up installing the new version, or the new version has a issue that prevents it working with the versions of other software (like MONO) that your installation uses.

The two approaches

Contributors to this section are named in text.

Some people upgrade by just copying in the files that the release announcement says have changed, others copy in all files from the downloaded zip. The first should only be used with caution, files like CumulusMX.exe.config can change between versions, but not be mentioned in a release announcement, and the developer will have been making edits to files since the previous release, and might forget exactly which files have been edited between releases. Also you may be upgrading from an earlier version and therefore be skipping several intermediate releases.

You may be able to see the dates when files were changed within the zip and therefore be able to decide for yourself if you compare those dates with the previous release you were using if you have kept the download for the version you were using. However, do not assume that only files with later date are needed; in some releases the new build regresses one or more files and so a file with an older date is the correct one to retain.

The rename old approach

  1. The popular approach, recommended by many forum contributors, in many different posts (including at this post by Mark Crossley for example is to rename your current install directory, then unzip the new release, letting it create a new CumulusMX folder (or whatever name you prefer and specify in unzip options).
  2. However, you will find that this option is not being recommended any more, because it relies on you knowing which files (see next 2 points) you need to copy back.
  3. Copy across Cumulus.ini and (if you use it) string.ini into that new directory, and then copy the contents of the data and Reports directories from your current install to the new install.
  4. Don't forget to copy any other set-up files across too.
    • The advantage (as Mark says) is that you ensure you do use all the files in the new release, and don't miss out any he may have forgotten to mention in his release announcement.
    • A key advantage is that you don't retain files that an old release migt have needed but the new release does not use.
    • Another advantage (as PaulMy says here for example) is that you retain your old set-up intact and can easily restore it should you have a problem with new release.
  5. Various people advise that you don't delete your old installation for a week or so; as you may notice something from the older version that you haven't copied across!
  6. Check again that you copied across (you may not have all of these files, but if you have them in old installation then you need them in new installation) strings.ini, twitter.txt, Cumulus.ini, and similar files in the same folder level as CumulusMX.exe, as well as all the files in the data and Reports sub-folders. Then, provided you are happy, you can delete the old release when you want to reduce clutter on your storage discs.

The overwrite approach

The general advice is that if you want custom pages for the administrative interface, or custom templates to process to produce your web pages, then these custom files should be given different names (and ideally be placed in different folders) to the standard files included in the release distributions. If you follow this advice, the approach described in this sub-section is the best for you, if you don't follow this advice, see later section on dealing with customised files.

    • if you have a lot of set-up files, or other custom files, (i.e. files with names that are not part of release), these won't be overwritten by files in the release distribution, but you do want to keep them;
    • or if you are downloading on a different computer, or on a different disc, to where you are running MX, it is easiest to keep your MX installation in the same place
  1. Note the forum administrator, David, (see this post) recommends this particular approach.
  2. After downloading a new release unzip it on the device/disc where you down load it. Next stop your existing Cumulus MX software and take a backup of your existing installation. After that, simply copy the files (optionally only those that have newer dates because they have changed) into the existing MX directory on the device where you run MX. Then you know all your existing files are there, and your MX can be run as before.
  3. The great advantage of this approach, and the reason that it is widely recommended as best, is that you can't lose any data or configuration files (i.e. files that are not part of the release distribution).
  4. Provided you did the backup, the only problem with this approach is that in some releases files are removed from the distribution, yet unless you check carefully, this approach may leave files that are no longer required in your installation, and it is just possible you might continue to use the obsolete file instead of the intended alternative. Such a non-standard installation is difficult for others to support. So do check through file by file to ensure you only have files in the distribution you have just installed, not any from earlier distributions. The only files that you still want (that are not in distribution) will be those that either configure your system or contain your data in log files.
    • The updated files can be tracked by their modification date, some will be specified in the release notes (find them on this forum). There is a list indicating what files vary in this Wiki article.
    • In general, especially if updating a major release, it is best to overwrite with all files in zip, whether they are listed as changed or not. This ensures you have a consistent set of files, and avoids any issues when the developer does not correctly list all the files that have changed, or a file might be regressed to an earlier version and not have a newer date than the file in your previous installation.

Example: downloading on PC and installing on pi

  1. On your pc: Download zip either from Software for latest version or from https://github.com/cumulusmx/CumulusMX/releases for earlier versions.
  2. On your pc: Use Filezilla (or similar file transfer program, or it might even be a copy over network depending on way set up) to transfer zip to pi (e.g. FileZilla CumulusMXDist3089.zip to /home/pi), but remember you might want to install to a different location (perhaps on an external USB drive, or SSD)
  3. Control C on pi to stop MX
  4. On pi go to directory where zip now is, for example it might be cd /home/pi
  5. Unzip the installation over your existing installation, by running on your Raspberry Pi this instruction: unzip -o CumulusMXDist3089.zip
  6. Restart MX
  7. Remove any files that were needed by earlier releases but release announcement says are not needed by the release you are installing

The customised approach

If you edit a file that is included in the release distribution, you should give it a different name (e.g. I have placed an underline in front of original file name for some of the files I edited in the admin interface), and/or place it in a different folder (e.g. I place all the cumulus templates I have created in a folder called cumulus_Templates). If you follow this advice, then you won't lose your files by following the advice above, regardless of whether you overwrite or not.

If you won't listen to best practice advice, and have decided to for example modify the provided Cumulus web templates without allocating new names, then installing the whole release distribution will overwrite your edited file(s) with the standard file content provided by the developer. Therefore you need to do a customised approach where you select which files from the distribution to copy into your installation. If you have changed one or more web templates, you might think it safe just to omit the relevant files in the zip within folder web, but some builds change the files in the webfiles folder too, and you need to retain consistency.

Updating when files within release might overwrite your own edits

To further explain the customised approach, here are some more examples.

web site files

If you have edited any files that sit on your web site, then these can remain on your web site. However, if those web pages are generated from templates that you expect MX to process and upload for you, you do need to take special action. Maybe you edited a template to give the look you desire, or the content you want (e.g. adding rain this month to this month page, or combining this month and this year page).

The first step is to see if the release notice says that file has been revised, if it has not then it is easy to keep your edited file by not copying in the replacement file from within the zip. If the release revises any file you previously edited, take a backup of your edited file, before you copy the new file into your folder. You can then use a file comparing tool to see what has changed in the release and what you changed and hopefully manage to merge to a new file that keeps any functionality change in a new release and keeps your customisation. Do think about changing name of the template file, or placing it in a different folder, and using the Extra Files settings. That will make it easier for you next time you do an upgrade to a new MX build.

If you have done major customisation to the standard website then you probably have followed the guidance and stored your new web page templates in a different directory and you use Extra Files to specify where they are, so they cannot be overwritten, and the new MX will still find them. It might be that new web tags have been added since whatever build you were running before, and that you decide to edit your customised template file, but that can be done any time after you install the new MX build.

admin interface files

If you have done any customisation to the interface (perhaps you don't like seeing solar in the tables when you don't measure that) then again you can skip over these files when copying. Hopefully you will have copies of the files that you have customised of the interface folder so you have ability to copy them back into installation if you do overwrite with the release version.

You do have to be careful, as many releases change the interface in some way, even if the change is not in the file you have customised, it might be in an asociated file, and all the various components of the interface have to work together as a coherent unit. For instance when feels like was added to MX, the api used for sending data from the MX engine to the admin interface was updated. In some releases, a new web page is added to the admin interface, with the consequence that all the navigation menus were adjusted in all web pages. Any HTML page in the admin interface may be edited to refer to a different styling page (so it, and the .css file, must be updated together) or to a different script (so the .HTML, and the .js file, must be updated together). The settings pages in the admin interface are dependent on the correct pair of json files to define the options or values that appear on that web page, so again all must be consistent.

To sum up, admin interface files are interdependent, and you cannot always update some, but not all, of the admin interface pages.

Be prepared to go back to the standard file for whatever you customised if something it depends upon has changed, after all you must not lose any vital functionality.

On my site, my own versions of interface files have a "_" (underline character) added to the start of the standard MX file name. This applies to both HTML pages, and JavaScript files that I have edited. I edit the menu items within my edited pages so those all go to my versions where I have a HTML customised page, leaving unchanged the menu items that can still go to a standard MX web page where I don't have my own version of the .HTML page. This makes it easy for me to navigate between my pages, as all of them link to my other pages. If I am on a standard MX page and want to go to one of my customised pages, I select the equivalent standard page, then edit the URL to add the underline and get easily to my page. This naming means I can always use a standard page instead of my customised page when I need to, and I never miss out on any new features.

After an upgrade if you use the standard web template files

If you do not use the example web template files, provided as standard, with a MX release, the remainder of this section can be ignored.

If you use the third party CumulusMX and Cumulus1 UI style Multilingual Websites web pages, as maintained by BCJKiwi, follow any instructions in his package as to which of the files in this section are required and any announcements in his forum topic about when such files need to be upgraded.

web folder from zip

If you do use the example web templates to produce web pages for your web server, and you have copied all files from the release distribution ....

....MX will find any new web template files in your web folder, MX will process those template files, and upload the web pages produced, as before with no further action by you.

webfiles folder from zip


PLEASE COULD SOMEONE USING LATEST MX RELEASE ENSURE THIS IS KEPT UP TO DATE


You do need a further action, if any files in this folder have been updated. The updated files must be uploaded to your web site.

This may be easy to forget if you use the standard web templates, because you thought that upload was a one-off and maybe you cannot remember what you did.

Certain third-party web-pages still use files from that folder, if one of the files they use has been updated, then ensure you have a backup of your web site before you upload the updated file, it should work, but it might not, and you might need to regress.

Let me go through the files currently in webfiles folder:

  • weatherstyle.css has not been updated since its last update by Steve Loft in 2009.
  • subfolder images contains 2 pictures that have stayed same since MX launch
  • subfolder js contains one file or two files (depending on which MX release you are using:
    • The standard web page trends.htm will only work if the correct version of any file here has been uploaded to your web site, and placed in a sub-folder "js" of where "trends.htm" is installed. .
      • BCJKiwi's user interface charts.php also requires the same JavaScript file (or files?), so upload the file(s0 anytime there is a change in a MX release.
    • The file \CumulusMX\webfiles\js\cumuluscharts.js has changed in a lot of releases, it is updated whenever there is a change in the JSON files used for providing data to the charts, and you must use the right upgrade.
    • The file 'historiccharts.js (it was introduced from release 3.8.7) is similar to previous script, but drives the historic chart functionality on your web server
  • subfolder lib currently contains 3 folders:
    • highstock - this folder is no longer needed on your web site (and the most recent MX releases have emptied it)
    • jquery - this folder is needed for "trends,htm" to work, but it has not been updated since 2014. Both the files included are obsolete packages, no longer officially available, and there are no plans to replace them.
    • steelseries - this folder has 3 sub-folders, I can only think of 2 MX releases when anything was changed here, and I don't anticipate any further changes.

After upgrade - checking for bugs in MX or mistakes in your installation

Start the new installation of MX and watch out for any errors - If the device you run MX on has a monitor, then look in the terminal/command window. In all cases look at the latest file in the MXdiags folder to see if any errors are reported.

In newer releases of MX, also see ServiceConsoleLog.txt

Also keep an eye on the support forum for first few days, some releases do have bugs, as developer cannot test all possible configurations.

Updating if you use a virus Checker

You may find that virus checkers such as Windows Defender reject your new version of MX. They need to be told it is safe.