Cumulus Wiki

MX on Windows OS

Revision as of 09:55, 11 April 2021 by Sfws (talk | contribs) (→‎ExportMySQL.exe: rewrite as no longer part of release distribution)

Introduction

This page is a simple summary for installing and running (non-beta) releases of MX on any computer running the Windows Operating System.


  This document is 'Work In Progress' so content may not be complete or accurate!

Download MX release

  1. To find the link to latest release distribution zip in the Cumulus Wiki, open the Software article in the Current_Release section.
  2. Download the MX distribution from the link that appears there, Mark will update it for each release he makes.
    • It is up to you, where you store the download, but it is a good idea to store it on a separate disc, or partition, if you have a choice.
    • Whether this location is the default, or you are asked to select location will depend on whether your browser's default settings have been changed.
  3. When download completes, use the mouse to click on the download file name, this should ask if you want to extract (unzip) it.
    • Decide where you want the full distribution to be stored, again you might have somewhere where you backup files.

'If the latest release has bugs (it is impossible for the developer to check all the ways in which versatile MX can be used), you can download, whatever older version of MX you have decided is sufficiently bug-free to install, from CumulusMX/releases.

Putting distribution where you want to run MX

The only rule about where you put the MX files is "avoid the 'Program File' hierachy", instead choose somewhere where you put files you edit, or place in the root directory of a disc or partition.

If you have previously run any release of Cumulus (perhaps the legacy Cumulus 1), then it is easiest if you overwrite that existing distribution.

Are you going to use the optional feature in MX to upload to a web server?

As this is an optional feature, the tasks here are optional.

To set up your web server for the first time

When you first want to use Cumulus web pages on your web server, you need a number of static (unchanging) files to be put onto your web server. The web pages that MX uploads for you reference that static files and will not look right without them. The files that only have to be uploaded once are found in \CumulusMX\webfiles and its sub-folders. You don't create a folder called webfiles on your web server, but you put the files and sub-folders into position relative to where MX will upload the htm files.

To do this, you will must invoke a FTP process outside of Cumulus, MX does not include any functionality to do this one-off upload for you. The filezilla client is a popular choice as it has probably the most friendly graphical user interface, although other software to do this is also available. You may prefer a tool that lets you do the uploads from a command line without requiring working with a graphical interface.

  1. The static files to be uploaded include the standard styling file \CumulusMX\webfiles\weatherstyle.css which you place in the directory specified for the uploads.
  2. Next you have three sub-folders, each of those sub-folders need to be replicated within the directory specified for the uploads.
    • For example \CumulusMX\webfiles\images\picture.jpg will be stored in a "images" sub-directory of the upload directory and is used as the background image for web pages.
      • There is nothing to stop you creating your own "picture.jpg" (instead of uploading the supplied one) and then Cumulus web pages will use that for the background image on each page.
    • Similarly \CumulusMX\webfiles\js\cumuluscharts.js needs to be stored in a "js" sub-directory of your upload directory (this is the script that allows you to change the chart shown on the trends page and uses the appropriate json file to populate it with data).
  3. The "lib" sub-folder contains further levels of sub-folders all to be replicated on your web site.
  4. The trends.htm web page also loads some library software from an internet Content Delivery Network (cdn) to invoke the JavaScript based Highstocks library.

Operating a web site with uploads from MX engine

The standard web pages

  • If you want to operate the 'standard' web site, then just the same as with Cumulus 1, you will need to upload the contents of the webfiles folder from the zip file (don't upload the containing webfiles folder itself).
    • Note that the MX web files are not the same as the ones for Cumulus 1, so make sure you upload the MX files if moving from Cumulus 1 to MX.
    • The standard gauges are now the SteelSeries gauges. The default versions do not display a graph when you hover over a gauge as happened when you added the stand-alone Steel Series gauges to Cumulus 1.
    • The trends web page in Cumulus 1 relied on that software generating graphs as images. In MX, the software generates files with time and value pairs, these are stored in json format, the trends page then uses a library package (Highstocks) to draw graphs from those data pairs.

Alternative ways to obtain web pages

You can choose to use some of the alternative web pages available from third parties and described on User Contributions page.

Using your own web pages

  • Of course you can use your own web pages, instead of the standard ones. Assuming they need to include figures that are available as web tags, there are three alternative ways to implement this:
    1. MX can process template files with a HTML structure and those web tags in the structure where values are required just as it does with the standard templates, and MX can upload the resulting web pages at either the real-time interval, the standard interval, or after end of day. All of this is covered on the Customised templates page in this Wiki.
    2. MX can process a file with a string of web tags mirroring the realtime.txt option in MX, and upload the resulting file so your web pages can use JavaScript for a one-off insert of the values or an Ajax routine to update the web page at a fixed interval.
    3. Alternatively, you can use template scripts processed locally by MX that don't create web pages, but are uploaded by MX at either the real-time interval, the standard interval, or after end of day. These scripts simply initialise script variables with values obtained from web tags. You then independently have a set of web pages resident only on your web server (they don't exist where you run MX) using a combination of HTML and script content that bring in the script(s) with the variables by the appropriate syntax. All of this is covered on the PHP web tags page in this wiki. As it suggests there, you might therefore have several files processed by Cumulus MX at these different intervals, converting the web tags into script variables, and then use AJAX (JavaScript that may use json format to bring in the variables) or PHP (using 'require_once 'filename'; syntax) to put those variables into a web page.

You may find this wiki page useful for understanding more about the different script languages.

Requirements for running MX on Windows

To run MX on Windows, you need .Net version of at least 4.5.2 installed. This is only available for Vista SP2, Windows 7 SP1, Windows 8, Windows 8.1.

For Windows 10 you use version 5, which should already be installed by your windows update feature. MX will also run on version 4.8; the .Net download for version 4.8 should be here https://dotnet.microsoft.com/download/dotnet-framework/net48.


.NET

Microsoft originally created .NET for all operating systems, but Microsoft then decided to restrict it to just Windows (and that was the position while MX was being developed, and was still true in early 2020), mostly to encourage greater dominance by Microsoft software and hardware.

Mono was then born based on .NET to work with all operating systems, Mono subsequently changed independently from .NET (although Microsoft still has a leading role).

Later, Microsoft launched an alternative called .NET Core that took out of .NET the parts that were Windows specific, and it ceased work on further development of .NET beyond version 4.x.x.

The most recent change, in November 2020, involved a change around of names, and the multi-operating system .NET Core product took over the .NET name as version 5

Are you using elevated administrative rights?

For those who downloaded the first MX Beta in January 2015, the code was only experimental and that version had to be run by a Windows Administrative User, but Steve Loft soon improved the code and now none of the code requires any elevated rights and it can be run by a normal user (or a user with administrative rights) without needing to be started by Run as administrator.

However, Cumulus MX initiates a web server, which is what runs the Admin Interface. To access that, all users need to be given elevated rights to the port on which the web server runs. By default this is port 8998, so that is used in the example below of the one-off command needed to give all users access to the port. You can use a -port=nnnn parameter when starting MX to make it use another port, if you use that then the command below needs revising accordingly.

To enter the command, first open a command window as administrator. One way to do this is to right click the windows symbol at the start of the windows task bar. The option to choose there is dependent on some settings which determine what appears when you right click:

  • the normal default on Windows 10 is Windows PowerShell (admin),
  • the normal default on earlier versions of Windows is Command Prompt (Administrator)
  • an alternative is Windows Terminal

Whichever of these you can use, the result is it opens a new window on your monitor with a prompt for typing. In that window type the command: 

netsh http add urlacl url=http://*:8998/ user=\users

You only need to do that once. If you do not issue this command, administrative rights are needed every time to access the port so you can use the admin interface.

Talking about command windows, if you want to check that the port is open for listening (i.e. able to access the admin interface) type netstat -an | findstr 8998 into the command window.

The admin interface URL http://*:8998/ needs to have that wildcard "*" replaced by a precise location if we are to access the admin interface. The missing part of the URL depends on how your local network is set up. If you are accessing the admin interface on the same device as that running MX (and you don't have another web server on that device) the "*" can be replaced by "localhost", i.e. http://localhost:8998/ will be used to load the admin interface into your browser. In the more general case when you want to access the admin interface from anywhere on your local wired and wireless interface, then the "*" needs to be replaced by a string of 4 numbers representing what is called a IPv4 address (w.x.y.z) of the device you have installed MX on.

Look at your hub or router (this should have come with instructions on how to access its settings in your browser) and on one screen it should show what devices are connected to your LAN and wifi. Look for the IPv4 address, for example 192.168.1.64, it has assigned to the device where MX is running. That is what should replace the "*". However, there is one more complication, either the Windows networking settings may change, or else your hub or router may reconfigure, both can happen at any time and both can assign a different IPv4 address to the device running MX.

To give your Computer a fixed address for the MX admin interface,

  1. first find the network card via Network and Sharing Centre (Control Panel),
  2. then click on Change Adapter Settings,
  3. then Right click on Ethernet or WiFi Adapter,
  4. next select Properties
  5. In the window that opens, right click on Internet Protocol Version 4 (TCP/IP 4),
  6. Next select properties
  7. Onn that pop up screen tell the computer to "use the following IP address"
  8. Fill out the form with
    • a subnet mask of 255.255.255.0 and
    • gateway address between 192.168.1.1 and 192.168.1.254 (depending on the address of your hub/router).


Running MX as a service

  This section has not yet been written!


Setting up for either manual or automatic running (without service)

To run Cumulus MX, Windows needs to know

  1. which .exe you want to run (ExportMySQL.exe or CumulusMX.exe)
  2. the path where all the required .dll files are located

Therefore it is best to always start MX using what Windows calls a shortcut, because when creating the shortcut you can enter all the required information into the properties. If you want MX to automatically start whenever you log into your PC, then the place to store your shortcut is C:\Users\...\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\Run_CumulusMX. Don't forget to put your Microsoft username where I have put ...

With this you might want to right click on that shortcut, select properties, then you can set the starting position for the command window, the colours and font it will use, and even choose to start minimised, amongst many other selections.

There are 3 ways on Windows to create a shortcut to run MX, as mentioned above you can't just click on the executable in file manager, because Windows needs to be told the path for loading all the related .dll files.


  1. Create a shortcut on your desktop (and/or the taskbar) for the CumulusMX.exe executable cmd.exe /C start CumulusMX C:\CumulusMX\CumulusMX.exe -debug, the "-debug" is optional, it starts the logging in debugging mode so the log created in MXDiags folder has more information. There are other optional parameters all listed later.
    • In that shortcut define the path where the executable is located as the path to start in.
    • Remember, if you have not done the netsh http add urlacl url=http://*:8998/ user=\users command mentioned above, you must run as administrator.
    • Add any of the parameters that can be used with the executable, as listed later, such as specifying a different port for the admin interface , or starting with debugging.
    • Choose whether to start as minimised or as a command window
    • You can also choose text colour (foreground) and background colour, you can choose where the window appears (so if you have two monitors you can choose which one it appears on), and how big that window is. There is a forum post by water01 about this.
    • Now you can click the shortcut to start the engine
  2. OR place the shortcut as defined above in the start up folder for the user account so MX automatically starts when you connect/log in (see a later section for how to find that start up folder).
  3. OR declare a task in the scheduler to start MX; in the Actions tab fill in fields as follows (the other tabs should be obvious):
      • Action Start a program from drop-down
      • Program/script cmd.exe (this is standard Windows environment to run something)
      • Add arguments /C start Start_MX \CumulusMX\CumulusMX.exe -debug -port=nnnn (the "/C" means this task will close once it has started the task, the "Start_MX" is how the task will be labelled as it is running, the next argument "\CumulusMX\CumulusMX.exe" actually starts the executable and it does not need a drive prefix as that is in next box.
      • Note in this example I have included next two optional parameters that can be used after the .exe call in that same box, here -debug (only include if you want full debugging logging) and -port=nnnn where nnnn is the port to be used for admin interface (only include if want to change from default 8998),
      • all optional parameters are listed later
      • Start in \CumulusMX (include a drive specifier if necessary)

Each time you want to run Cumulus MX on Windows:

  1. First start the engine in one of the 3 ways from last sub-section
  2. Next start the admin interface, it does not need to run all the time, but only when you need it (when you first use MX you will need it to access the settings where you tell MX what type of station you have and what units you want to use, and set various timing options), it normally runs on port 8998 (to vary that there is a -port parameter that is followed by required port and that port parameter has to be entered every time you start MX if you are not using the default port). More information on admin interface in separate article.

Try start /min C:\Cumulus\CumulusMX.exe to run MX as a minimised package (although in Windows you can change the properties of the shortcut you use to start minimised).

Stopping Cumulus MX on Windows pc

The recommended way is to click into the command window in which MX is running, hold down Control key and press C. It is normal for there to be a short wait, then a message "Cumulus Terminating" and then after another short wait, it will say "Cumulus Stopped" and immediately after that the command window will close.

Some people, click in the task bar and select close, or click the X button on top right of command window. Although these are not official advice, they do seem to work.

There are packages that can be programmed to send a control C to a running task, and to not continue until the task window has closed. Remember to also program in a subsequent delay in that package, to make sure the package waits for MX to close, or do a check that MX has released all the files it might need to update.

You should not issue a TASKKILL instruction, as that will prevent MX correctly writing out to all the files it should update on exit. Consequently, it will not restart correctly and may actually lose settings and data.



Executables

The MX package, at time of typing this, includes two executables:

CumulusMX.exe

Whilst effectively MX is run by a CumulusMX.exe, you actually need to ensure all the other components are loaded, so you either have a package that runs it for you, or you click a shortcut that includes the necessary path setting.

Optional parameters to add to the instruction to run the MX engine

Beta builds in MX version 3.0.0 had an optional parameter -wsport nnnn that determined which port (represented by a 4 digit number nnnn) was used for WebSockets. That parameter is now deprecated as WebSockets in all builds since 3045 uses the same port as the rest of the Admin Interface. The remaining parameters that are still available are described in subsequent sub-sections.

Parameter for changing Port

When Cumulus starts, it will display the URL of the user interface. It runs on port 8998 by default; if this is not suitable for some reason you can over-ride it using the '-port' parameter on the command line, e.g. to use port 9999 instead: 

CumulusMX.exe -port 9999

Parameter for adding debugging

MX has a default level of logging that stores in the MXDiags_folder folder a log file that shows some of the interaction with the weather station and some of the output actions done as MX runs. A new log is started each time MX is restarted.

If there is a problem, then there is a great benefit in actually increasing the level of detail in these logs; and that is done either within the settings (options section of station settings) in admin interface while MX is running, or by adding 1 or 2 parameters when you start MX. Obviously this log file continues to grow, the longer MX is left running, and if debugging is switched on the file will grow in size must faster. Consequently, the default is not to add the extra debugging information and the settings can be used to switch it off again if you do have it switched on. Whether you start it with a parameter or enable it within settings, stopping MX will end the extra debugging, and on restart it will default back to no debugging unless turned on again with parameter or setting.

You can also add CumulusMX.exe -debug (to have full debugging of actions by MX turned on as MX starts), and/or CumulusMX.exe -Logging=1 (for the station to MX transfers to have increased debugging logging). 

CumulusMX.exe -debug -Logging=1

Since this parameter is applied when you start MX, it applies while MX continues to run. Obviously, it must be applied every time you start MX if you want this increased level of logging to continue every time you restart MX.

The comments in the MX source suggests -debug turns on both debug and data logging (see Station_Settings#Options in admin interface settings), but I believe that is wrong as per example above, there are 2 separate parameters.

ExportMySQL.exe

The package described here has been replaced by ExportToMySQL.exe, which is an improved version with slight differences in parameters and functionality. The new package is documented at Export To My SQL


This second exe file (see link in heading) was available in the original MX beta package that Steve Loft developed in April 2015, it remained part of the standard release distribution until March 2021. As noted above, it is no longer part of the MX release distribution and has been replaced by Mark Crossley with a separate package with a slightly more meaningful name.

Sadly few people even notice ExportMySQL.exe exists, and if they do, it is unlikely they know how to use it. Hopefully, some people will read this section and find out!

Obviously, the original executable was updated when Mark Crossley added the Feels Like fields to log files.

Put simply, this executable will read log files and insert (insert ignore) rows into an existing database table. Since it only does inserts, despite the name of this function, it is not just for MySQL tables, the included SQL should work with whatever database table type you have.

The executable has a mandatory single parameter that tells it which log files to read, there are only 3 possible parameters ("dayfile", "monthly", or path to a file). It needs to know what locale (or culture settings) it is to use to work out what character separates each item in the log file list. It also needs to read your Cumulus.ini file, as it takes these "input parameters" from MySQL section in that:

  • Host
  • Port
  • User
  • Pass
  • Database
  • MonthlyTable
  • DayfileTable

Daily summary log file

  1. Use the feature in the admin interface:
    • Settings menu
    • MySQL settings page
    • In Dayfile.txt upload section, give your database table a name, or accept default Dayfile.
    • Click Save to ensure this setting is updated
  2. Now scroll down to Create database table (save settings first)
    • Here click Create Dayfile
  3. Now you have a database table ready, you can use the executable to read all lines in your CumulusMX/data/dayfile.txt log file.
  4. Open a terminal display (if you are using Windows then, open a Command Window, a Windows Powershell window, or a Windows Terminal window)
  5. Run this executable in that terminal display (or command window) by using sudo mono ExportMySql.exe daily or ExportMySql.exe daily depending on device.
  6. In the terminal display (or command window) you will see Parameter = daily confirming what you entered and in the line below that a rapidly updating code that is the primary key displayed for each row it tries to insert into the table. If that primary key already exists in the table, it will still show the key, but no insert will take place.
  7. If you want MX to continue adding new rows to this database table, still in the admin interface, still in MySQL settings page:
    • Return to Dayfile.txt upload section, and select Enable.

Standard Log files

  1. Use the feature in the admin interface:
    • Settings menu
    • MySQL settings page
    • In Monthly log file upload section, give your database table a name, or accept default Monthly.
    • Click Save to ensure this setting is updated
  2. Now scroll down to Create database table (save settings first)
    • Here click Create Monthly
  3. If you want MX to continue adding new rows to this database table, still in the admin interface, still in MySQL settings page:
    • In the same Monthly log file upload section, now select Enable.
  4. Now you have a database table ready, you can use the executable to read all lines in either one (if path to that file is in parameter), or every (if parameter is monthly) standard log file.
    • If the parameter is "monthly" it will look in folder data for every file it can find with a file name of datestring + "log.txt" where datestring is a 3 letter code (in your locale) for each month (1 to 12) followed by a 2 digit year (from "00" to "99") so that is how it finds every standard log file in the folder.
  5. Open a terminal display (if you are using Windows then, open a Command Window, a Windows Powershell window, or a Windows Terminal window)
  6. Run this executable in that terminal display (or command window) by using sudo mono ExportMySql.exe monthly or ExportMySql.exe monthly depending on device.
    • Alternatively, replace monthly parameter by a full path to a single standard log file, and it will process just that log file.
  7. In the terminal display (or command window) you will see Parameter = monthly confirming what you entered and in the line below that a rapidly updating code that is the primary key (omitting the first two digits of the year) displayed for each row it tries to insert into the table. If that primary key already exists in the table, it will still show the key, but no insert will take place. So you can run this again to pick up any additions to the latest log file since the original run. Also notice that if you use the parameter "monthly" the order in which it will process different standard log files is not predicable, they probably will not be in any particular order, but as one feature of SQL databases is that the row order is not able to be determined, it does not matter if rows are not added in chronological order.
    • It is worth noting that it is safe to run this procedure while MX is also running, because this procedure only updates log entries that exist as this procedure reads the logs, and MX only adds new entries to the log and at the same time uploads that new entry (if enabled) to the database table.

Please be aware that the transfer to the database table adds two columns where bearings in the original log file given in degrees are output as compass directions, and these use up to 3 letters of how the compass directions are defined in the strings.ini file. Thus the number of columns in the database table will be at least 2 more than the number of fields in the log files. It is also important to stress that whilst the database table must contain one column defined for each field (plus the extra 2) being uploaded, you can add even more columns to your table if you want and populate those some other way. For example, I have added a Canadian Humidity Index (Humidex) column which is not in the standard logs, but is calculated by Cumulus, and can be calculated from columns that are uploaded from the standard log. Humidex is not uploaded by either ExportMySQL or the normal CumulusMX process, but neither objects to extra columns being there.

When testing this, I had some log files produced by various old versions of Cumulus 1 in my MX data folder as well as the log files in has generated since I swapped to MX. Plus I had used a PHP script to add feels like to those log files produced before version 3.6.0 and to correct feels like for those log entries made by versions 3.6.0 to 3.6.9 inclusive because they used a different formula to the one being used from version 3.6.10. This php script is a web page with a HTML form and can be obtained from the forum in Create Missing for MX

I notice that the database rows produced by those short log file lines produced by say version 1.9.0 had nulls entered for all subsequent columns, except Feels Like and this column was initialised at 0.0!

For those log files produced by the final version 1.9.4, all columns are populated although feels like is set to 0.0.

Retrieved from "https://www.cumuluswiki.org/index.php?title=MX_on_Windows_OS&oldid=8430"