5,838
edits
MX on Linux: Difference between revisions
From Cumulus Wiki
Jump to navigationJump to search
m
Update as so much has changed since the page was first created
m (Update as so much has changed since the page was first created) |
|||
|
If you are running MX on on any computer running the Microsoft Windows Operating System, then you should be reading the [[MX on Windows OS]] page instead.
{{TOCright}}
=Page Content=
This page:
* describes the options available for installing MX, and the other Cumulus packages
* describes the pre-requisite '''MONO''' software needed to run the various Cumulus executables, (for Raspberry Pi only), how to add the Mono repository to your system, and how to upgrade MONO
* explains the few key Linux commands it uses
* describes the administrative interface and instructions for configuring MX
* it tries to be useful to anyone who has never used MX, and anyone who knows Cumulus, but has not run MX on Linux before
* describes the various options available to run MX
* describes the optional parameters you can add when starting MX
* describes the other executables
There are various related pages to get more information:
*Go to [[:Category:Terminology]] for links to pages that explain terminology used by Cumulus (some of these need updating for MX)
*Go to [[:Category:Cumulus MX]] for links to all pages in this Cumulus Wiki that relate specifically to MX
*[[MX Administrative Interface|Admin interface]] provides information on configuration and web pages for viewing your weather data locally
*Go to [[:Category:Cumulus Files]] for links to all pages describing the sub-folders and files used by MX
*If you encounter a problem when running MX, see [[What to do when I have a problem with MX]]
*The [[Cumulus MX FAQ]] page was created with snippets from the forum, but nobody has yet sorted this out into a useful page or updated it for recent releases
*If you were using the original (now legacy) Cumulus software, please read [[Migrating_from_Cumulus_1_to_MX]], although that is mostly directed at those using MX on the same Windows PC as they used for Cumulus 1, and was written for an old MX release, it will help you understand configuration differences.
*If you want to use a script language, you might want to read [[PHP|PHP Hypertext Pre-processor and JavaScript]] page
=Using MX on UNIX-derived Operating Systems=
==Device Coverage==
Linux is available based on a multitude of different kernels (the building block for the operating system), on a multitude of devices.
It is hoped that contributions to this page will be made by Cumulus users with a range of such devices. The page has been originated by a contributor using the Raspberry Pi Operating System (this is based on Debian, one of the Linux kernels). Where appropriate, this page gives instructions specific to a [[Raspberry Pi computer page|Raspberry Pi computer]].
:Until somebody creates a separate page for Apple Mac computers (that would be a good idea), this page is the closest.
*The few people who do have difficulties are those who have good knowledge of Microsoft systems and feel scared to swap to something different.
Microsoft has had a deliberate policy of being different
In the Cumulus support forum, there are many posts from people who are struggling with MX on PCs, as it seems people often find “installing”, and using, MX is more difficult when using Microsoft Windows.
=For those using Raspberry Pi computers=
= Do you have a Operating System? =
New hardware might come preloaded with an operating system, or might allow you to choose which operating system to install on it.
Please see [[Raspberry Pi computer page]] if you want guidance on choosing which model to buy and how to install an operating system on those computers, so you are ready to install MX.
=Do you know how to install packages?=
If you already know how to install packages, your computer has the pre-installed image, or you have installed MX before, you will want to skip the sub-sections that follow on here, and continue reading at [[#Cumulus packages]].
Otherwise, here is a short technical digression to explain the commands.
==The various components to commands for installation==
===sudo===
By default, a Linux user will log in as a default user with limited rights. For example, on the Raspberry Pi Operating System, there is a single default user, initially called "Pi", with their home folder that can be referenced as "~".
The initial "sudo" part of many commands gives us super-user (root) rights when executing the instruction that follows.
If we are working away from the folder(s) owned by the default user, we can consider whether we want to change the ownership of these parts of the file system (how to do this for a MX installation is explained later) so that we can access the files with our default user without using "sudo" for file/folder commands.
In some limited cases, it might make sense to just change "read" rights,so that the default user can read a file/folder, but other actions are not possible without using "sudo" prefix. This limits our ability to accidentally delete files that other processes need.
===apt===
The second part of our installation commands is “apt” meaning “Advance Package Tool”. In simple terms, it runs the “package manager” used in Linux.
If you are looking online for a tutorial on how to install packages, you might see “apt-get” or "apt-cache" used in examples they quote. It should be safe to replace these older package managers with "apt". I have included a little technical explanation at the end of this page if this really worries you.
===install===
|-
|update
|
|-
|upgrade
|
|-
|autoremove
|
|-
|remove
|If you want to remove just one component manually, after "apt", use “remove” followed by the name of component you no longer want
|-
|purge
| To remove any installed package and delete all related configuration files, after "apt" type "purge" then the name of the package we no longer want
|-
|search
The basic syntax is either one or two hyphens, followed by one or two letters (each letter has to be a specific case). Various examples will be seen on this page, but here just one is explained here.
=== Name ===
The final part of the
=Install starts here=
If your computer has the pre-installed image, or you have installed MX before, you can skip the sub-sections that follow on here and continue reading at [[#Cumulus packages]].
== Preparing for an install==
Each of these can be understood from information in previous section.
==Changing the Source List==
The "source list" [[#The various components to commands for installation|mentioned earlier]] may not contain all the repositories we need for our installations.
Consequently, type <code>sudo apt search mono-complete</code>, to find out whether the mono package is available from one of the repositories already in our source list, and if the version available is compatible with MX (Release announcements for MX should specify which versions of Mono will work). Each of the parts of that command was explained earlier.
If mono-complete is not available (or only available in an older version incompatible with MX), then we have to add a new repository, and the one to add depends on which Linux kernel is used by our Operating System.
There are a number of sub-sections below, please check which applies to you, and only read that one.
===Add the Mono repository for a Raspberry Pi===
=Cumulus packages=
Note use of plural in section name above, the following sub-sections will install
==Handling zip files==
Each release is presented as a zip.
The download and unzip procedure is exactly same on your Linux computer, and on a Windows PC. So if you have two devices available, you can download on either device, and if it is not the computer you want to install on, you can use a file transfer package to move the files between devices, or use a drive (or even a memory card) with a partition formatted so that you can read it on both devices. Windows and Linux partitions are formatted in different ways. While it is likely that Linux can read a Microsoft formatted partition, Microsoft Windows can never read a Linux formatted partition.
When your browser saves the zip it might be into a folder called “downloads” on your computer, or you may be able to save into another folder that you prefer (perhaps on a different partition). Your browser might even remember the folder you used before for files of type zip.
When the download has completed, whatever computer type this is on, mouse click (it might need a right click or a double click depending on settings) on the download file and it should unzip (it may create a folder whose name is taken from the zip file name in the same folder by default, or it may ask you where you want to unzip to). If you are unable to use a mouse, there should be a keyboard code to use. If you are using a file manager, with a graphical interface, there may be a different way to select the file and unzip it.
==Where to install all packages?==
For simplicity on this page EXISTING PATH (the contents of this will start with a slash “/”, but not end with a slash) is used to represent any location in the Linux file structure where you decide to install all the Cumulus packages.
The phrase “EXISTING PATH” is used, because it is most likely you want to create the sub-folder called “/CumulusMX” (note where capital letters must be used) in a part of the Linux file structure that already exists.
It is important to minimise the length of the path name, because this path name has to be passed between various different software languages (and longer paths risk truncation).
* You can create sub-folder called “/CumulusMX” as you unzip a MX release, or you can type <code>sudo mkdir EXISTING PATH/CumulusMX</code> first (note that EXISTING PATH is explained above and always starts with a slash “/”).
* By using the phrase EXISTING PATH this advice avoids telling you to install Cumulus where you do not want it:
*# Many people with a Raspberry Pi, and a little technical understanding, add an external drive to reduce wear on the internal micro-SD card, and keep their Cumulus files away from the drive that holds the operating system.
*#* This page is not going to get technical by telling you how to create, or mount, Linux partitions on your external drive. If your drive was bought from a Raspberry Pi reseller, they might help you.
*# Other people using a Raspberry Pi without that technical expertise, might use ‘’’/home/pi’’’ for EXISTING PATH as that is the default folder for the default user (Pi) and can be referenced as "~" in file path instructions they issue (although Cumulus will not understand that shorthand)
*#* Within that ‘’’/home/pi’’’ folder, the default user has full permissions automatically.
*# The developer suggests you use ‘’’/opt’’’ for EXISTING PATH (which should be available on any Linux computer).
*#* By default, the code Mark provides for installing Cumulus as a service, will run that service as a root user, and the root user has full permissions in /opt (and everywhere else)
*#* If you do choose a EXISTING PATH outside your home folder, you might want to change the ownership of the "CumulusMX" sub-folder, to the default user (Pi), if so, type <code>sudo chown -R pi: EXISTING PATH/CumulusMX</code>. The advantages of that command is you no longer need "sudo" to access the files (however, if you are running MX as a service, you also need to edit the user in the script provided to create the service, so MX does not create files with root ownership - this will be covered later)
==Packages to install==
<big>We shall install the Cumulus software listed on [[Software]] page</big>:
# '''CumulusMX’’’:
#* '''CumulusMX.exe’’’ is written in C#
#* Download '''CumulusMX zip file’’’ from [[Software#Latest_build_distribution_download]]
# [[Software#Create_Missing|'''Create Missing''']]:
#* '''CreateMissing.exe''' is another C# package
#* Using '''CreateMissing.exe''' is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki (it will populate missing fields in [[standard log files]] and/or missing lines in [[dayfile.txt]]),
#* Simple Instructions for using this executable are on the github page where they are found, you can find the link for that at [[Software#Create_Missing]] in this Wiki.
# '''ExportToMySQL'''
#* '''ExportToMySQL.exe''' is also written in C#
#* Download '''ExportToMySQL.exe''' package from [[Software#ExportToMySQL]]
#* '''ExportToMySQL.exe''' is not (at the time this was written) documented in this Wiki although [[MX_Administrative_Interface#MySQL_settings]] does describe a similar utility (written by Steve Loft) that was actually included in early MX release downloads.
As at 9 March 2020, there is another utility, '''CreateRecord''', initialised in the Github areas managed by the developer where Cumulus is worked on. This will, if my understanding is correct, read [[dayfile.txt]] and use that to update the various [[:Category:Ini Files|extreme record files]]. However, at the time of writing this, it is nothing more than a concept that needs to be coded, and (as far as I know) there has been no progress on that utility for at least 4 months.
===Alternative download link for older MX releases===
Skip this sub-section if either you have installed the "pre-built disc image", or the current MX release is stable (it has been available for a while and nobody has reported any bugs).
Check if posts in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] tell you that the current release of MX has one or more bug(s) that affects one or more aspect(s) of MX that you intend to use.
Remember, it is impossible for the developer to check all the ways in which versatile MX can be used:
* Different weather station types (the developer only has a Davis),
* Different computer types (development is mostly on Microsoft Windows),
* Plus a whole host of optional features, and different external upload sites, (typically each of these optional features are only used by a sub-set of Cumulus users).
Anyway, '''you can download any earlier build, without the bug''', from [https://github.com/cumulusmx/CumulusMX/releases CumulusMX/releases].
You need to ensure that you use the right version of "CreateMissing.exe" and "ExportToMySQL.exe" utilities for the MX release you are running, so if you are using an old MX release, you will need to go directly to the [https://github.com/cumulusmx Cumulus MX github] page, and navigate to the utility of interest, to download an older version of these utilities that matches your older MX.
==Upgrading a Cumulus package==
Upgrading to a new MX release is explained [[Updating_MX_to_new_version|here]], but basically follow instructions above, and install over your existing files. The alternative is to install in a new folder (or rename the old one), and copy across files not in the release from old location to new location.
== Report and data files to copy across from any previous Cumulus location ==
All the (optional) files in the [[Reports folder]] can be copied across from a previous installation. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
All the files in the [[data folder]] can also be copied across. If your previous installation was not on Linux, see [[#Line terminators in .txt files]]
If your previous Cumulus installation was version 1.9.4, or earlier, then you need to do a lot of reading:
* [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]]
* [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files
* [[Migrating from Cumulus 1 to MX]] gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date
==Configuration Files to copy across from any previous Cumulus installation==
'''This is an optional file'''. Its [[strings.ini|purpose]] is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language.
You create a “strings.ini” file by selecting some of the parameters from the [[samplestrings.ini]] file that is included in each MX release, and modifying the value for the listed attributes
The sections that appear in '''samplestrings.ini''', and the parameters that appear within a section, changed drastically between Cumulus 1.9.4 and MX.
The content of "samplestrings.ini" is changing as MX is developed. Therefore, your existing “strings.ini” might need to be modified. There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it. Instead, you need to manually check each parameter you have in your “strings.ini” file to see if that parameter is in “samplestring.ini” included in the release you have installed. You may also find new parameters in “samplestring.ini” that you wish to add to your “strings.ini” file to tailor new functionality to your preferences.
===Cumulus.ini===
If your old release was 3.8.0 or later, then copy the [[Cumulus.ini]] file used in that old installation into your new one. The file is growing with lots of new parameters, and the tables on the page I have just linked indicated when parameters were introduced, and where those settings may be changed to make it simple for you to ensure all settings are right for your new release.
If your old release was older that 3.8.0, then you will see [[:Category:Configuration Files|the Configuration Files page]] shows different links for documentation on older "cumulus.ini" files. Because of those differences, many people prefer to rename their existing “Cumulus.ini” file when upgrading from such an old release, or making an install on a new device. To create a new file, they work through all the settings manually, so that MX can create a fresh file, with them having confidence every setting reflects their preference. However, you do need to be aware that such older releases included a lot of "read-only" parameters that had to be set manually in the file, you may find some of those parameters are still needed by you, so you will want to edit the new file, with MX stopped, and copy any of those old "read-only" parameters that might not yet be settings you configure within the MX interface (although the number of these is diminishing with each new release).
If you previously used an older release of Cumulus, but in this new installation will be using the latest release (latest is what is normally best, unless it has bugs), you may want to read up on all the changes between your old release and the current release, not just changes that affect the configuration file.
==Moving from Microsoft Windows to Linux==
===Line terminators in .txt files===
If you are a novice to computers, skip this sub-section and the next, go directly to [[#File Names & Paths]].
You need to have some technical understanding to do an action that might encounter the issues discussed here. For normal MX usage, line terminators do not matter.
If you are moving from Microsoft Windows to Linux,
If you run your Linux computer in a headless mode, accessing its files by a remote terminal session, be aware that the line terminator used by the remote computer may be applied to files affected by whatever command you do remotely. Equally, running a Cumulus executable (MX or one of the utilities) may create new files with the wrong end of line terminator. The latest releases have been amended to match existing files, when they create a new file to replace an old file (or because that file is missing/corrupted).
If you try to edit a file outside MX, the tool you choose to use might be able to detect that it is getting Microsoft files, and discard the extra end of line character.
However, if you are reading a comma separated value file (such as MX uses for various files) in a script, your script might not detect the end of line encoding:
* If the script expects the Linus LF, and finds CR LF, then the final field of each CSV line has an invalid character in it, so your script will not be able to understand that field as a numerical value (or time-stamp).
* If the script expects CR only, and finds CR LF, then the first field of the each line presented to the script (except the first line) starts with a “LF” and the script will not recognise it as a date.
* If the script expects CR only, but just finds LF, the script will believe the whole file is just one line, and the fields before and after the LF will be treated as a single field so your script will not be able to understand that field as a numerical value (or time-stamp).
* If the script was written for a Microsoft Windows environment, it will expect CR LF, but might be confused if the end of line is different
===Changing line terminators===
== Optional parameters to add to the instruction to run the MX engine ==
This list of parameters covers every parameter, if you are using the latest release skip the historic 3.0.0 sub-sections.
IMPORTANT: The "sudo" prefix shown in code examples for the various parameters can be omitted if the user that is running MX owns the folder "CumulusMX" and all its contents. In the [[#Where to install all packages?|installation notes earlier]] possible locations and ownership issues were mentioned. That earlier section also defined EXISTING PATH, if that is not mentioned in the code examples, it is assumed you have already issued a <code>cd EXISTING PATH</code> to be in the right folder for issuing the command.
Note that you ''may'' need to supply your root password after typing any 'sudo ...' command line if the instruction changes something set by another package (e.g. changing the locale for MX from the locale set by mono). The system will prompt you for this if it is needed.
=== Version 3.0.0 Beta builds of MX ===
The following two parameters cannot be used with MX since it came out of 3.0.0 beta.
==== web sockets ====
Beta builds in MX version 3.0.0 had an optional parameter <code>sudo mono EXISTING PATH/CumulusMX/CumulusMX.exe -wsport nnnn</code> that determined which port (represented by a 4 digit number ''nnnn'') was used for '''Web Sockets'''.
That parameter is now deprecated as Web Sockets in all builds since 3045 uses the same port as the rest of the [[MX_Administrative_Interface#The_API_interface|Admin Interface]], see Port parameter below.
When Cumulus starts, it will display the URL of the user interface. It runs on port 8998 by default; if this is not suitable for some reason you can over-ride it using the '-port' parameter on the command line, e.g. to use port 9999 instead:
<pre>sudo mono CumulusMX.exe -port 9999</pre>
If there is a problem, then there is a great benefit in actually increasing the level of detail in these logs; and that is done either within the settings (on recent MX releases this is on '''Program Settings''' page of admin interface - please see [[MXdiags_folder]] page for details) or by adding a parameter:
:<code>sudo mono CumulusMX.exe -debug</code>
On Linux and (in particular) OS X, Cumulus MX may not be given the correct locale to use, and you may get the default US locale even if that is not your locale. It will output the local it is using when it starts; if it is not correct, close it down and start it again, this time specifying your locale on the command line, using the -lang parameter . For example, in the UK, on a non-Windows device type:
:<code>sudo mono CumulusMX.exe -lang en-GB</code>
Other locale examples: '''CumulusMX.exe Current culture: English (United States)''', '''CumulusMX.exe -lang de-DE''', '''CumulusMX.exe -lang el-GR''' (this is one of the locales that reads numbers with '''integer,decimal''' format), '''CumulusMX.exe -lang nl-NL'''.
If you are not sure what value you need to supply for the -lang parameter, there is a list here - http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. You need to supply the code in the first column ("Language Culture Name") in that list.
Note that this does not affect the language used by Cumulus MX (although it may in the future), it affects the decimal separator and the list separator.
=== Parameter for running as service ===
You don't use this parameter in a terminal or interactive instruction for running MX. Instead it appears within a file that we use to set up for running MX as a service.
For the latest release, (see [https://cumulus.hosiene.co.uk/viewtopic.php?p=146473#p146473 this later release announcement]:
#There is a task to do just once to configure the service
#Find the '''EXISTING PATH/CumulusMX/MXutils/linux/''' sub-folder, that might be in home directory and therefore found using "~/CumulusMX/MXutils/linux" as explained elsewhere on this page
#* At time of typing this, the sub-folder only contains one file, the one we need to edit
# As described later there is a choice of editors, but you can use <code>sudo nano cumulusmx.service</code> to edit the service configuration file
# Within the provided file you should find a [Service] section:
<pre>[Service]
User=root
Group=root
ExecStart=/usr/bin/mono-service -d:/home/install/CumulusMX CumulusMX.exe -service
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock</pre>
There are some edits needed to that section:
# Replace '''User=root''' if you want the service to run as a different user
# Edit the line that begins with '''ExecStart='''
#* The mandatory change is to replace '''/home/install''' by what you have selected for EXISTING PATH
#* An optional change is to add additional parameters after the '''-service''' (select from '''-debug''', -locale, -port) as described in sub-sections above
#save file with a new name '''cumulusmx(new).service''' (using a new name stops it being overwritten when we upgrade MX
#now copy file
:<code>sudo cp "EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx(new).service" /etc/systemd/system/cumulusms.service</code>
Here is how the file might look after the changes:
<pre>[Unit]
Description=CumulusMX service
Documentation=https://cumuluswiki.org/a/Main_Page
After=network-online.target
[Service]
User=root
Group=root
ExecStart=/usr/bin/mono-service -d:/opt/CumulusMX CumulusMX.exe -service -debug
Type=forking
ExecStopPost=/bin/rm /tmp/CumulusMX.exe.lock
[Install]
WantedBy=multi-user.target</pre>
== Setting up as a service release 3.8.0 ==
Skip this sub-section for latest release.
For historic reasons, note the original instructions in the 3.8.0 [https://cumulus.hosiene.co.uk/viewtopic.php?p=145048#p145048 release announcement]:
# Ensure you are in the folder containing CumulusMX.exe
# Type <code> mono-service -l:/var/run/cmx.pid CumulusMX.exe -service</code>
# (to verify) note this does not allow you to add -port, -debug, -locale parameters
==Running as a service==
The full set of commands to use with this service are at [[Raspberry_Pi_Image#systemctl_commands|systemctl_commands]],they are not duplicated here, so there is only one place to do any update.
In the [[#Parameter for running as service|earlier instructions]] it was suggested you change the user for the service, if you have done that you can omit the "sudo" in <code>sudo systemctl start cumulusmx</code> which manually restarts the service. Equally, you can omit "sudo" from any command that accesses Cumulus files created by the service.
== Running any MX executable with a terminal session left open ==
This is alternative to the running as service as described above.
The simplest instruction to run Cumulus MX is <code>cd EXISTING PATH/CumulusMX && sudo mono CumulusMX.exe</code>.
* This is two commands issued together, the first changes the working folder, the second actually starts the main executable
** To run "Create Missing" or '''ExportToMySQL''', just edit what appears after "mono" to the correct executable name
* If you have followed advice at [[#Where to install all packages?]], the user you are using will own the "EXISTING PATH/CumulusMX" folder and you can omit the "sudo".
* You can add parameters after "CumulusMX.exe" (select from '''-debug''', -locale, -port) as described in sub-sections above
You can start it off directly on your Pi, and then
The styling and library files were a one-off upload from '''CumulusMX/webfiles'''. These release use [[Cumulus_template_file|template files]], these are [[Customised_templates#What_is_meant_by_.27Cumulus_processes_templates.27|processed by MX to add the variable data]], and this will create web pages that are uploaded to your web site.
Please read [[Customised_templates]] for further information about the various pages originally provided, and how you
=== Comparison with legacy Cumulus 1 web pages ===
* Note that
** so if moving from Cumulus 1 to MX, delete all your Cumulus 1 files from the "web" and "webfiles" sub-folders, and all files from your web server; then upload files from the new "webfiles" folder.
* The standard gauges are
** The new default gauges page does not display a graph when you hover over a gauge (Cumulus 1 generated some images that the stand alone Steel Series gauges page could use)
** (The Trends page provided as standard in Cumulus 1 simply displayed images of graphs uploaded to the web server)
== Alternative ways to obtain web pages ==
| |||
