MX on Linux: Difference between revisions
(306 intermediate revisions by 7 users not shown) | |||
Line 1: | Line 1: | ||
=Using MX on UNIX-derived Operating Systems= |
=Using MX on UNIX-derived Operating Systems= |
||
[[Category:Cumulus MX]] |
|||
MX runs on any UNIX-derived operating systems (OS): |
|||
* including those found on Apple Mac computers, |
|||
* and those found on a multitude of devices running Linux. |
|||
UNIX is a long established operating system, and both UNIX and its derivatives have good long term compatibility. This means that commands are generally easy to learn just once and then you can normally continue to use what you have learnt. |
|||
MX runs on various UNIX-derived operating systems (OS), including those found on Apple Mac computers, and those found on devices running Linux. |
|||
Most devices also have a graphical user interface that can do the more straightforward tasks without needing to know all the commands. |
|||
==Introduction== |
|||
This article focuses on a device called the Raspberry Pi. It is a small board of electronics that can actually run Windows, Mac OS, Chrome OS, various Linux distributions, or the Raspbian OS (based on Ubantu Linux). In this article, the focus is on the last OS, because that is easiest to install, so look elsewhere on the web for details of installing alternatives. |
|||
==Why install MX on Linux?== |
|||
The article will give you some guidance on: |
|||
*Choosing a Pi model to buy |
|||
*Setting it up, |
|||
** either connecting it to a keyboard, mouse, and monitor (could be TV), and internet (wired Ethernet or wireless) |
|||
** or running it headless, where all instructions are typed into another device and the Pi has no keyboard, mouse, or monitor, but is connected to internet (wired Ethernet or wireless) |
|||
*Installing OS |
|||
*Installing Mono |
|||
*Installing Apache, PHP, MariaDB, PhpMyAdmin, and copying your database from another device |
|||
*Running MX |
|||
Contributions to the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] suggest that: |
|||
=Which Raspberry Pi to buy= |
|||
*Use on a Raspberry Pi (RPi) computer is very popular |
|||
*In general, people find installing, and running, MX on Linux is easy |
|||
*The few people who do have difficulties are those who have good knowledge of Microsoft systems and therefore are so convinced they cannot cope with a swap to something different, that they give up too easily! |
|||
Microsoft has had a deliberate policy of being different to traditional computers (all others are mostly based on UNIX). |
|||
==First make a list of what you need== |
|||
You may know that this Wiki started with a single page covering MX regardless on which operating system was used, that did not work. |
|||
*Do you want to use a mouse and key board? |
|||
** If so, a model with multiple USB sockets is advisable (like 3B+) |
|||
** This also applies if you want to be able to plug in a USB stick (perhaps for transferring files between devices, e.g. Cumulus configuration and data folder files) |
|||
*Do you want a wired connection to your hub or router? |
|||
** Maybe you are going to update external sites, a wired connection may provide a faster and more dependable communication than a wireless link |
|||
** If so, a model including an Ethernet socket is advisable (like 0W or 2B+) |
|||
** Remember that if you are operating the pi in headless mode, a wired or wireless connection to your LAN is needed for your other device to communicate with the Pi |
|||
*Will your MX need to update a database, feed data to a web site, upload to external sites, or control other devices? |
|||
** If so, a model Zero will have to do each task in turn, and you will see some delay in information updates, plus you will need to use a larger time between updates |
|||
** If so, a model Three (or Four) will be noticeably faster, and permit you to select more options in MX, and to update external sites more frequently |
|||
* What interface does your weather station use? |
|||
** If your station communicates to MX via wireless, then you choose a model that supports wireless at the right frequency |
|||
** If your station communicates via Ethernet, then either a wired or wireless connection is possible to Pi as the station will be plugged into your hub or router |
|||
** If your station communicates to MX via USB, then choose between the model 0W with one USB socket, and the model 3B+ with 4 USB sockets |
|||
** If your station uses another communication port (such as serial interface), then you need the components that support that interface |
|||
If you are running MX on on any computer running the Microsoft Windows Operating System, then you should be reading the [[MX on Windows OS]] page instead. In the Cumulus support forum, there are many posts from people who are struggling with MX on PCs. It appears this is not because Microsoft computers are so more readily available and therefore known about; but because people often find “installing”, and using, MX is more difficult when using the more complex Microsoft Windows operating system, and people tend not to understand basic issues such as avoiding "Program Files". |
|||
== Now research how the various models relate to your needs == |
|||
You can look up online what features are included in the various Pi models, and how they differ in power consumption, and cost. But your decision also needs to consider what you need. Here, I won't describe all the different models, but concentrate on just 2 for simplicity. The model Zero W is appealing as it has low power consumption, it is perfectly adequate for running MX (but has limited speed, it runs the various threads MX uses sequentially) especially if you only use standard MX functionality and don't ask MX to do all the optional extras, and has limited interfaces for peripherals. The model Three B plus is appealing as it has medium power consumption, but can cope better with the multiple threads that MX starts, and has more interfaces built in, such as 4 USB 2 ports. |
|||
{{Template:Version badge Mx}}'''This page focuses on aspects of MX that are specific to the Linux operating systems.''' |
|||
* Raspberry Pi Zero W |
|||
** Pi Zero W has WiFi and one micro-USB port which is all that is needed for headless running. |
|||
** Installing onto a faster Pi might speed parts of the installation process, but for actual ‘production’ running this slower Pi will be perfectly adequate. |
|||
** It could run a web server, but that might really slow it down. |
|||
** If you run this headless, all updates are done remotely, so the connectivity and speed of the actual Pi are less important |
|||
* Raspberry Pi Three B Plus |
|||
** The faster speed of this Pi although NOT necessary for running Cumulus MX, will cope better if you are asking MX to do lots of processing (e.g. updating database tables or external sites as well as standard processing). |
|||
** Pi Three B Plus has a socket for an external power supply, Ethernet socket (supports wired link); a HDMI socket for audio/video to TV, or computer monitor; a standard audio socket for external microphone, or speakers; 4 standard USB 2 sockets for weather station, mouse, keyboard, USB stick, or other storage device; plus other connections (e.g. camera). |
|||
** This might be better if you also want to run a web server, or other tasks on the same Pi. |
|||
** Also consider this model if it is to be used on a remote site so when you visit it is useful to be able to plug in a monitor and other peripherals, and to spend as little time on updating as possible. |
|||
Still believe it will be too complex for you? The developer has created [[Software#Raspberry_Pi_Image|an image you can download]] for those prepared to run two computers (a RPi for actually running MX and another computer for all interactions with MX). Read all about it, on [[Raspberry_Pi_Image]] page, and decide if that is for you. |
|||
Other models are available, but you need to check their specification against your needs. For example, the current model Four has more capability, but is less appealing as it also consumes more power. |
|||
==Device Coverage== |
|||
== What else to buy == |
|||
Linux is available based on a multitude of different kernels (the building block for the operating system), on a multitude of devices. |
|||
This obviously depends on your PI model and on your weather station connection type. |
|||
This page has been originated by a contributor using the Raspberry Pi Operating System (this is based on Debian, one of the Linux kernels). Be aware therefore that some instructions on this page are specific to a [[Raspberry Pi computer page|Raspberry Pi computer]] with its default operating system. |
|||
You may want to buy a case. A case specifically designed for your Pi model will have cut-outs in the right place for each interface connection and will have sufficient ventilation for the electrical components to not over-heat. |
|||
For other devices, the inclusion of the correct instructions is totally dependent on whether any contributor has edited this page to cover your device in the context of that section of this page. It is hoped that contributions to this page will be made by Cumulus users with a range of different devices so this page is useful to more people. |
|||
You may need a power supply. This could be an official Raspberry Pi power supply. Alternatively, any power supply unit that has a micro USB connector will do, the power consumption of a Pi (whichever model) is fairly small, but it will be powered on 24/7, so a low power consumption ‘switched mode’ type is preferred – i.e. one that does not become warm when plugged in with nothing attached. You may have a suitable one from a mobile phone. |
|||
:Until somebody creates a separate page for Apple Mac computers (that could be a good idea, as there are some significant differences), this page is the best source of advice. |
|||
You may need to buy connection leads. You may need a HDMI lead to connect your PI to your TV or a spare computer monitor. You may need a USB lead to connect to your weather station (the model Zero requires a micro USB, the model 3 requires a standard A end USB) which probably has a USB A end connection. If your weather station connects by Ethernet, you will need one lead to connect the station to the router and possibly another to connect the hub or router to the Pi. |
|||
==Further Information== |
|||
= Setting up your Pi = |
|||
There are various related pages to get more information: |
|||
You can find, online, instructions about setting up a Pi. The obvious place to look is [https://www.raspberrypi.org/documentation/setup/ the manufacturer's web site], or the same documentation on [https://github.com/raspberrypi/documentation/blob/master/setup/README.md their github page]. However, you might look at [http://www.okdo.com/gettingstarted a supplier's web site], or various other sources of "how to" advice. |
|||
* If you encounter a problem when running MX, please see [[What to do when I have a problem with MX]] |
|||
* If MX gives you a message saying "you are not running the latest version", please see [[Updating_MX_to_new_version|Guide to upgrading MX]] |
|||
* If you are puzzled by the terminology, please see [[:Category:Terminology]] for links to pages that explain terminology used by Cumulus (these pages were written for the legacy Cumulus 1 and may need updating for MX) |
|||
* If you need to know more about files in the installation, please see [[:Category:Cumulus Files]] for links to all Wiki pages describing the sub-folders and files used by MX |
|||
* Go to [[:Category:Cumulus MX]] for links to all pages in this Cumulus Wiki that relate specifically to MX |
|||
* [[MX Administrative Interface|Admin interface]] provides information on configuration and web pages for viewing your weather data locally |
|||
* The [[Cumulus MX FAQ]] page was created with snippets from the forum, but nobody has yet sorted this out into a useful page or updated it for recent releases |
|||
* If you were using the original (now legacy) Cumulus software, please read [[Migrating_from_Cumulus_1_to_MX]], although that is mostly directed at those using MX on the same Windows PC as they used for Cumulus 1, and was written for an old MX release, it will help you understand configuration differences. |
|||
* If you want to use a script language, you might want to read [[PHP|PHP Hypertext Pre-processor and JavaScript]] page |
|||
* If you will be using the standard web pages (from release 3.10.1) see [[New_Default_Web_Site_Information|this page]] |
|||
* If you want to write your own customised templates, read [[Customised_templates]]. |
|||
* If you want to explore alternative web pages from third-parties, start [[:Category:User Contributions|on User Contributions page]]. |
|||
== The Micro-SD card == |
|||
A Pi requires either a class 4, or a class 10, micro-SD card (or whatever goes with your Pi model) with a minimum of 8 GB. Various suppliers offer cards of 16 to 64 GB with NOOBS pre-installed ready for use in a Pi. You may even buy a kit that includes a Pi board with components and interfaces on it, a power supply, a micro-SD card, some connection leads, and a case that you can fit the board into to protect it. |
|||
=Preparing your computer for installing the Cumulus MX suite= |
|||
Setting up a Pi is simple assuming you have bought a micro-SD card that is, either pre-installed with [https://github.com/raspberrypi/documentation/blob/master/installation/noobs.md NOOBS], or with the Raspbian Pi operating system pre-installed. NOOBS makes it easy to install (by default) Rasbian (insert the SD card into the Pi and when you power up the Pi, the operating system will be installed during that first boot. If you have a model with an Ethernet connection, and that is connected first, then NOOBS can offer a choice of other operating systems, or you can download another compatible operating system and install it on the micro-SD card. |
|||
Please see [[Preparing your Linux computer for MX]] page if you have not installed MX on Linux before. |
|||
If you want to be able to use the micro-SD card in other devices (like your PC), you may need to also buy an adapter which allows the micro-SD card to be plugged into a standard SD socket. This may be included if you buy a micro-SD card not sold specifically for the Pi. |
|||
That page covers: |
|||
== The Operating System == |
|||
* [[Preparing your Linux computer for MX#Operating systems|Installing Operating Systems]] |
|||
* Package manager |
|||
** [[Preparing your Linux computer for MX#Interactive Package management on RPi|Interactive version]] |
|||
** [[Preparing your Linux computer for MX#Using Package Manager in terminal mode|Terminal version]] |
|||
** [[Preparing your Linux computer for MX#The various components to commands for installation|Commands]] |
|||
* [[Preparing your Linux computer for MX#USB HID|Human Interface Devices]] |
|||
* [[Preparing your Linux computer for MX#Installing Mono instruction|'''Installing Mono''']] |
|||
* [[Preparing your Linux computer for MX#Moving from Microsoft Windows to Linux|Preparing Microsoft Windows files for Linux]] |
|||
* [[Preparing your Linux computer for MX#File Names & Paths|Guide to file names and paths]] |
|||
As said before, a number of operating systems can be installed on your Pi, even Windows. I leave it to you to read, on-line, how to install the operating system you choose, normally the instructions will be found from where you download the system you have selected. |
|||
If you have a micro-SD card pre-installed with an operating system, you are ready to go! |
|||
Please be advised some of the above is rather technical reading, but ''Mono is required to run the Cumulus packages'' described next. So do ensure that you installed Mono before continuing. |
|||
If you have a micro-SD card pre-installed with NOOBS, as described above, the first boot will install the operating system. By default, it will be the latest Raspbian OS when the card was manufactured. |
|||
==Technical aside== |
|||
If you want to install Raspbian, and it is not pre-installed, download from https://www.raspberrypi.org/downloads. That imager is run on any device, say your pc, and then you select '''write''' to save it onto the micro-SD card (don't forget this overwrites anything already on the card). This should work without a need to format the card first,(but if you do need to format it, do so using a SD card formatter downloaded from https://www.sdcard.org/downloads/formatter_4/index.html, not the Windows format tool). After this image has been stored it will have created two partitions (one the boot partition can be accessed by Windows, but the larger partition is invisible to Windows). |
|||
Please note this Wiki page talks about "folders" for compatibility with the [[MX on Windows OS]] page, but Linux prefers to call them directories. |
|||
== Pre-configuring the Pi for headless operation == |
|||
Linux has a well defined filesystem, represented as a hierarchic tree starting at the root "/", that is divided into directories (one of which will be "/boot" and hold the kernel), each of those first level directories can be divided into second level directories, this second level is sometimes referenced to as defining the "scope", an indication that each is meant to be used for a specific purpose. The scope can be sub-divided again at lower levels representing "categories" (categories cover items like binary code, documentation, configuration, hardware, source code, runtime and content), and at a lower level still "applications" (i.e. related to specific programs) with further sub-levels for various options within those applications. Many Linux distributions will use logical links so references to a directory at one level in the hierarchy will actually redirect to files in a different directory, this might be because different programs expect to see files in different places or just to enforce ownership and writing rights. |
|||
You can store files in the boot partition of the micro-SD card that can be accessed by Windows. When the card is inserted in a Pi, these two files will be removed from that partition, one will switch on a setting that is by default off, the other file will be moved to a new location and enable wireless connection to your local network. |
|||
For the purposes of this Wiki, the terminology "operating system" is used for the whole Linux distribution, you will find that Linux technical people prefer to talk about Linux distributions including: |
|||
=== Secure Shell Home === |
|||
# a "kernel" for the underlying handling of files, network and so on; |
|||
# one or more "shell" components for the handling of commands entered in terminal mode, including those that run programs (whether included in distribution or added later); |
|||
# an optional graphical user interface for simpler access to commands and programs. |
|||
For simplicity the terminology "terminal" is used for how you access the shell, this term refers to seeing the command prompt if your Linux is running without a graphical user interface, or to a window that you can open within the graphical interface where commands can be typed. Depending on your Linux, that window might be called "Terminal", "Konsole", "xterm", "gnome-terminal", "uxterm", or even something else. If you are accessing your Linux computer over a network from a computer running Microsoft Windows, then again you may encounter a number of terms for how to access the shell on your Linux computer, "Command Window", "Windows Powershell", or "Windows Terminal". Equally you may use software that calls it a teletype mode, e.g. PuTTY software. |
|||
Secure Shell (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. By default, this optional way for another device to see your PI is turned off. You can turn it on by the configuration application on your Pi, but you may want it turned on as your Pi boots up and you can do that by adding a file to your micro-SD card. |
|||
=Cumulus packages= |
|||
The file must be named "SSH" with those three letters in capitals, but with no file extension and no contents. On a Windows PC, if you right click (while viewing the boot directory on the card) there is an option called '''New''' and if you select ''a text file'' it will create an empty file with the extension '''.txt'''. (In windows there is an option to hide extensions which is on by default, so you may need to deselect this option ['''New''' menu -> ''Options''] to see this extension). On Windows you can open the file using Notepad to verify it is empty, if you gave accidentally created a file of another type like word processing it will be full of characters some of which do not display. Anyway, you must remove any extension from the file name so it is really just '''SSH'''. |
|||
* This section covers: |
|||
When the card is inserted into your Pi, on boot this file will be removed and the SSH option will be enabled. |
|||
** CumulusMX.exe |
|||
** ExportToMySQL.exe |
|||
** CreateMissing.exe |
|||
(At time of writing this "CreateRecords.exe" is proposed, and under development, but not released). |
|||
=== Wireless Network === |
|||
Again, create a new file, again if you are using Windows you can create it as a text file and then replace the '''.txt''' extension with a ''.conf'' extension, store it in the boot directory on your micro-SD card under the name '''wpa_supplicant.conf'''. |
|||
==Handling zip files== |
|||
Using a text editor that won't add any unwanted control characters, add the following text using UTF-8 encoding: |
|||
<pre>ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev |
|||
update_config=1 |
|||
country=GB |
|||
Each release is presented as a zip. It does not matter which device (if you have two or more computers), or which browser (it can be default browser for your device or the browser you like best) you use for the download. When your browser saves the zip it might be into a folder called “downloads” on your computer, or you may be able to save into another folder that you prefer (perhaps on a different partition). Your browser might even remember the folder you used before for files of type zip. |
|||
network={ |
|||
ssid="YourNetwork" |
|||
psk="YourNetworkPassword" |
|||
key_mgmt=WPA-PSK |
|||
}</pre> |
|||
In general, any device will load a suitable application to use to unzip the package when you click on a filename that ends in '''.zip'''. You might need to do a "right click" and choose the application, it depends on your settings. |
|||
*Obviously, if you are not in United Kingdom, you will replace '''GB''' by the country code that applies to you. |
|||
*Within the first set of quotes, replace '''YourNetwork''' by whatever ''Service Set Identifier'' is used for your wireless network. You may have typed this into your mobile phone. It may be shown on a card that slips into a slot on your hub or router. Whatever it is, and it can be up to 32 characters (letters, numbers, and symbols), type it within the double quotes. Some routers come with a default SSID that is the manufacturer's name, and leave it to you to pick a SSID that is unique to you using up to 32 characters to personalise it. |
|||
*Within the next set of quotes, which relate to the replace '''YourNetworkPassword''' by whatever Pre-Shared-Key (password) is used for your wireless network. You may have typed this into your mobile phone. Hopefully, you have changed it from whatever was shown as the initial password on the card that slips into a slot on your hub or router (even if all you have done is add a prefix or suffix that means something to you). |
|||
*Most wireless networks will use Wi-Fi Protected Access (WPA) or (from 2006) Wi-Fi Protected Access II (WPA2) protocols, so '''WPA-PSK''' is correct for you. Note that your Pi is only able to use these protocols. The earlier Wired Equivalent Privacy (WEP) was officially withdrawn in 2004 as too easy to crack, so it is not supported on a new Pi. |
|||
Be aware, you may need to adjust the settings within that application for how it handles the file structure. The preferences may determine whether the unzip process preserves the file structure used when the zip was created (i.e. each file remains in any sub-folder) or it ignores the folder structure. For the Cumulus context, it is essential to preserve the folder structure. You may also be asked where you want the files to be extracted to, or the default settings might always use a particular destination (and that might be a '''tmp''' folder). |
|||
Should you wish to set up your Pi with several network definitions, please see [https://cumulus.hosiene.co.uk/viewtopic.php?p=139422#p139422 Notes by ExperiMentor] (a contributor to the Cumulus support forum in Switzerland). |
|||
For example on the Raspberry Pi operating system, there is a package called '''xarchiver''', in its Graphical User Interface (GUI), there is a menu called "Action", and the final option in that menu is "Preferences". There, in "Archive" section, you can select "zip" as the preferred archive format (using a drop-down) and whether you want the application to confirm with you before deleting any files; in "Advanced" section, you can select the directory to use for the extraction. If you are using the lite version of the RPi OS, then you need to edit the '''/home/pi/.config/xarchiver/xarchiverrc''' file to set preferences, before you use the archiver package. Once you have started the archiver package, and told it which file to process, you can click on '''Extract files''', |
|||
=== Other configuration === |
|||
the GUI presents a screen of options: |
|||
* "Extract to:", use the icon to browse to the required location if it has not been set up in preferences |
|||
* "Ensure a containing directory", tick this if it has not been set in the configuration file |
|||
* "Files", select "All files", the advice is to overwrite all of any existing files if you are upgrading, but you definitely need all files if this is a new install |
|||
* "Options" |
|||
** Tick "Extract files with full path", this is essential if you are going to successfully install any of the Cumulus software |
|||
** Tick "Overwrite existing files", the advice is to overwrite all of any existing files if you are upgrading, it may not always be clear which files have been updated since an earlier release, and there are a lot of interdependencies between different files |
|||
It is worth stressing here, if you decide to customise any files that are included in a release distribution, then you should at the very least add something like an "_" character to the file name to make your tailored file different to the standard file. The best practice is to put any files you tailor, or any additional files you create, outside the CumulusMX folder. |
|||
There are various other configurations you need to do on your PI. Unlike SSH, these can't be done by storing files on the micro-SD card. You need to use the raspbian configuration tool '''raspi-config''', and this can be accessed on your Pi either in a Graphical User Interface (GUI), or by running a command in Terminal. The same command can be run from a remote device if you successfully have SSH running. On a Windows pc, this will typically involve use of '''PuTTY''' software (an SSH client for Windows) downloaded from <tt>https://www.putty.org/</tt>. In both cases the command to use is <tt>sudo raspi-config</tt>. |
|||
If you have chosen to do the download on a different device to that on which you will install, you can unzip on either device. To transfer either the downloaded .zip file, or the extracted file structure, between devices, you can use a file transfer package, or use a portable drive (a memory stick or even a memory card) with a partition formatted so that you can read it on both devices. Windows and Linux partitions are formatted in different ways. While it is likely that Linux can read a Microsoft formatted partition, Microsoft Windows can never read a Linux formatted partition. |
|||
==== Mandatory ==== |
|||
==Where to install all packages?== |
|||
When you use SSH for the first time to connect to your Pi, you will see a warning that SSH is enabled but the password has not been changed, which is a security risk. So it is mandatory to change the network password for your Pi from the default '''raspberry''' to something that you can remember but makes life hard for anybody trying to hack into your system with malicious intent. |
|||
For simplicity on this page CHOSEN PATH (the contents of this will start with a slash “/”, but not end with a slash) is used to represent any location in the Linux file structure where you decide to install all the Cumulus packages. |
|||
Within the Raspberian configuration utility, you will see an option to change password. You will need to enter the new password twice before it replaces the old one. |
|||
The phrase “CHOSEN PATH” is used, because it is most likely you want to create the sub-folder called “/CumulusMX” (note where capital letters must be used) in a part of the Linux file structure that already exists. |
|||
The default network (host) name for your Pi is '''raspberrypi''', obviously we need to replace that as well with a name that personalises it to you and does not make it easy for a hacker to know what device is represented by that network name. Once we rename this, if we are working headless, we will see an error message: '''sudo: unable to resolve host raspberrypi''', appearing when we leave the configuration utility. These can be safely ignored (it's just because you renamed the Pi) and will disappear after next reboot of your Pi. |
|||
It is important to minimise the length of the path name, because this path name has to be passed between various different software languages (and longer paths risk truncation). |
|||
Within the Raspberian configuration utility, you will see a '''Network Options''' option, it is there that you change the network name. Network options can also be configured by clicking an icon on the Pi (this icon might be two red crosses if network settings are missing, two parallel arrows if the network settings are not correctly set, or the wireless symbol if your wireless network is working). Hopefully, for you the WiFi network and password have already been set by the '''wpa-supplicant.con'''f file added to micro-SD card earlier. |
|||
==== Recommended ==== |
|||
===Creating the CumulusMX sub-folder=== |
|||
If you are going to use the Pi in headless mode, you must select this next configuration. In '''Boot Options''', ''Desktop / CLI'', select '''Console Autologin'''. |
|||
* You can create sub-folder called “/CumulusMX” as you unzip a MX release, or you can type <code>sudo mkdir CHOSEN PATH/CumulusMX</code> first (note that CHOSEN PATH is explained above and always starts with a slash “/”). |
|||
The default locale for a Pi is normally '''en_GB.UTF-8'''. Whatever locale you use, if you have already been using Cumulus (1 or MX), you need to ensure the locale matches the one used for your log files. The versions of MX released in the middle of 2020 are very fussy that all dates use the same delimiter, so you need to check the chosen locale continues to use the same date separator as before. The locale is affected by the version of Mono you install and whether you use the locale parameter when starting MX, so I cannot cover all options. |
|||
* By using the phrase CHOSEN PATH this advice avoids telling you to install Cumulus where you do not want it: |
|||
*# Many people with a Raspberry Pi, and a little technical understanding, add an external drive to reduce wear on the internal micro-SD card, and keep their Cumulus files away from the drive that holds the operating system. |
|||
*#* This page is not going to get technical by telling you how to create, or mount, Linux partitions on your external drive. If your drive was bought from a Raspberry Pi reseller, they might help you. |
|||
*# Other people using a Raspberry Pi without that technical expertise, might use ‘’’/home/pi’’’ for CHOSEN PATH as that is the default folder for the default user (Pi) and can be referenced as "~" in file path instructions they issue (although Cumulus will not understand that shorthand) |
|||
*#* Within that ‘’’/home/pi’’’ folder, the default user has full permissions automatically. |
|||
*# The developer suggests you use ‘’’/opt’’’ for CHOSEN PATH (which should be available on any Linux computer). |
|||
*#* By default, the code Mark provides for installing Cumulus as a service, will run that service as a root user, and the root user has full permissions in /opt (and everywhere else) |
|||
*#* (Novices: Skip this step) If you do choose a CHOSEN PATH outside your home folder, then a more technical user can change the ownership of the "CumulusMX" sub-folder, to the default user (Pi) with <code>sudo chown -R pi: CHOSEN PATH/CumulusMX</code>, and reduce the need to use "sudo" on many actions. |
|||
==Packages to install== |
|||
Anyway, the default locale is fine if you are in the UK, you use decimal points for real numbers, you use commas for list separators, and you don't have dates with month first! |
|||
<big>We shall install the Cumulus software listed on [[Software]] page</big>: |
|||
To change the locale, enter '''Localisation Options'''. Note that there is a Wi-Fi network country, but that has already been set by the '''wpa-supplicant.con'''f file added to micro-SD card earlier. In the same option area, there are some more options: |
|||
# '''CumulusMX''': |
|||
# Change Time-zone, by default UTC is used all year round. In the UK if your Cumulus is set to roll over at 10am in summer, you will wish to change the time-zone to UK time. |
|||
#* '''CumulusMX.exe''' is written in C#, that has been developed by Mark Crossley, but still contains some code by Steve Loft |
|||
# Change Keyboard Layout if needed, keyboards can support different numbers of characters, and can have different currency symbols, so select whatever is relevant to you |
|||
#* Download '''CumulusMX zip file’’’ from the link at [[Software#Latest_build_distribution_download]] |
|||
# [[Software#Create_Missing|'''Create Missing''']]: |
|||
#* '''CreateMissing.exe''' is also written in C#, created, and developed, by Mark Crossley |
|||
#* Download '''Create Missing zip file''' from the link at [[Software#Create_Missing]] |
|||
#** This takes you to a github page with a "ReadMe" providing minimal instructions |
|||
#* Using '''CreateMissing.exe''' is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki |
|||
#* (it will populate missing fields in [[standard log files]] and/or missing lines in [[dayfile.txt]]). |
|||
# '''ExportToMySQL''' |
|||
#* '''ExportToMySQL.exe''' is also written in C# by Mark Crossley |
|||
#* Download '''Export To My SQL zip file''' from the link at [[Software#ExportToMySQL]] |
|||
#** This takes you to a github page with a "ReadMe" providing minimal instructions |
|||
#* '''ExportToMySQL.exe''' is not (at the time this was written) documented in this Wiki although [[MX_Administrative_Interface#MySQL_settings]] does describe a similar utility (written by Steve Loft) that was actually included within early CumulusMX zip downloads. |
|||
As at 9 March 2020, another utility, '''CreateRecord''', has been initialised in the Github areas managed by the developer where Cumulus is worked on, although it appears to be just a concept on github. This will, if my understanding is correct, read [[dayfile.txt]] and use that to update the various [[:Category:Ini Files|extreme record files]]. The developer is still aiming to make this available, but his work on it (on his computer) has been stalled by the level of pressure being applied for bug-fixes and changes to MX itself. |
|||
====To leave configuration==== |
|||
===Alternative download link for older package releases=== |
|||
Select ‘Finish’. |
|||
Because the developer uses Git Hub to manage releases, the older releases remain available. |
|||
== Installing Mono == |
|||
====Old Cumulus MX packages==== |
|||
Sponsored by Microsoft, Mono is an open source implementation of Microsoft's .NET Framework based on the ECMA standards for C# and the Common Language Runtime. |
|||
Skip this subsection if either you have installed the "pre-built disc image", or the current MX release is stable (it has been available for a while and nobody has reported any bugs). |
|||
Check if posts in the [https://cumulus.hosiene.co.uk/viewforum.php?f=40 Cumulus Support Forum] tell you that the current release of MX has one or more bug(s) that affects one or more aspect(s) of MX that you intend to use. |
|||
=== Preparing for Mono installation === |
|||
Remember, it is impossible for the developer to check all the ways in which versatile MX can be used: |
|||
Quite often when we try to install, or update, packages on our Pi we will see messages about dependencies, and in some cases error messages saying the installation has failed or been aborted. Before we can install Mono, there are other packages required and these depend upon which Raspbian operating system we have installed, see [https://www.mono-project.com/download/stable/#download-lin-raspbian Mono instructions for Raspberian]. Here are 2 of the options (if your Mono installation fails, then you selected wrong one): |
|||
* Different weather station types (the developer only has a Davis), |
|||
* Different computer types (development is mostly on Microsoft Windows), |
|||
* Plus a whole host of optional features, and different external upload sites, (typically each of these optional features are only used by a sub-set of Cumulus users). |
|||
Anyway, '''you can download any earlier MX build, without the bug''', from [https://github.com/cumulusmx/CumulusMX/releases CumulusMX/releases]. |
|||
For Raspberian 9 (stretch): |
|||
<pre>sudo apt install apt-transport-https dirmngr gnupg ca-certificates |
|||
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF |
|||
echo "deb https://download.mono-project.com/repo/debian stable-raspbianstretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list |
|||
sudo apt update |
|||
</pre> |
|||
====Old utilities==== |
|||
For Raspberian 10 (buster): |
|||
<pre>sudo apt install apt-transport-https dirmngr gnupg ca-certificates |
|||
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF |
|||
echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list |
|||
sudo apt update</pre> |
|||
The zips for "CreateMissing.exe" and "ExportToMySQL.exe" utilities do NOT contain the '''''.dll''''' components that they need when they are running. |
|||
This means that each version of "CreateMissing.exe" and "ExportToMySQL.exe" utilities is dependent on it being used with a release of Cumulus MX that does have correct '''''.dll''''' components in its release distribution. |
|||
That in turn means you can't use the latest version of the utility with older MX releases, nor can you use an old utility version with latest MX release. This is why [[Software#By_Mark_Crossley|utilities downloads]] make clear which MX release is the minimum for them. |
|||
The older versions of these "CreateMissing.exe" and "ExportToMySQL.exe" utilities are available by going directly to the [https://github.com/cumulusmx Cumulus MX github] page, and navigating to the utility of interest. However, to use those older versions, you also need to download the corresponding older MX release, because the MX distributions contain the .dll files that the utilities require, they are not in the utilities zip. Because of this complication, novice users are advised not to attempt to use the older utilities, even if the latest version appears to have a bug. Technical users may be able to work out which .dll files are needed and can be safely added back (if they are not left over from when that past MX release was in use). An alternative is to create a new folder with the old release packages (MX and the utility of interest), a copy of the latest Cumulus.ini file, and a copy of all files from /data sub-folder; then afterwards copy back the changed files into original /data folder. |
|||
==Upgrading a Cumulus package== |
|||
Always check the release announcements in [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 Cumulus MX announcements] for any action needed in planned upgrade. In brief, all files from new release distribution replace the files from previous release, and the download/unzip is as covered above. |
|||
No further action needed for upgrade of "Create Missing" or "Export To My SQL" or "Create Records". See below for upgrade of main Cumulus MX package. |
|||
If you are running an older MX release, before skipping in-between versions please check [[Updating_MX_to_new_version|here]]. |
|||
If you run MX as a service, then: |
|||
# Ensure you are not doing any changes to settings |
|||
# Leave MX running as you copy files from the new release distribution over the existing files |
|||
# Try to pick a time just after MX has done any standard interval store of readings or upload, so that it is least busy |
|||
# Use <code>sudo systemctl restart cumulusmx</code> to stop and then restart MX picking up the new files |
|||
# Result, downtime of MX kept to a minimum, so avoiding losing data |
|||
If you run MX interactively, do the unzip into a temporary location before you stop MX, then copy all files from temporary location over your existing files, and finally restart MX. Depending on your weather station type, it might or might not offer historic data catch-up, so you might lose some data while MX is stopped, and therefore should keep the downtime to a minimum. |
|||
The alternative is to install in a new folder (or rename the old one), and copy across files not in the release from old location to new location, but in that alternative you might forget some files. |
|||
==Changing location of Cumulus MX== |
|||
If your install, or upgrade, is creating MX in a different place to where you previously ran Cumulus, then you will want to copy files across that are not in the zip extract distribution. |
|||
===Configuration Files to copy across from any previous Cumulus installation=== |
|||
= old notes = |
|||
There are two configuration files that are not included in any MX release: |
|||
These comprehensive notes describe how to install Cumulus MX on a Pi Zero, using a PC to do some of the work: |
|||
*[[strings.ini]] (note all lower case) – optional file to customise output |
|||
*[[Cumulus.ini]] (note initial capital, then lower case) – main configuration file |
|||
Here, it must be stressed that having either or both of these files in an existing Cumulus installation does not imply such file or files can be understood by the new MX release you have just installed. |
|||
Just copy the existing files from old to new installation, if |
|||
''' |
|||
# Your locale is still the same |
|||
Download useful PC software and install on your PC''' |
|||
# All files on your new install are in same paths as on your old install (some settings involve specifying paths) |
|||
These instructions are for a Windows PC. Steps would be similar on a Mac, but programs and details would differ. Should also be possible with an Android tablet. |
|||
# Your old installation has a relatively recent MX release (compare the "y" in 3.y.z,between old and new installation, a difference of more than 1 in that middle figure means you do not have a recent release) |
|||
* SD Formatter (the Windows Format facility will NOT do) |
|||
# Your old installation was on a Unix-based computer (not a computer running Microsoft Windows Operating System) |
|||
** |
|||
* balenaEtcher (for unzipping and burning images to SD cards) [Previously named 'Etcher'] <tt>https://etcher.io/</tt> |
|||
* Win32DiskImager (for backup & restore of SD card images) <tt>https://sourceforge.net/projects/win32diskimager/</tt> |
|||
* PuTTY |
|||
* FileZilla (an FTP file transfer program for Windows) <tt>https://filezilla-project.org/download.php</tt> |
|||
Please see the table below for more advice, but the problem is that content of both files has changed as MX has been developed, so some content is no longer understood, and some new content has been added. |
|||
'''Download Raspbian Pi Operating System''' |
|||
* Save it on your PC, from https://www.raspberrypi.org/downloads/raspbian/ |
|||
* "RaspBIAN Buster Lite" is probably OK, but other than small file size it offers no advantage over installing the full version of RaspBIAN Buster. These instructions are being tested using "Raspbian Buster with desktop and recommended software", the largest of all, which could allow you to do other things more easily. |
|||
* Just click on “Download Zip” (torrent might be faster if you have the ability, but not worth installing just for this) |
|||
* Do not unzip it |
|||
* These instructions have been tested with kernel version 4.14, released 18 April 2018 and with kernel version 4.14, released 13 November 2018 [March 2019] and kernel version 4.19 released 10 July 2019 |
|||
Some of the differences between versions of '''Cumulus.ini''' file can be seen by comparing the different pages in this Wiki documenting this file: [[Cumulus.ini (Beta)]], [[Cumulus.ini (Cumulus 1)]], [[Cumulus.ini (MX 3.0.0 to 3.7.0)]], [[Cumulus.ini (preserving history)]], and [[Cumulus.ini]], but even that does not tell the whole story. MX release 3.12.0 needs to be installed if your old Cumulus was earlier than that, because it is the only release with code to rename the old "Cumulus.ini" and create a new file containing the new set of settings, and new names for some old settings, but without any old settings that are no longer recognised. Please see [[Updating MX to new version]] page for more information about the need to step slowly through from old releases to the newest. |
|||
'''Install Pi Operating System onto Micro SD card''' |
|||
''Format the SD card'' |
|||
* Put Micro SD card in PC (use adapter if needed) |
|||
** If re-using a previous Pi SD card, click ‘Cancel’ on the warning about needing to format the card |
|||
* Run SD Card Formatter (click Yes to ‘Allow to make changes to your device’). |
|||
** Need to use this program rather than the Format tool in File Explorer, because Pi SD cards end up with a very small ‘Windows accessible’ partition and a large partition containing Linux. SD Card Formatter allows reclaim of the large partition. |
|||
* Your SD card should automatically populate in the ‘Drive’ box. In case you have another SD card in your PC, ensure the correct card is selected! |
|||
* Click ‘Format’ and check and accept the Warning messages |
|||
If you are upgrading from an older release, please read the table for advice. |
|||
'''Copy the Pi Raspbian Operating System onto the card''' |
|||
* Run '''balenaEtcher''' on your PC |
|||
* Click ‘Select Image’ and choose the ‘Raspbian Buster’ operating system zip file that was downloaded earlier |
|||
* SD card should be automatically populated. In case you have another SD card in your PC, ensure the correct card is selected! |
|||
* Click ‘Flash!’. The operating system will be copied to the card. This takes about 10 minutes, followed by another 8 minutes to ‘Verify’ |
|||
* Cancel any messages about needing to Format the card - they are just indicating that Etcher has installed the partition that cannot be read by Windows |
|||
* On completion, the card is ‘ejected’ from the PC. Physically remove it and then straight away reinsert it so that the content can be viewed in File Explorer |
|||
* TWO drives will now be visible for the SD card. You will likely see a warning that one of the drives needs to be formatted before it can be used. ‘Cancel’ that warning and ignore that drive. |
|||
* View the other drive, which is named ‘boot’ in File Explorer |
|||
* On the View tab, ensure the ‘File Name extensions’ is ticked |
|||
* Right click and select ‘New’, ‘Text document’. Change its name to SSH (deleting the .txt extension; you need to make an empty file called SSH not SSH.txt). Click ‘Yes’ to ‘Are you sure you want to change the extension?’ |
|||
* Right click and select ‘New’, ‘Text document’. Change its name to wpa_supplicant.conf (deleting the .txt extension; you need to make a file called wpa_supplicant.conf not wpa_supplicant.conf.txt). Click ‘Yes’ to ‘Are you sure you want to change the extension?’ |
|||
* Right click on this new file and select ‘Open with Notepad’ or ‘Open with …’ then select Notepad. Enter the following content exactly as below (copy and paste) then edit your country code (if needed), WiFi network’s SSID and password: NOTE: Change GB as needed to be the code for your country. The quote marks should appear in the file, that is ssid="YourNetwork" not ssid=YourNetwork . Same for psk. |
|||
{| class="wikitable" |
|||
|- |
|||
! scope="col" style="width:450px; color:blue" | Cumulus.ini !! scope="col" style="width:450px; color:navy" | strings.ini |
|||
|- |
|||
| Your old installation will have this file. In general, ''if your old installation was any release before 3.8.0'', the advice is give the old file a different name when you copy it across to the new installation, and let MX create the file as you work through all the settings. |
|||
| '''This is an optional file'''. Its [[strings.ini|purpose]] is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language. |
|||
|- |
|||
| When you work through the Settings pages, MX will create this file if it does not exist. |
|||
* See [[#Moving from Microsoft Windows to Linux]] if your old installation is on a Microsoft operating system, as several changes will be needed for extra web file settings on your Linux computer |
|||
* If your old installation was of the legacy software then also see [[Migrating from Cumulus 1 to MX]] |
|||
* As MX evolves, the former "read-only" settings in this file are becoming "advanced" settings in the interface. |
|||
| You create a “strings.ini” file by '''selecting some of the parameter'''s from the [[Samplestring.ini]] file that is included in each MX release, and ''modifying the value for the listed attributes'' as you type just those you selected (under the same group titles - these are enclosed in [ ] as before). |
|||
The sections that appear in '''samplestring.ini''', and the parameters that appear within a section, depend upon which release you are using. So be cautious if you try to reuse a "strings.ini" file originally created by the legacy software, you may find you need to specify your customisation using different parameters in the latest "samplestring.ini". |
|||
* Not essential, but I like to keep copies of both those files for future use. They can be on the SD card with different names eg ‘SSH - Copy’ and ‘wpa_supplicant.conf - Copy’ as well as on your PC |
|||
|- |
|||
* The function of these 2 files is to connect your Raspberry Pi to your network as soon as it boots, and allows you to connect to and control it from your PC by SSH using PuTTY. This avoids needing to connect a keyboard, mouse and monitor to the Raspberry Pi. It is particularly useful for Pi Zero W (or Pi Zero) which hasn’t got enough USB connections and no Ethernet (wired network) connection. This is called ‘Headless operation’. |
|||
| The content of "Cumulus.ini" is changing as MX is developed, the [https://cumulus.hosiene.co.uk/viewtopic.php?f=40&t=17887 Release Announcements] normally list any new parameters as they appear in the file, without always mentioning those that have become redundant. The announcements tend to avoid any detail, so you have to guess ''from the attribute'' what values it might take, and generally have no idea of where in the settings pages to make any change. |
|||
* Right click on the ‘boot’ SD card in left pane of File Explorer and ‘Eject’ it safely. |
|||
To remove any parameters no longer used in this file, see [[Cumulus.ini#How_to_Remove_Redundant_parameters_from_file|remove redundant parameters]] |
|||
'''Setting up the Raspberry Pi''' |
|||
* With nothing plugged into the Raspberry Pi, take the Micro SD card from your PC and put it in the Pi. |
|||
* In a later step, you will need to find out the Raspberry Pi’s IP address by looking at your network router’s web interface. I can’t help you with doing that. If you don’t know how to, an alternative is to connect a keyboard, mouse and monitor to the Raspberry Pi at this stage |
|||
* Plug the power supply into the Raspberry Pi. It will boot up (note flashing red and/or green LEDs depending on model). |
|||
* On your PC, log into your network router’s web interface and identify the Pi’s IP address, which will be in the form xxx.xxx.xxx.xxx, for example 192.168.1.123 |
|||
** NOTE: If you will be switching from a faster “build” Raspberry Pi to a “production” Raspberry Pi Zero W, the IP address will change, so you’ll need to repeat this step later |
|||
** While in your network router for the ‘production’ Pi you will be using, set up some port forwarding that will be needed later. |
|||
** Forward port 8998 to your Pi’s IP address for TCP protocol if you want to be able to access the Cumulus web interface from the external internet (this brings potential security risk though). [Forwarding port 8002 as well was previously needed]. |
|||
* Start PuTTY on your PC. In the box for ‘Host Name or IP address’, enter the Pi’s IP address from above. In the adjacent ‘Port’ box, enter 22. Connection type should be SSH. Click ‘Open’. |
|||
* A window opens. The first time you do this you will probably see a long message asking to confirm it is OK to connect to a not-previously-known device. Click ‘Yes’. |
|||
* Login to the Pi. Username is pi [lower case] and password is raspberry [lower case] |
|||
* You will see a warning that SSH is enabled but the password has not been changed, which is a security risk. We will change the password in a moment |
|||
* Type |
|||
<pre>sudo raspi-config</pre> |
|||
If your old file contains any [[Cumulus.ini (Cumulus 1)|legacy read-only]] parameters not yet converted into advanced settings, or any [[Cumulus.ini (MX 3.0.0 to 3.7.0)|MX read-only parameters not yet converted into advanced settings]], you may need to manually add such missing parameters back into new file by stopping MX (after finishing all the settings you can configure in the interface), doing an external file edit, and then restarting MX. |
|||
* Note, to copy from here (usually need to do 1 line at a time), select it then CTRL-C. To paste into the PuTTY window, right click. |
|||
| The content of "samplestring.ini" is changing as MX is developed: |
|||
* Therefore, your existing “strings.ini” might need to be modified. |
|||
* There is no automatic way to check your “strings.ini” file, if MX does not understand any parameter in this file, it ignores it. |
|||
* Instead, you need to manually check each parameter you have in your “strings.ini” file to see if that parameter is still in the “samplestring.ini” included in the release you have installed. |
|||
* You may also find new parameters in “samplestring.ini” that you wish to add to your “strings.ini” file to tailor new functionality to your preferences. |
|||
|} |
|||
If you previously used an older release of Cumulus, but in this new installation will be using the latest release (latest is what is normally best, unless it has bugs), you may want to read up on all the changes between your old release and the current release, not just changes that affect the configuration file. |
|||
If you have used Cumulus 1 before, and decide to start with a new "Cumulus.ini" file, then you will need to work through all settings, to ensure they are set as you want. Please remember that when you use MX for first time, it uses that date it was first run as a starting date, and ignores any data found with earlier dates. Therefore you must change that start date: |
|||
'''MX interface''' --> Setting menu --> '''Station Settings''' --> Click on ''General Settings'' --> Click on '''Advanced Options''' --> Edit ''Records Began Date'' following instructions below that field |
|||
==="data" directory=== |
|||
Please see [[:Category:Files_with_Comma_Separated_Values]], [[:Category:Ini_Files]], and [[Weather_Diary]], for information if you are moving from Cumulus 1 to MX. Otherwise just copy files from any existing folder to your new one. |
|||
You may also wish to read: |
|||
** [[Amending dayfile]] tells you about how MX is far more fussy about the content in [[dayfile.txt]] |
|||
** [[:Category:Ini Files|.ini files]] explains how time-stamps are formatted differently in the extreme tracking files |
|||
** [[Migrating from Cumulus 1 to MX]] gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date |
|||
Complications occur if the locale used by '''Mono''' or the locale specified when starting MX using [[#-lang parameter for changing Locale|-lang parameter]] differs from the locale for your previous device (please see [[:Category:Files with Comma Separated Values]] because some locales separate fields with commas, some separate integer and decimal parts of real numbers with commas; not to mention all sorts of issues with how dates are formatted). The main MX developer proposed that the format of files with comma separated values will be fixed from a release planned for September 2020, so all dates will use one standard format, all numbers will use decimal points, and the field separating character will be fixed. |
|||
Update May 2022, this has been put on hold, no public MX release has this restriction yet. |
|||
==="Reports" directory=== |
|||
By default MX now creates monthly and annual reports that are in the style used by NOAA in USA. If you have been using this functionality before (and it is optional) then you need to file transfer, or copy, all the files that were in the old [[Reports folder]] into the new folder of that name. Do look at that cross-reference, and read about the encoding default differences between Cumulus 1 and MX. |
|||
=MX can cause problems with storage= |
|||
MX now assumes by default that you are going to use its [[New Default Web Site Information|Default Web Site]]. That means that by default MX will re-generate temporary files in its [[Web folder|/web sub-folder]] on a frequent time-scale. That number of files writes will considerably shorten the working life-time of the "high capacity micro-SD" card that is the default storage for the Raspberry Pi. It will also considerably shorten the life of any flash memory (e.g. memory card) that you might install MX on. |
|||
In the steps below, you will need to press '''y''' to agree to proceed at various times |
|||
The expected life of any storage device, and the extent to which its life is shortened depends on the actual device. The external devices that have the longest life (and therefore can cope most easily with multiple read/write actions) are solid state discs (SSD). Also the larger the capacity of the storage device, the more places on the device where files can be stored and the storing algorithm will try to spread the storing evenly across the entire storage area, so wear at any one location is reduced. |
|||
If you have been building the Micro SD card on a fast Pi, now is the time to switch to the 'production' Pi, for which a slower Pi Zero W is more than adequate. |
|||
Shut down the Raspberry Pi safely. |
|||
<pre>sudo halt</pre> |
|||
All Linux computers will have some random access memory chips (RAM) and it is worthwhile to define part of that RAM as a drive used for temporary files. For a Raspberry Pi computer, a typical approach would be to edit the fstab file, adding the line ''tmpfs /run/tmp tmpfs nodev,nosuid,size=1M 0 0'', but the size you choose will depend on RAM available and what temporary files are being created. For maximum life of the "high capacity micro-SD" card if that is what your computer boots from, you should create a symbolic link path that maps the '''/tmp''' folder used by the system to your '''/run/tmp''' you have just defined in RAM. The difficulty will be that you cannot create a logical redirect on '''/tmp''' if the folder is already in use, so that makes it too complicated to explain here. |
|||
'''Move the micro SD card to the Pi Zero W'''. |
|||
Power on the Pi Zero W. Your SSH (PuTTY) session will close out and you'll need to reconnect after the Pi has rebooted. Use username pi and the new password you chose earlier. |
|||
==web directory== |
|||
'''Add the ‘Mono’ package''' |
|||
* Simplification: Mono is a package which allows programs to be written cross-platform so that they will run on Linux (including Raspberry Pi), Windows and Mac OS, similar to the Windows ‘.NET Framework’. |
|||
* The previous anomaly with the USB library not working with later versions of mono, affecting Fine Offset stations and the later Oregon Scientific stations (WMR88/100/200 etc) has been fixed (''in CumulusMX build 3044 onwards'') and these and other stations should now be fine with later/current versions of mono. I am currently using a Fine Offset with mono v5.18 |
|||
* Process is to install a security certificate, add the mono server to the list of software sources [sources.list] that the Pi searches, then install the mono-complete package: |
|||
<pre>sudo apt-get update && sudo apt-get upgrade |
|||
sudo apt-get install -y mono-complete |
|||
sudo apt autoremove</pre> |
|||
All the files in this folder come from the download. |
|||
At the time of writing (''18 Sep 2019''), this gets Mono v6.0.0.334, which works with Buster (RaspBIAN 10). However, there have been reports of incompatabilities which require use of an older version of Mono. These may have now been fixed, or alternatively may be related to use of locales other than en_GB.UTF-8 . Please see other threads in Support Forum for discussions. |
|||
NOTE: ''29 Feb 2020'': added '''-y''' into the line '''sudo apt-get install -y mono-complete''' . This makes the install bypass the usual 'Continue Y/n?' prompt¨which was causing strange problems for some, e.g. worked if just pressed 'Enter' to accept default 'Y', but aborted installation if pressed 'Y Enter'. Bizarre. |
|||
''' |
|||
Reboot your Raspberry Pi''' |
|||
This would be a reasonable time to reboot your Pi: |
|||
<pre>sudo reboot</pre> |
|||
However, when you are running MX, it may try to create temporary files here, and following the advice above, you may decide to set up symbolic links so any attempt to create a temporary file in the "web" folder is redirected to the temporary folder you set up in RAM. |
|||
Your SSH (PuTTY) session will close out and you'll need to reconnect after the Pi has rebooted. Use username pi and the new password you chose earlier. |
|||
''' |
|||
Install Cumulus MX on the Raspberry Pi''' |
|||
Download it [[Software|from here]] to your PC, unzip on your PC which makes a directory named CumulusMX. Remember where that directory is located then on PC run FileZilla |
|||
# In the ‘Host’ box, enter the Raspberry Pi’s IP address eg 192.168.1.123 |
|||
# In Username, enter pi |
|||
# In Password enter your pi’s password |
|||
# In Port, enter 22 |
|||
# Click ‘Quickconnect’. Raspberry Pi’s directory structure appears on the right and your PC’s directory structure is on the left. |
|||
# In the LEFT window, navigate to where you unzipped the download of Cumulus MX earlier. Ensure can see the folder name ‘CumulusMX’ in the lower left window |
|||
# In the RIGHT window, ensure that the folder /home/pi is shown (see top right window; contents in bottom right window include .cache, .config etc) |
|||
# Drag the folder ‘CumulusMX’ to an empty area in the lower right window (not onto one of the existing directories). Watch progress as this copies the whole CumulusMX folder and contents to directory ~/CumulusMX on the Pi |
|||
# Close FileZilla |
|||
The links you need depend on which options you select in settings, you might find it easier to wait until you have run MX for a while to see what files are created that end in ".json". |
|||
'''On Raspberry Pi PuTTY window:''' |
|||
<pre>sudo halt</pre> |
|||
If MX is currently running, you need to stop it, or at least alter any options that generate .json files. Then you must delete those files that end in ".json", except that you don't delete "websitedataT.json". |
|||
Plug your USB weather station into the Raspberry Pi – USB cable into the OTG connector (probably via an adaptor lead) if using Raspberry Pi Zero W. |
|||
If you have an ethernet or WiFi linked weather station then you won't need to do this - I don't have one so I don't know exact details. Steve below says you need to enter the IP address during Cumulus setup, but then also adjust a disconnect period if you are also using Weatherlink software. |
|||
In a terminal session, issue commands in the following format for each file (this example relates to Raspberry Pi and uses "/var/tmp" which was defined in the extra line added to fstab earlier): |
|||
'''Running Cumulus''' |
|||
On PC, run PuTTY again and log in to the Pi as before (note you can save the IP address between sessions) |
|||
<pre>cd ~/CumulusMX |
|||
sudo mono CumulusMX.exe</pre> |
|||
<code>sudo ln -s /run/tmp/websitedata.json CHOSEN PATH/CumulusMX/web/websitedata.json</code> |
|||
The next thing you will want to do is access Cumulus via its '''user interface''' from your PC, so that you can update the '''settings'''. Using the IP address for your Pi, in your internet browser, enter: 192.168.y.z:8998 (where y and z are numbers you will need to find from seeing how your router connects to your Pi. You’ll first see a dashboard page, then can access the Settings menu. |
|||
Notes: |
|||
To make Cumulus run each time the Pi is rebooted (and force reboot in the early hours each day) |
|||
* The "-s" flag is what says you are creating a symbolic link |
|||
On the Pi, type: |
|||
* Full paths are given both for the file that MX is to be redirected to, and after it the path where it expects to create the file |
|||
<pre>sudo crontab -e</pre> |
|||
* "CHOSEN PATH" is defined in [[#Where to install all packages?]], but basically it starts with a "/" and defines the path to get to where "CumulusMX" is a sub-folder. |
|||
* The text "websitedata.json" is just one file in the set of files linked from [[:Category:JSON Files]]. |
|||
=Running MX= |
|||
On first run select the text editor you prefer (defaults to #1, nano, the easiest) |
|||
Then add the following lines at the end of the file: |
|||
<pre># Start Cumulus as background task 30s after reboot (delay to allow WiFi to startup) |
|||
@reboot (sleep 30;cd /home/pi/CumulusMX;sudo mono CumulusMX.exe) & |
|||
There are multiple subsections here, you are unlikely to need to read them all. Look at each, and decide if it applies to you. |
|||
# Reboot each day at 0253 |
|||
53 02 * * * sudo reboot</pre> |
|||
== Parameters == |
|||
'''To stop the Pi and restart it without CumulusMX running''' |
|||
(eg you need to do that if upgrading the CumulusMX version) type the following |
|||
<pre>sudo crontab -e</pre> |
|||
CumulusMX.exe can take a number of optional parameters as summarised here: |
|||
'''edit to put a # at the start of the line''' "@reboot..." |
|||
{| class="wikitable" border="1" |
|||
Ctrl-X to save the change to crontab and reboot using |
|||
!style="width:30px" | Parameter |
|||
<pre>sudo reboot</pre> |
|||
!style="width:600px" | Description |
|||
|- |
|||
! scope="row" | -port nnnn |
|||
| This parameter can be used whether MX is running interactively or as a service. Used to change the port where the web server for the MX interface runs, when Cumulus starts, it will display the URL of the interface where you change the settings, this is port 8998 by default. To use it when running MX in interactive mode, type <code>sudo mono CumulusMX.exe -port 9999</code> and the interface will run at port 9999 instead. |
|||
|- |
|||
! scope="row" | -service |
|||
| This parameter is not available when running interactively. It is used in a service definition file, please see [[#Running MX as a Linux "systemd" service]] for all details |
|||
|- |
|||
! scope="row" | -lang {locale} |
|||
| This parameter can be used whether MX is running interactively or as a service. Used to change the locale that MX will use from the default on your computer. To use it when running MX in interactive mode, type <code>sudo mono CumulusMX.exe -lang en_GB</code. There is a list of locale codes at http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. Remember this changes whether MX uses decimal comma or decimal point (although the intention is that all Cumulus files will use decimal points, this will still affect the output from api calls, web tags, etc.) and how it names files that include letetrs representing a month abbreviation. |
|||
|- |
|||
! scope="row" | -debug |
|||
| This parameter can be used whether MX is running interactively or as a service. This is only available for [https://cumulus.hosiene.co.uk/viewtopic.php?p=138839#p138839 release 3.4.4 - Build 3068] onwards. This switches on debug and (from 3.1.0) data logging from the start-up of Cumulus MX. Please note this increases size of files created in [[MXdiags_folder]]. As an alternative to using this parameter, you can switch debug and data logging on and off within the MX interface settings, see the aforementioned link for instructions. |
|||
|- |
|||
! scope="row" | -wsport |
|||
| Applied to MX beta 3.0.0, no longer available, set the port for web sockets, now incorporated into the -port parameter. |
|||
|- |
|||
! scope="row" | -logging |
|||
| Applied to MX beta 3.0.0, no longer available, enabled just data logging, now incorporated into the -debug parameter |
|||
|} |
|||
When your pi restarts, CumulusMX will no longer be running. You can then do your version upgrade or other task. |
|||
To revert to normal auto-running of CumulusMX, go through the same again, but this time edit crontab to remove the # from the start of the line "@reboot...". Save changes and reboot - CumulusMX will be running. |
|||
==Your first run of MX on Linux== |
|||
Updating a version of CumulusMX is easily done as follows using this: |
|||
1. Stop CumulusMX running (it locks files while it is running) |
|||
Once you have got all the files sorted out as described above, you need to run MX. |
|||
2. Install the updated CumulusMX version into a new directory - I call mine CumulusMX3xyz (where xyz are the last 3 digits of the build number) so that I can easily see which build it is |
|||
3. copy the following from the old CumulusMX directory to the new CumulusMX3xyz directory: |
|||
On the first run of MX, unless you have run a recent release before, you need to work through either the [[First_Run_of_MX|'''Config wizard''']] or all the individual settings pages (or both) as accessed from "Settings" menu. It is suggested you run MX interactively (see below) to do this, as you will then need to close MX, and then start it up again. |
|||
Information about settings is on other Wiki pages ([[MX Administrative Interface]] and [[Cumulus.ini]]). |
|||
== Run interactive or as a service== |
|||
MX can be run in two different ways. |
|||
It is advised that you run MX interactively to begin with, and only run it as a service when you are happy that all settings are correct, and that any uploading or other external tasks are working correctly. |
|||
Running interactively allows MX to display error messages to you, and to confirm when it is running normally. Just in case it is not obvious .... if you start any executable interactively in a terminal window on your Pi, you must leave that session running, or that executable will stop running |
|||
If you run MX as a service you do not get any direct feedback, and cannot see if there has been a problem or failure. Running as a "systemd" service was first made available at Patch release 3.8.4 - b3094 (14 September 2020). |
|||
==Running MX interactively== |
|||
To run MX interactively, you must open a terminal window, and leave it open until after you have closed MX. |
|||
The simplest instruction to run Cumulus MX is <code>cd CHOSEN PATH/CumulusMX && sudo mono CumulusMX.exe [optional parameters]</code>. |
|||
* This is two commands issued together, the first changes the working folder, the "&&" means that first command has to succeed before the second command is obeyed and actually starts the main executable |
|||
* This example has not included any optional parameters, as they are rarely needed, but the optional parameters available are as listed in table earlier. |
|||
When you want MX to stop, you must (for Linux) within your terminal session hold down the "Ctrl" button on your keyboard, and press "c". A word of caution here, if you are accessing your Linux computer over a network from another computer, do be careful about using any control sequences, as it is possible that your "Ctrl" "C" sequence will be applied to an application other than Cumulus MX, if that terminal session has started more than one application. The issue is that all running applications use the same terminator, it should be applied to whatever is regarded as the "foreground" application at the moment the control key sequence is used, which is guaranteed to be MX if that terminal session has only been used for running MX, and MX has not launched any external applications. After that you can choose to close the terminal window. |
|||
===Interactive advice=== |
|||
If you have followed advice at [[#Where to install all packages?]], the user you are using will own the "CHOSEN PATH/CumulusMX" folder and you ''may'' be able to omit the "sudo" befor the "mono". I say "may" because there are other reasons why you may need to run as root user, too technical to explain here. |
|||
You can start it off directly on your Pi, and then |
|||
*optionally disconnect the keyboard, |
|||
*switch off monitor or TV attached to your Pi, |
|||
*Just ensure you leave Pi on (with that window minimised) so that terminal session continues running. |
|||
===Running MX interactively from a remote computer=== |
|||
This is similar to running a terminal session on the machine that you installed MX on. If your remote computer is running Microsoft Windows, then the option to run a terminal session, may be called "terminal", "powershell window", "command window", or you might install software such as "PuTTY" to provide the teminal (TTY is the abbreviation for "teletype", a device that was commonly used to access computers in the 1970s and early 1980s). |
|||
These won't be explained any further here, but be aware that control key sequences may not work, and you may need to type "exit" to close the session. |
|||
Use <code>'''ps -ef | grep -i cumulus | grep -v grep'''</code> to see if Cumulus is running or not. |
|||
== Running another executable with a terminal session left open == |
|||
Open a terminal window, then navigate to the folder where you have installed the 3 Cumulus executables: |
|||
To run "Create Missing utility" type <code>cd CHOSEN PATH/CumulusMX && sudo mono CreateMissing.exe</code>. It does not take any parameters, so that is all you need to know, although it is fully documented at [[Calculate_Missing_Values#CreateMissing.exe]] in this Wiki. |
|||
To run [[Software#Export_To_MySQL|Export To My SQL]], you change the name of the executable above and add the necessary parameters, follow that link for more details. |
|||
==Running MX as a Linux "systemd" service== |
|||
There is a one-off task to define a service file, after that you can simply issue commands to stop/start/restart the service. |
|||
For more information, see [https://cumulus.hosiene.co.uk/viewtopic.php?p=146473#p146473 original release announcement]. |
|||
===The Service definition file=== |
|||
The MX download includes a file that can be used as a starting point for the service definition. Find this file at ''CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service''. You do have to edit this file, and then you have to copy it to a new location, before you start the MX service for the first time. |
|||
# Open the file at ''CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service'' using an editor (see [[Preparing_your_Linux_computer_for_MX#editing_files|editing_files]]). |
|||
#* On a Raspberry Pi with a graphical interface, use the file manager to navigate to this file, right click the file named, and select "Geany's Programmers Editor". |
|||
#* If you are accessing from another computer, using a terminal session, then "nano" is a suitable editor (explained at link just mentioned). |
|||
# Within the provided file you should find a [Service] section: |
|||
<pre>[Service] |
|||
User=root |
|||
Group=root |
|||
ExecStart=/usr/bin/mono-service -d:/home/install/CumulusMX CumulusMX.exe -service |
|||
Type=forking |
|||
ExecStopPost=/bin/rm -f /tmp/CumulusMX.exe.lock</pre> |
|||
*Be aware that what quoted above applies from MX 3.16.0 (b.3182, 30 Apr 2022) onwards, earlier releases did not include the "-f" flag in final line quoted above. |
|||
:There is more in the file, but for now focus on the line including "ExecStart=/usr/bin/mono-service -d:". Don't change any of the bit I just quoted. |
|||
Almost certainly you will need to change "/home/install/CumulusMX" on that line. Replace that with "CHOSEN PATH/CumulusMX", i.e. the full path to the directory that the executables are being stored in. |
|||
The final line, with all possible parameters, could read: <code>'ExecStart=/usr/bin/mono-service -d:CHOSEN PATH/CumulusMX CumulusMX.exe -service -debug -port 999 - lang el-GR</code> |
|||
* Note the space between the path (just looked at) and the executable file, |
|||
* Note the mandatory parameter "-service" that follows a space after the "CumulusMX.exe", you must leave that untouched, |
|||
* Note you can remove/keep the rest of the line after the -service i.e. the other parameters (some with their values) -lang, -port, or -debug, (as defined in table earlier), are all optional. |
|||
Technical user may do other edits on the file, these will be described later, for now save the changed file under a new name (so it won't be lost when you do a MX upgrade that replaces original file) within your MX installation: |
|||
# Open the "File" menu, and select "Save as" and enter a new name '''cumulusmx_edited.service''' |
|||
# Exit out of the editor you are using |
|||
# Next, open a terminal session |
|||
# Now copy file to where it is needed to run the service <code>sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_edited.service /etc/systemd/system/cumulusmx.service</code> |
|||
# Now type <code>sudo systemctl daemon-reload</code>, this tells "systemd" that it needs to reload all service definitions because either one has changed, or a new one has been added |
|||
# Finally, '''optionally''', create a symbolic link to that file using <code>sudo systemctl enable cumulusmx</code> if you want the service to automatically start after a reboot |
|||
====Technical users - additional edits==== |
|||
Novice users, skip this subsection. ''The changes in this subsection have to be made with other changes that are not covered here'' (they depend on your weather station type, and your computer type, so are not appropriate to a Wiki page trying to generalise, and anyway your contributor is not a technical expert). |
|||
:Look at the '''[Service]''' part of the file quoted above. |
|||
This states Cumulus should use root for both the user it runs under and for the group permissions it uses. ''If you have the technical expertise'', you might choose to run MX in a different user, if your weather station type allows MX to run in a different user. If so, replace the "root" in its two locations. (Please note some weather stations require other changes outside this file before Cumulus can make contact, one example is discussed [https://cumulus.hosiene.co.uk/viewtopic.php?t=20413 on support forum here], but there are other topics that may be relevant). |
|||
You may also wish to add an extra line after the "Group" line <code>ExecStartPre=/bin/sleep 5</code>, this [https://cumulus.hosiene.co.uk/viewtopic.php?p=163754#p163754 is to delay the starting of MX by 5 seconds while other services start] (on a reboot of your computer) that might affect MX. (For some users, change 5 into 10, it all depends what else is being started). |
|||
:Look at the rest of the file, the '''[Unit]''' part. |
|||
For releases 3.8.4 to 3.15.0: you will see one reference to '''network-online.target''' in <code>After=network-online.target</code>. |
|||
For release 3.15.1 build 3170 (19 March 2022) onwards: you will see an extra line <code>Wants=network-online.target</code> |
|||
If you are a technical user, you might decide to edit the [Unit] part of the file, you have to decide what is needed in your context, only you know what other services are started by systemd on your computer, you can list all items using something like <code>systemctl list-unit-files</code> to see the services, but you still need to understand what each does. |
|||
If your computer has online access, then it can look up the correct time online and adjust its clock. However, it might not even try to do that for say 10 minutes after being booted, and so there may be a benefit in making MX wait until after systemd has asked for the time to be synced, and asked that the local file-system is made ready so MX can read/update/store files. To achieve this, you might choose to add a blank line after '''<nowiki>Documentation=https://cumuluswiki.org/a/Main_Page</nowiki>''' and in that blank line, type <code>Requires= time-sync.target local-fs.target</code>. Using "Requires" ensures these requesting events have happened before MX can start, if they fail, MX will not be started, this example has not specified a time that MX should wait for the other services to start! |
|||
For that ''time-sync.target'' to work, you need to '''enable''', by creating the symbolic links needed, the appropriate services outside this edit: |
|||
<pre>sudo systemctl enable --now systemd-timesyncd.service |
|||
sudo systemctl enable --now systemd-time-wait-sync.service |
|||
</pre> |
|||
Here is a quick explanation of all entries in this UNIT section: |
|||
# Entries |
|||
#* The terminology "After" tells "systemd" that what is named can be started after MX, in this case it does not guarantee that the network service (to send data to a remote web server) will be started |
|||
#* The terminology "Wants" tells "systemd" that what is named is wanted now, i.e. try to start before MX, but still start MX even if the starting of the network service fails. |
|||
#* The terminology ''Requires'' tells "systemd" that the "cumulusmx" service should not be started until the services specified on that line have successfully started |
|||
#*# The '''local-fs.target''' specifies that the ''cumulusmx'' service requires the file service to have started, i.e. checks your computer can read files before it attempts to start the ''cumulusmx'' service |
|||
#*# The '''time-sync.target''' specifies that the ''cumulusmx'' service requires the computer to have synced with some time source (see notes below), which could be useful if your weather station type does not time-stamp readings stored in the console, and you want to ensure MX reads correct time from your computer |
|||
''Don't forget to save the file under a new name, and copy it as instructed in previous subsection.'' |
|||
- your CumulusMX/Cumulus.ini file |
|||
====Technical Notes only relevant to Raspberry Pi==== |
|||
- your CumulusMX/data directory |
|||
This Wiki page has tried to avoid being too specific to particular hardware, but to avoid misunderstanding the last subsection, a little does need to be said to justify the claim that only technical users, who understand all the other changes needed, should make changes mentioned there. |
|||
- your CumulusMX/twitter.txt file (if you have personalised it) |
|||
A standard Raspberry Pi computer does not include a clock chip. Instead one of the packages it loads as a service on booting is called "fake-hwclock", and that sets the clock to what the date/time was when it was last running, irrespective of how many days/hours it has been off. That counts as a time sync for the purposes of instruction specified above. You can buy and fit a real-time clock chip, and configure your computer to use that, but even that RTC will only keep time when it is kept powered, and even then it will drift off unless periodically able to be corrected by a time from internet. |
|||
- your CumulusMX/web directory (if you have personalised any web files) |
|||
The issue is your Cumulus MX on restarting will skip the catch-up of historic data (should your weather station settings make that available), because the dummy clock makes MX think the computer was not off for long. Subsequent measurements will then get logged against the wrong time until the correct time is found on the internet (NTP). At that moment, the time will suddenly jump, this is serious if this means the "rollover" time has been skipped over, as it implies the "dayfile.txt" will miss a line, and many measurements will be logged to wrong day. In my experience it can be anything from 2 minutes to 10 minutes after switch on before my RPi does a time sync over the internet. |
|||
4. Change your startup instruction to use the version in the new directory eg cd /home/pi/CumulusMX3050;sudo mono CumulusMX.exe |
|||
You might expect <code>sudo systemctl disable fake-hwclock.service</code> (or remove the service, and modify the scripts that call it) could ensure the computer (if online) has to get a time found on the internet (NTP). Nothing is as simple as it might seem! |
|||
With that method you can easily revert back to the old version if something has gone wrong. If all is well, you can delete the old directory after a few days/weeks/months/if you need the space. |
|||
===Commands to do actions on a service=== |
|||
'''Updating mono version''' |
|||
*First, stop CumulusMX as above by editing crontab. |
|||
*Then remove the present version of mono: |
|||
<pre>sudo apt-get purge libmono* cli-common mono-runtime |
|||
sudo apt-get autoremove</pre> |
|||
You will need to start (or restart) MX after you have defined (or redefined) the service as instructed above. The specific commands to use with MX service are at [[Raspberry_Pi_Image#systemctl_commands|systemctl_commands]], here I simply repeat the basic commands that can be used with any service (status, enable, disable, start, stop, and restart). |
|||
*Then install the new version |
|||
<pre>sudo apt-get install mono-complete</pre> |
|||
Don't forget you may need to type <code>sudo systemctl daemon-reload</code> to tell "systemd" that it needs to reload all service definitions whenever either one has changed, or a new one has been added. |
|||
*Finally re-enable auto running by editing crontab to remove the # and finally |
|||
<pre>sudo reboot</pre> |
|||
In all these commands, '''just replace [service_name] with ''cumulusmx''''' (or enter the name of another service). |
|||
Above Instructions: Last edited by ExperiMentor on Sun 01 Mar 2020 8:17 am, |
|||
* <code>sudo systemctl status [service_name]</code> |
|||
** (displays whether named service has started, whether it has failed, whether it has stopped, also whether enabled, extra information will be added should status change) |
|||
** type the single character "q" to quit updating status display and return to prompt |
|||
* <code>sudo systemctl enable [service_name]</code> |
|||
** (typed just once, and service named will automatically start when your Linux computer is booted) |
|||
** the confirmation message says a link has been created |
|||
* <code>sudo systemctl disable [service_name]</code> |
|||
** (used when you don't want an automatic restart of the named service) |
|||
* <code>sudo systemctl start [service_name]</code> |
|||
** (will start the named service) |
|||
* <code>sudo systemctl stop [service_name]</code> |
|||
** (will stop the named service) |
|||
** Closing MX with "cumulusmx" as the named service this way does a proper shutdown |
|||
* <code>sudo systemctl restart [service_name]</code> |
|||
** (issues a stop, then start, command to named service) |
|||
** You can upgrade MX by installing new files over the existing ones, while MX is left running, and then use this command to pick up new release with minimum downtime. |
Latest revision as of 08:38, 20 September 2023
Using MX on UNIX-derived Operating Systems
MX runs on any UNIX-derived operating systems (OS):
- including those found on Apple Mac computers,
- and those found on a multitude of devices running Linux.
UNIX is a long established operating system, and both UNIX and its derivatives have good long term compatibility. This means that commands are generally easy to learn just once and then you can normally continue to use what you have learnt.
Most devices also have a graphical user interface that can do the more straightforward tasks without needing to know all the commands.
Why install MX on Linux?
Contributions to the Cumulus Support Forum suggest that:
- Use on a Raspberry Pi (RPi) computer is very popular
- In general, people find installing, and running, MX on Linux is easy
- The few people who do have difficulties are those who have good knowledge of Microsoft systems and therefore are so convinced they cannot cope with a swap to something different, that they give up too easily!
Microsoft has had a deliberate policy of being different to traditional computers (all others are mostly based on UNIX).
You may know that this Wiki started with a single page covering MX regardless on which operating system was used, that did not work.
If you are running MX on on any computer running the Microsoft Windows Operating System, then you should be reading the MX on Windows OS page instead. In the Cumulus support forum, there are many posts from people who are struggling with MX on PCs. It appears this is not because Microsoft computers are so more readily available and therefore known about; but because people often find “installing”, and using, MX is more difficult when using the more complex Microsoft Windows operating system, and people tend not to understand basic issues such as avoiding "Program Files".
This page focuses on aspects of MX that are specific to the Linux operating systems.
Still believe it will be too complex for you? The developer has created an image you can download for those prepared to run two computers (a RPi for actually running MX and another computer for all interactions with MX). Read all about it, on Raspberry_Pi_Image page, and decide if that is for you.
Device Coverage
Linux is available based on a multitude of different kernels (the building block for the operating system), on a multitude of devices.
This page has been originated by a contributor using the Raspberry Pi Operating System (this is based on Debian, one of the Linux kernels). Be aware therefore that some instructions on this page are specific to a Raspberry Pi computer with its default operating system.
For other devices, the inclusion of the correct instructions is totally dependent on whether any contributor has edited this page to cover your device in the context of that section of this page. It is hoped that contributions to this page will be made by Cumulus users with a range of different devices so this page is useful to more people.
- Until somebody creates a separate page for Apple Mac computers (that could be a good idea, as there are some significant differences), this page is the best source of advice.
Further Information
There are various related pages to get more information:
- If you encounter a problem when running MX, please see What to do when I have a problem with MX
- If MX gives you a message saying "you are not running the latest version", please see Guide to upgrading MX
- If you are puzzled by the terminology, please see Category:Terminology for links to pages that explain terminology used by Cumulus (these pages were written for the legacy Cumulus 1 and may need updating for MX)
- If you need to know more about files in the installation, please see Category:Cumulus Files for links to all Wiki pages describing the sub-folders and files used by MX
- Go to Category:Cumulus MX for links to all pages in this Cumulus Wiki that relate specifically to MX
- Admin interface provides information on configuration and web pages for viewing your weather data locally
- The Cumulus MX FAQ page was created with snippets from the forum, but nobody has yet sorted this out into a useful page or updated it for recent releases
- If you were using the original (now legacy) Cumulus software, please read Migrating_from_Cumulus_1_to_MX, although that is mostly directed at those using MX on the same Windows PC as they used for Cumulus 1, and was written for an old MX release, it will help you understand configuration differences.
- If you want to use a script language, you might want to read PHP Hypertext Pre-processor and JavaScript page
- If you will be using the standard web pages (from release 3.10.1) see this page
- If you want to write your own customised templates, read Customised_templates.
- If you want to explore alternative web pages from third-parties, start on User Contributions page.
Preparing your computer for installing the Cumulus MX suite
Please see Preparing your Linux computer for MX page if you have not installed MX on Linux before.
That page covers:
- Installing Operating Systems
- Package manager
- Human Interface Devices
- Installing Mono
- Preparing Microsoft Windows files for Linux
- Guide to file names and paths
Please be advised some of the above is rather technical reading, but Mono is required to run the Cumulus packages described next. So do ensure that you installed Mono before continuing.
Technical aside
Please note this Wiki page talks about "folders" for compatibility with the MX on Windows OS page, but Linux prefers to call them directories.
Linux has a well defined filesystem, represented as a hierarchic tree starting at the root "/", that is divided into directories (one of which will be "/boot" and hold the kernel), each of those first level directories can be divided into second level directories, this second level is sometimes referenced to as defining the "scope", an indication that each is meant to be used for a specific purpose. The scope can be sub-divided again at lower levels representing "categories" (categories cover items like binary code, documentation, configuration, hardware, source code, runtime and content), and at a lower level still "applications" (i.e. related to specific programs) with further sub-levels for various options within those applications. Many Linux distributions will use logical links so references to a directory at one level in the hierarchy will actually redirect to files in a different directory, this might be because different programs expect to see files in different places or just to enforce ownership and writing rights.
For the purposes of this Wiki, the terminology "operating system" is used for the whole Linux distribution, you will find that Linux technical people prefer to talk about Linux distributions including:
- a "kernel" for the underlying handling of files, network and so on;
- one or more "shell" components for the handling of commands entered in terminal mode, including those that run programs (whether included in distribution or added later);
- an optional graphical user interface for simpler access to commands and programs.
For simplicity the terminology "terminal" is used for how you access the shell, this term refers to seeing the command prompt if your Linux is running without a graphical user interface, or to a window that you can open within the graphical interface where commands can be typed. Depending on your Linux, that window might be called "Terminal", "Konsole", "xterm", "gnome-terminal", "uxterm", or even something else. If you are accessing your Linux computer over a network from a computer running Microsoft Windows, then again you may encounter a number of terms for how to access the shell on your Linux computer, "Command Window", "Windows Powershell", or "Windows Terminal". Equally you may use software that calls it a teletype mode, e.g. PuTTY software.
Cumulus packages
- This section covers:
- CumulusMX.exe
- ExportToMySQL.exe
- CreateMissing.exe
(At time of writing this "CreateRecords.exe" is proposed, and under development, but not released).
Handling zip files
Each release is presented as a zip. It does not matter which device (if you have two or more computers), or which browser (it can be default browser for your device or the browser you like best) you use for the download. When your browser saves the zip it might be into a folder called “downloads” on your computer, or you may be able to save into another folder that you prefer (perhaps on a different partition). Your browser might even remember the folder you used before for files of type zip.
In general, any device will load a suitable application to use to unzip the package when you click on a filename that ends in .zip. You might need to do a "right click" and choose the application, it depends on your settings.
Be aware, you may need to adjust the settings within that application for how it handles the file structure. The preferences may determine whether the unzip process preserves the file structure used when the zip was created (i.e. each file remains in any sub-folder) or it ignores the folder structure. For the Cumulus context, it is essential to preserve the folder structure. You may also be asked where you want the files to be extracted to, or the default settings might always use a particular destination (and that might be a tmp folder).
For example on the Raspberry Pi operating system, there is a package called xarchiver, in its Graphical User Interface (GUI), there is a menu called "Action", and the final option in that menu is "Preferences". There, in "Archive" section, you can select "zip" as the preferred archive format (using a drop-down) and whether you want the application to confirm with you before deleting any files; in "Advanced" section, you can select the directory to use for the extraction. If you are using the lite version of the RPi OS, then you need to edit the /home/pi/.config/xarchiver/xarchiverrc file to set preferences, before you use the archiver package. Once you have started the archiver package, and told it which file to process, you can click on Extract files, the GUI presents a screen of options:
- "Extract to:", use the icon to browse to the required location if it has not been set up in preferences
- "Ensure a containing directory", tick this if it has not been set in the configuration file
- "Files", select "All files", the advice is to overwrite all of any existing files if you are upgrading, but you definitely need all files if this is a new install
- "Options"
- Tick "Extract files with full path", this is essential if you are going to successfully install any of the Cumulus software
- Tick "Overwrite existing files", the advice is to overwrite all of any existing files if you are upgrading, it may not always be clear which files have been updated since an earlier release, and there are a lot of interdependencies between different files
It is worth stressing here, if you decide to customise any files that are included in a release distribution, then you should at the very least add something like an "_" character to the file name to make your tailored file different to the standard file. The best practice is to put any files you tailor, or any additional files you create, outside the CumulusMX folder.
If you have chosen to do the download on a different device to that on which you will install, you can unzip on either device. To transfer either the downloaded .zip file, or the extracted file structure, between devices, you can use a file transfer package, or use a portable drive (a memory stick or even a memory card) with a partition formatted so that you can read it on both devices. Windows and Linux partitions are formatted in different ways. While it is likely that Linux can read a Microsoft formatted partition, Microsoft Windows can never read a Linux formatted partition.
Where to install all packages?
For simplicity on this page CHOSEN PATH (the contents of this will start with a slash “/”, but not end with a slash) is used to represent any location in the Linux file structure where you decide to install all the Cumulus packages.
The phrase “CHOSEN PATH” is used, because it is most likely you want to create the sub-folder called “/CumulusMX” (note where capital letters must be used) in a part of the Linux file structure that already exists.
It is important to minimise the length of the path name, because this path name has to be passed between various different software languages (and longer paths risk truncation).
Creating the CumulusMX sub-folder
- You can create sub-folder called “/CumulusMX” as you unzip a MX release, or you can type
sudo mkdir CHOSEN PATH/CumulusMX
first (note that CHOSEN PATH is explained above and always starts with a slash “/”). - By using the phrase CHOSEN PATH this advice avoids telling you to install Cumulus where you do not want it:
- Many people with a Raspberry Pi, and a little technical understanding, add an external drive to reduce wear on the internal micro-SD card, and keep their Cumulus files away from the drive that holds the operating system.
- This page is not going to get technical by telling you how to create, or mount, Linux partitions on your external drive. If your drive was bought from a Raspberry Pi reseller, they might help you.
- Other people using a Raspberry Pi without that technical expertise, might use ‘’’/home/pi’’’ for CHOSEN PATH as that is the default folder for the default user (Pi) and can be referenced as "~" in file path instructions they issue (although Cumulus will not understand that shorthand)
- Within that ‘’’/home/pi’’’ folder, the default user has full permissions automatically.
- The developer suggests you use ‘’’/opt’’’ for CHOSEN PATH (which should be available on any Linux computer).
- By default, the code Mark provides for installing Cumulus as a service, will run that service as a root user, and the root user has full permissions in /opt (and everywhere else)
- (Novices: Skip this step) If you do choose a CHOSEN PATH outside your home folder, then a more technical user can change the ownership of the "CumulusMX" sub-folder, to the default user (Pi) with
sudo chown -R pi: CHOSEN PATH/CumulusMX
, and reduce the need to use "sudo" on many actions.
- Many people with a Raspberry Pi, and a little technical understanding, add an external drive to reduce wear on the internal micro-SD card, and keep their Cumulus files away from the drive that holds the operating system.
Packages to install
We shall install the Cumulus software listed on Software page:
- CumulusMX:
- CumulusMX.exe is written in C#, that has been developed by Mark Crossley, but still contains some code by Steve Loft
- Download CumulusMX zip file’’’ from the link at Software#Latest_build_distribution_download
- Create Missing:
- CreateMissing.exe is also written in C#, created, and developed, by Mark Crossley
- Download Create Missing zip file from the link at Software#Create_Missing
- This takes you to a github page with a "ReadMe" providing minimal instructions
- Using CreateMissing.exe is fully documented at Calculate_Missing_Values#CreateMissing.exe in this Wiki
- (it will populate missing fields in standard log files and/or missing lines in dayfile.txt).
- ExportToMySQL
- ExportToMySQL.exe is also written in C# by Mark Crossley
- Download Export To My SQL zip file from the link at Software#ExportToMySQL
- This takes you to a github page with a "ReadMe" providing minimal instructions
- ExportToMySQL.exe is not (at the time this was written) documented in this Wiki although MX_Administrative_Interface#MySQL_settings does describe a similar utility (written by Steve Loft) that was actually included within early CumulusMX zip downloads.
As at 9 March 2020, another utility, CreateRecord, has been initialised in the Github areas managed by the developer where Cumulus is worked on, although it appears to be just a concept on github. This will, if my understanding is correct, read dayfile.txt and use that to update the various extreme record files. The developer is still aiming to make this available, but his work on it (on his computer) has been stalled by the level of pressure being applied for bug-fixes and changes to MX itself.
Alternative download link for older package releases
Because the developer uses Git Hub to manage releases, the older releases remain available.
Old Cumulus MX packages
Skip this subsection if either you have installed the "pre-built disc image", or the current MX release is stable (it has been available for a while and nobody has reported any bugs).
Check if posts in the Cumulus Support Forum tell you that the current release of MX has one or more bug(s) that affects one or more aspect(s) of MX that you intend to use.
Remember, it is impossible for the developer to check all the ways in which versatile MX can be used:
- Different weather station types (the developer only has a Davis),
- Different computer types (development is mostly on Microsoft Windows),
- Plus a whole host of optional features, and different external upload sites, (typically each of these optional features are only used by a sub-set of Cumulus users).
Anyway, you can download any earlier MX build, without the bug, from CumulusMX/releases.
Old utilities
The zips for "CreateMissing.exe" and "ExportToMySQL.exe" utilities do NOT contain the .dll components that they need when they are running.
This means that each version of "CreateMissing.exe" and "ExportToMySQL.exe" utilities is dependent on it being used with a release of Cumulus MX that does have correct .dll components in its release distribution.
That in turn means you can't use the latest version of the utility with older MX releases, nor can you use an old utility version with latest MX release. This is why utilities downloads make clear which MX release is the minimum for them.
The older versions of these "CreateMissing.exe" and "ExportToMySQL.exe" utilities are available by going directly to the Cumulus MX github page, and navigating to the utility of interest. However, to use those older versions, you also need to download the corresponding older MX release, because the MX distributions contain the .dll files that the utilities require, they are not in the utilities zip. Because of this complication, novice users are advised not to attempt to use the older utilities, even if the latest version appears to have a bug. Technical users may be able to work out which .dll files are needed and can be safely added back (if they are not left over from when that past MX release was in use). An alternative is to create a new folder with the old release packages (MX and the utility of interest), a copy of the latest Cumulus.ini file, and a copy of all files from /data sub-folder; then afterwards copy back the changed files into original /data folder.
Upgrading a Cumulus package
Always check the release announcements in Cumulus MX announcements for any action needed in planned upgrade. In brief, all files from new release distribution replace the files from previous release, and the download/unzip is as covered above.
No further action needed for upgrade of "Create Missing" or "Export To My SQL" or "Create Records". See below for upgrade of main Cumulus MX package.
If you are running an older MX release, before skipping in-between versions please check here.
If you run MX as a service, then:
- Ensure you are not doing any changes to settings
- Leave MX running as you copy files from the new release distribution over the existing files
- Try to pick a time just after MX has done any standard interval store of readings or upload, so that it is least busy
- Use
sudo systemctl restart cumulusmx
to stop and then restart MX picking up the new files - Result, downtime of MX kept to a minimum, so avoiding losing data
If you run MX interactively, do the unzip into a temporary location before you stop MX, then copy all files from temporary location over your existing files, and finally restart MX. Depending on your weather station type, it might or might not offer historic data catch-up, so you might lose some data while MX is stopped, and therefore should keep the downtime to a minimum.
The alternative is to install in a new folder (or rename the old one), and copy across files not in the release from old location to new location, but in that alternative you might forget some files.
Changing location of Cumulus MX
If your install, or upgrade, is creating MX in a different place to where you previously ran Cumulus, then you will want to copy files across that are not in the zip extract distribution.
Configuration Files to copy across from any previous Cumulus installation
There are two configuration files that are not included in any MX release:
- strings.ini (note all lower case) – optional file to customise output
- Cumulus.ini (note initial capital, then lower case) – main configuration file
Here, it must be stressed that having either or both of these files in an existing Cumulus installation does not imply such file or files can be understood by the new MX release you have just installed.
Just copy the existing files from old to new installation, if
- Your locale is still the same
- All files on your new install are in same paths as on your old install (some settings involve specifying paths)
- Your old installation has a relatively recent MX release (compare the "y" in 3.y.z,between old and new installation, a difference of more than 1 in that middle figure means you do not have a recent release)
- Your old installation was on a Unix-based computer (not a computer running Microsoft Windows Operating System)
Please see the table below for more advice, but the problem is that content of both files has changed as MX has been developed, so some content is no longer understood, and some new content has been added.
Some of the differences between versions of Cumulus.ini file can be seen by comparing the different pages in this Wiki documenting this file: Cumulus.ini (Beta), Cumulus.ini (Cumulus 1), Cumulus.ini (MX 3.0.0 to 3.7.0), Cumulus.ini (preserving history), and Cumulus.ini, but even that does not tell the whole story. MX release 3.12.0 needs to be installed if your old Cumulus was earlier than that, because it is the only release with code to rename the old "Cumulus.ini" and create a new file containing the new set of settings, and new names for some old settings, but without any old settings that are no longer recognised. Please see Updating MX to new version page for more information about the need to step slowly through from old releases to the newest.
If you are upgrading from an older release, please read the table for advice.
Cumulus.ini | strings.ini |
---|---|
Your old installation will have this file. In general, if your old installation was any release before 3.8.0, the advice is give the old file a different name when you copy it across to the new installation, and let MX create the file as you work through all the settings. | This is an optional file. Its purpose is to allow customisation of some of the outputs from Cumulus. You might want to use customisation to abbreviate (or extend) some outputs, or to change those outputs into another language. |
When you work through the Settings pages, MX will create this file if it does not exist.
|
You create a “strings.ini” file by selecting some of the parameters from the Samplestring.ini file that is included in each MX release, and modifying the value for the listed attributes as you type just those you selected (under the same group titles - these are enclosed in [ ] as before).
The sections that appear in samplestring.ini, and the parameters that appear within a section, depend upon which release you are using. So be cautious if you try to reuse a "strings.ini" file originally created by the legacy software, you may find you need to specify your customisation using different parameters in the latest "samplestring.ini". |
The content of "Cumulus.ini" is changing as MX is developed, the Release Announcements normally list any new parameters as they appear in the file, without always mentioning those that have become redundant. The announcements tend to avoid any detail, so you have to guess from the attribute what values it might take, and generally have no idea of where in the settings pages to make any change.
To remove any parameters no longer used in this file, see remove redundant parameters If your old file contains any legacy read-only parameters not yet converted into advanced settings, or any MX read-only parameters not yet converted into advanced settings, you may need to manually add such missing parameters back into new file by stopping MX (after finishing all the settings you can configure in the interface), doing an external file edit, and then restarting MX. |
The content of "samplestring.ini" is changing as MX is developed:
|
If you previously used an older release of Cumulus, but in this new installation will be using the latest release (latest is what is normally best, unless it has bugs), you may want to read up on all the changes between your old release and the current release, not just changes that affect the configuration file.
If you have used Cumulus 1 before, and decide to start with a new "Cumulus.ini" file, then you will need to work through all settings, to ensure they are set as you want. Please remember that when you use MX for first time, it uses that date it was first run as a starting date, and ignores any data found with earlier dates. Therefore you must change that start date: MX interface --> Setting menu --> Station Settings --> Click on General Settings --> Click on Advanced Options --> Edit Records Began Date following instructions below that field
"data" directory
Please see Category:Files_with_Comma_Separated_Values, Category:Ini_Files, and Weather_Diary, for information if you are moving from Cumulus 1 to MX. Otherwise just copy files from any existing folder to your new one.
You may also wish to read:
- Amending dayfile tells you about how MX is far more fussy about the content in dayfile.txt
- .ini files explains how time-stamps are formatted differently in the extreme tracking files
- Migrating from Cumulus 1 to MX gives some advice about differences in settings, but be aware that the way MX handles settings varies by release, and information on the linked page may be out of date
Complications occur if the locale used by Mono or the locale specified when starting MX using -lang parameter differs from the locale for your previous device (please see Category:Files with Comma Separated Values because some locales separate fields with commas, some separate integer and decimal parts of real numbers with commas; not to mention all sorts of issues with how dates are formatted). The main MX developer proposed that the format of files with comma separated values will be fixed from a release planned for September 2020, so all dates will use one standard format, all numbers will use decimal points, and the field separating character will be fixed.
Update May 2022, this has been put on hold, no public MX release has this restriction yet.
"Reports" directory
By default MX now creates monthly and annual reports that are in the style used by NOAA in USA. If you have been using this functionality before (and it is optional) then you need to file transfer, or copy, all the files that were in the old Reports folder into the new folder of that name. Do look at that cross-reference, and read about the encoding default differences between Cumulus 1 and MX.
MX can cause problems with storage
MX now assumes by default that you are going to use its Default Web Site. That means that by default MX will re-generate temporary files in its /web sub-folder on a frequent time-scale. That number of files writes will considerably shorten the working life-time of the "high capacity micro-SD" card that is the default storage for the Raspberry Pi. It will also considerably shorten the life of any flash memory (e.g. memory card) that you might install MX on.
The expected life of any storage device, and the extent to which its life is shortened depends on the actual device. The external devices that have the longest life (and therefore can cope most easily with multiple read/write actions) are solid state discs (SSD). Also the larger the capacity of the storage device, the more places on the device where files can be stored and the storing algorithm will try to spread the storing evenly across the entire storage area, so wear at any one location is reduced.
All Linux computers will have some random access memory chips (RAM) and it is worthwhile to define part of that RAM as a drive used for temporary files. For a Raspberry Pi computer, a typical approach would be to edit the fstab file, adding the line tmpfs /run/tmp tmpfs nodev,nosuid,size=1M 0 0, but the size you choose will depend on RAM available and what temporary files are being created. For maximum life of the "high capacity micro-SD" card if that is what your computer boots from, you should create a symbolic link path that maps the /tmp folder used by the system to your /run/tmp you have just defined in RAM. The difficulty will be that you cannot create a logical redirect on /tmp if the folder is already in use, so that makes it too complicated to explain here.
web directory
All the files in this folder come from the download.
However, when you are running MX, it may try to create temporary files here, and following the advice above, you may decide to set up symbolic links so any attempt to create a temporary file in the "web" folder is redirected to the temporary folder you set up in RAM.
The links you need depend on which options you select in settings, you might find it easier to wait until you have run MX for a while to see what files are created that end in ".json".
If MX is currently running, you need to stop it, or at least alter any options that generate .json files. Then you must delete those files that end in ".json", except that you don't delete "websitedataT.json".
In a terminal session, issue commands in the following format for each file (this example relates to Raspberry Pi and uses "/var/tmp" which was defined in the extra line added to fstab earlier):
sudo ln -s /run/tmp/websitedata.json CHOSEN PATH/CumulusMX/web/websitedata.json
Notes:
- The "-s" flag is what says you are creating a symbolic link
- Full paths are given both for the file that MX is to be redirected to, and after it the path where it expects to create the file
- "CHOSEN PATH" is defined in #Where to install all packages?, but basically it starts with a "/" and defines the path to get to where "CumulusMX" is a sub-folder.
- The text "websitedata.json" is just one file in the set of files linked from Category:JSON Files.
Running MX
There are multiple subsections here, you are unlikely to need to read them all. Look at each, and decide if it applies to you.
Parameters
CumulusMX.exe can take a number of optional parameters as summarised here:
Parameter | Description |
---|---|
-port nnnn | This parameter can be used whether MX is running interactively or as a service. Used to change the port where the web server for the MX interface runs, when Cumulus starts, it will display the URL of the interface where you change the settings, this is port 8998 by default. To use it when running MX in interactive mode, type sudo mono CumulusMX.exe -port 9999 and the interface will run at port 9999 instead.
|
-service | This parameter is not available when running interactively. It is used in a service definition file, please see #Running MX as a Linux "systemd" service for all details |
-lang {locale} | This parameter can be used whether MX is running interactively or as a service. Used to change the locale that MX will use from the default on your computer. To use it when running MX in interactive mode, type sudo mono CumulusMX.exe -lang en_GB</code. There is a list of locale codes at http://msdn.microsoft.com/en-gb/library/ee825488%28v=cs.20%29.aspx. Remember this changes whether MX uses decimal comma or decimal point (although the intention is that all Cumulus files will use decimal points, this will still affect the output from api calls, web tags, etc.) and how it names files that include letetrs representing a month abbreviation.
|
-debug | This parameter can be used whether MX is running interactively or as a service. This is only available for release 3.4.4 - Build 3068 onwards. This switches on debug and (from 3.1.0) data logging from the start-up of Cumulus MX. Please note this increases size of files created in MXdiags_folder. As an alternative to using this parameter, you can switch debug and data logging on and off within the MX interface settings, see the aforementioned link for instructions. |
-wsport | Applied to MX beta 3.0.0, no longer available, set the port for web sockets, now incorporated into the -port parameter. |
-logging | Applied to MX beta 3.0.0, no longer available, enabled just data logging, now incorporated into the -debug parameter |
Your first run of MX on Linux
Once you have got all the files sorted out as described above, you need to run MX.
On the first run of MX, unless you have run a recent release before, you need to work through either the Config wizard or all the individual settings pages (or both) as accessed from "Settings" menu. It is suggested you run MX interactively (see below) to do this, as you will then need to close MX, and then start it up again.
Information about settings is on other Wiki pages (MX Administrative Interface and Cumulus.ini).
Run interactive or as a service
MX can be run in two different ways.
It is advised that you run MX interactively to begin with, and only run it as a service when you are happy that all settings are correct, and that any uploading or other external tasks are working correctly.
Running interactively allows MX to display error messages to you, and to confirm when it is running normally. Just in case it is not obvious .... if you start any executable interactively in a terminal window on your Pi, you must leave that session running, or that executable will stop running
If you run MX as a service you do not get any direct feedback, and cannot see if there has been a problem or failure. Running as a "systemd" service was first made available at Patch release 3.8.4 - b3094 (14 September 2020).
Running MX interactively
To run MX interactively, you must open a terminal window, and leave it open until after you have closed MX.
The simplest instruction to run Cumulus MX is cd CHOSEN PATH/CumulusMX && sudo mono CumulusMX.exe [optional parameters]
.
- This is two commands issued together, the first changes the working folder, the "&&" means that first command has to succeed before the second command is obeyed and actually starts the main executable
- This example has not included any optional parameters, as they are rarely needed, but the optional parameters available are as listed in table earlier.
When you want MX to stop, you must (for Linux) within your terminal session hold down the "Ctrl" button on your keyboard, and press "c". A word of caution here, if you are accessing your Linux computer over a network from another computer, do be careful about using any control sequences, as it is possible that your "Ctrl" "C" sequence will be applied to an application other than Cumulus MX, if that terminal session has started more than one application. The issue is that all running applications use the same terminator, it should be applied to whatever is regarded as the "foreground" application at the moment the control key sequence is used, which is guaranteed to be MX if that terminal session has only been used for running MX, and MX has not launched any external applications. After that you can choose to close the terminal window.
Interactive advice
If you have followed advice at #Where to install all packages?, the user you are using will own the "CHOSEN PATH/CumulusMX" folder and you may be able to omit the "sudo" befor the "mono". I say "may" because there are other reasons why you may need to run as root user, too technical to explain here.
You can start it off directly on your Pi, and then
- optionally disconnect the keyboard,
- switch off monitor or TV attached to your Pi,
- Just ensure you leave Pi on (with that window minimised) so that terminal session continues running.
Running MX interactively from a remote computer
This is similar to running a terminal session on the machine that you installed MX on. If your remote computer is running Microsoft Windows, then the option to run a terminal session, may be called "terminal", "powershell window", "command window", or you might install software such as "PuTTY" to provide the teminal (TTY is the abbreviation for "teletype", a device that was commonly used to access computers in the 1970s and early 1980s).
These won't be explained any further here, but be aware that control key sequences may not work, and you may need to type "exit" to close the session.
Use ps -ef | grep -i cumulus | grep -v grep
to see if Cumulus is running or not.
Running another executable with a terminal session left open
Open a terminal window, then navigate to the folder where you have installed the 3 Cumulus executables:
To run "Create Missing utility" type cd CHOSEN PATH/CumulusMX && sudo mono CreateMissing.exe
. It does not take any parameters, so that is all you need to know, although it is fully documented at Calculate_Missing_Values#CreateMissing.exe in this Wiki.
To run Export To My SQL, you change the name of the executable above and add the necessary parameters, follow that link for more details.
Running MX as a Linux "systemd" service
There is a one-off task to define a service file, after that you can simply issue commands to stop/start/restart the service.
For more information, see original release announcement.
The Service definition file
The MX download includes a file that can be used as a starting point for the service definition. Find this file at CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service. You do have to edit this file, and then you have to copy it to a new location, before you start the MX service for the first time.
- Open the file at CHOSEN PATH/CumulusMX/MXutils/linux/cumulusmx.service using an editor (see editing_files).
- On a Raspberry Pi with a graphical interface, use the file manager to navigate to this file, right click the file named, and select "Geany's Programmers Editor".
- If you are accessing from another computer, using a terminal session, then "nano" is a suitable editor (explained at link just mentioned).
- Within the provided file you should find a [Service] section:
[Service] User=root Group=root ExecStart=/usr/bin/mono-service -d:/home/install/CumulusMX CumulusMX.exe -service Type=forking ExecStopPost=/bin/rm -f /tmp/CumulusMX.exe.lock
- Be aware that what quoted above applies from MX 3.16.0 (b.3182, 30 Apr 2022) onwards, earlier releases did not include the "-f" flag in final line quoted above.
- There is more in the file, but for now focus on the line including "ExecStart=/usr/bin/mono-service -d:". Don't change any of the bit I just quoted.
Almost certainly you will need to change "/home/install/CumulusMX" on that line. Replace that with "CHOSEN PATH/CumulusMX", i.e. the full path to the directory that the executables are being stored in.
The final line, with all possible parameters, could read: 'ExecStart=/usr/bin/mono-service -d:CHOSEN PATH/CumulusMX CumulusMX.exe -service -debug -port 999 - lang el-GR
- Note the space between the path (just looked at) and the executable file,
- Note the mandatory parameter "-service" that follows a space after the "CumulusMX.exe", you must leave that untouched,
- Note you can remove/keep the rest of the line after the -service i.e. the other parameters (some with their values) -lang, -port, or -debug, (as defined in table earlier), are all optional.
Technical user may do other edits on the file, these will be described later, for now save the changed file under a new name (so it won't be lost when you do a MX upgrade that replaces original file) within your MX installation:
- Open the "File" menu, and select "Save as" and enter a new name cumulusmx_edited.service
- Exit out of the editor you are using
- Next, open a terminal session
- Now copy file to where it is needed to run the service
sudo cp EXISTING_PATH/CumulusMX/MXutils/linux/cumulusmx_edited.service /etc/systemd/system/cumulusmx.service
- Now type
sudo systemctl daemon-reload
, this tells "systemd" that it needs to reload all service definitions because either one has changed, or a new one has been added - Finally, optionally, create a symbolic link to that file using
sudo systemctl enable cumulusmx
if you want the service to automatically start after a reboot
Technical users - additional edits
Novice users, skip this subsection. The changes in this subsection have to be made with other changes that are not covered here (they depend on your weather station type, and your computer type, so are not appropriate to a Wiki page trying to generalise, and anyway your contributor is not a technical expert).
- Look at the [Service] part of the file quoted above.
This states Cumulus should use root for both the user it runs under and for the group permissions it uses. If you have the technical expertise, you might choose to run MX in a different user, if your weather station type allows MX to run in a different user. If so, replace the "root" in its two locations. (Please note some weather stations require other changes outside this file before Cumulus can make contact, one example is discussed on support forum here, but there are other topics that may be relevant).
You may also wish to add an extra line after the "Group" line ExecStartPre=/bin/sleep 5
, this is to delay the starting of MX by 5 seconds while other services start (on a reboot of your computer) that might affect MX. (For some users, change 5 into 10, it all depends what else is being started).
- Look at the rest of the file, the [Unit] part.
For releases 3.8.4 to 3.15.0: you will see one reference to network-online.target in After=network-online.target
.
For release 3.15.1 build 3170 (19 March 2022) onwards: you will see an extra line Wants=network-online.target
If you are a technical user, you might decide to edit the [Unit] part of the file, you have to decide what is needed in your context, only you know what other services are started by systemd on your computer, you can list all items using something like systemctl list-unit-files
to see the services, but you still need to understand what each does.
If your computer has online access, then it can look up the correct time online and adjust its clock. However, it might not even try to do that for say 10 minutes after being booted, and so there may be a benefit in making MX wait until after systemd has asked for the time to be synced, and asked that the local file-system is made ready so MX can read/update/store files. To achieve this, you might choose to add a blank line after Documentation=https://cumuluswiki.org/a/Main_Page and in that blank line, type Requires= time-sync.target local-fs.target
. Using "Requires" ensures these requesting events have happened before MX can start, if they fail, MX will not be started, this example has not specified a time that MX should wait for the other services to start!
For that time-sync.target to work, you need to enable, by creating the symbolic links needed, the appropriate services outside this edit:
sudo systemctl enable --now systemd-timesyncd.service sudo systemctl enable --now systemd-time-wait-sync.service
Here is a quick explanation of all entries in this UNIT section:
- Entries
- The terminology "After" tells "systemd" that what is named can be started after MX, in this case it does not guarantee that the network service (to send data to a remote web server) will be started
- The terminology "Wants" tells "systemd" that what is named is wanted now, i.e. try to start before MX, but still start MX even if the starting of the network service fails.
- The terminology Requires tells "systemd" that the "cumulusmx" service should not be started until the services specified on that line have successfully started
- The local-fs.target specifies that the cumulusmx service requires the file service to have started, i.e. checks your computer can read files before it attempts to start the cumulusmx service
- The time-sync.target specifies that the cumulusmx service requires the computer to have synced with some time source (see notes below), which could be useful if your weather station type does not time-stamp readings stored in the console, and you want to ensure MX reads correct time from your computer
Don't forget to save the file under a new name, and copy it as instructed in previous subsection.
Technical Notes only relevant to Raspberry Pi
This Wiki page has tried to avoid being too specific to particular hardware, but to avoid misunderstanding the last subsection, a little does need to be said to justify the claim that only technical users, who understand all the other changes needed, should make changes mentioned there.
A standard Raspberry Pi computer does not include a clock chip. Instead one of the packages it loads as a service on booting is called "fake-hwclock", and that sets the clock to what the date/time was when it was last running, irrespective of how many days/hours it has been off. That counts as a time sync for the purposes of instruction specified above. You can buy and fit a real-time clock chip, and configure your computer to use that, but even that RTC will only keep time when it is kept powered, and even then it will drift off unless periodically able to be corrected by a time from internet.
The issue is your Cumulus MX on restarting will skip the catch-up of historic data (should your weather station settings make that available), because the dummy clock makes MX think the computer was not off for long. Subsequent measurements will then get logged against the wrong time until the correct time is found on the internet (NTP). At that moment, the time will suddenly jump, this is serious if this means the "rollover" time has been skipped over, as it implies the "dayfile.txt" will miss a line, and many measurements will be logged to wrong day. In my experience it can be anything from 2 minutes to 10 minutes after switch on before my RPi does a time sync over the internet.
You might expect sudo systemctl disable fake-hwclock.service
(or remove the service, and modify the scripts that call it) could ensure the computer (if online) has to get a time found on the internet (NTP). Nothing is as simple as it might seem!
Commands to do actions on a service
You will need to start (or restart) MX after you have defined (or redefined) the service as instructed above. The specific commands to use with MX service are at systemctl_commands, here I simply repeat the basic commands that can be used with any service (status, enable, disable, start, stop, and restart).
Don't forget you may need to type sudo systemctl daemon-reload
to tell "systemd" that it needs to reload all service definitions whenever either one has changed, or a new one has been added.
In all these commands, just replace [service_name] with cumulusmx (or enter the name of another service).
sudo systemctl status [service_name]
- (displays whether named service has started, whether it has failed, whether it has stopped, also whether enabled, extra information will be added should status change)
- type the single character "q" to quit updating status display and return to prompt
sudo systemctl enable [service_name]
- (typed just once, and service named will automatically start when your Linux computer is booted)
- the confirmation message says a link has been created
sudo systemctl disable [service_name]
- (used when you don't want an automatic restart of the named service)
sudo systemctl start [service_name]
- (will start the named service)
sudo systemctl stop [service_name]
- (will stop the named service)
- Closing MX with "cumulusmx" as the named service this way does a proper shutdown
sudo systemctl restart [service_name]
- (issues a stop, then start, command to named service)
- You can upgrade MX by installing new files over the existing ones, while MX is left running, and then use this command to pick up new release with minimum downtime.