MX on Linux

Using MX on UNIX-derived Operating Systems

MX runs on various UNIX-derived operating systems (OS), including those found on Apple Mac computers, and those found on devices running Linux.

Introduction

This article focuses on a device called the Raspberry Pi. It is a small board of electronics that can actually run Windows, Mac OS, Chrome OS, various Linux distributions, or the Raspbian OS (based on Ubantu Linux). In this article, the focus is on the last OS, because that is easiest to install, so look elsewhere on the web for details of installing alternatives.

The article will give you some guidance on:

• Choosing a Pi model to buy
• Setting it up,
  • either connecting it to a keyboard, mouse, and monitor (could be TV), and internet (wired Ethernet or wireless)
  • or running it headless, where all instructions are typed into another device and the Pi has no keyboard, mouse, or monitor, but is connected to internet (wired Ethernet or wireless)
• Installing OS
• Installing Mono
• Installing Apache, PHP, MariaDB, PhpMyAdmin, and copying your database from another device
• Running MX

Which Raspberry Pi to buy

First make a list of what you need

• Do you want to use a mouse and key board?
  • If so, a model with multiple USB sockets is advisable (like 3B+)
  • This also applies if you want to be able to plug in a USB stick (perhaps for transferring files between devices, e.g. Cumulus configuration and data folder files)
• Do you want a wired connection to your hub or router?
  • Maybe you are going to update external sites, a wired connection may provide a faster and more dependable communication than a wireless link
  • If so, a model including an Ethernet socket is advisable (like 0W or 2B+)
  • Remember that if you are operating the pi in headless mode, a wired or wireless connection to your LAN is needed for your other device to communicate with the Pi
• Will your MX need to update a database, feed data to a web site, upload to external sites, or control other devices?
  • If so, a model Zero will have to do each task in turn, and you will see some delay in information updates, plus you will need to use a larger time between updates
  • If so, a model Three (or Four) will be noticeably faster, and permit you to select more options in MX, and to update external sites more frequently
• What interface does your weather station use?
  • If your station communicates to MX via wireless, then you choose a model that supports wireless at the right frequency
  • If your station communicates to MX via USB, then choose between the model 0W with one USB socket, and the model 3B+ with 4 USB sockets
  • If your station uses another communication port (such as serial interface), then you need the components that support that interface

Now research how the various models relate to your needs

You can look up online what features are included in the various Pi models, and how they differ in power consumption, and cost. But your decision also needs to consider what you need. Here, I won't describe all the different models, but concentrate on just 2 for simplicity. The model Zero W is appealing as it has low power consumption, but has limited speed, it runs the various threads MX uses sequentially, and has limited interfaces for peripherals. The model Three B plus is appealing as it has medium power consumption, but can cope better with the multiple threads that MX starts, and has more interfaces built in, such as 4 USB 2 ports. Obviously, the model Four has more capability, but is less appealing as it also consumes more power.

• Raspberry Pi Zero W
  • Pi Zero W has WiFi and one micro-USB port which is all that is needed for headless running.
  • Using a faster Pi might speed parts of the installation process, but are overkill for actual ‘production’ running.
  • It can run a web server, but that will slow it down.
• Raspberry Pi Three B Plus
  • This faster Pi is NOT needed for running Cumulus MX, but it will cope better if you are asking MX to do lots of processing.
  • Pi Three B Plus has a socket for an external power supply, Ethernet socket (supports wired link); a HDMI socket for audio/video to TV, or computer monitor; a standard audio socket for external microphone, or speakers; 4 standard USB 2 sockets for weather station, mouse, keyboard, USB stick, or other storage device; plus other connections (e.g. camera).
  • This might be better if you also want to run a web server, or are on a remote site so you do want to be able to plug in a monitor and other peripherals.

What else to buy

This obviously depends on your PI model and on your weather station connection type.

You may want to buy a case. A case specifically designed for your Pi model will have cut-outs in the right place for each interface connection and will have sufficient ventilation for the electrical components to not over-heat.

You may need a power supply. This could be an official Raspberry Pi power supply. Alternatively, any power supply unit that has a micro USB connector will do, the power consumption of a Pi (whichever model) is fairly small, but it will be powered on 24/7, so a low power consumption ‘switched mode’ type is preferred – i.e. one that does not become warm when plugged in with nothing attached. You may have a suitable one from a mobile phone.

You may need to buy connection leads. You may need a HDMI lead to connect your PI to your TV or a spare computer monitor. You may need a USB lead to connect to your weather station (the model Zero requires a micro USB, the model 3 requires a standard A end USB) which probably has a USB A end connection. If your weather station connects by Ethernet, you will need one lead to connect the station to the router and possibly another to connect the hub or router to the Pi.

Setting up your Pi

You can find, online, instructions about setting up a Pi. The obvious place to look is the manufacturer's web site, or the same documentation on their github page. However, you might look at a supplier's web site, or various other sources of "how to" advice.

The Micro-SD card

A Pi requires either a class 4, or a class 10, micro-SD card (or whatever goes with your Pi model) with a minimum of 8 GB. Various suppliers offer cards of 16 to 64 GB with NOOBS pre-installed ready for use in a Pi. You may even buy a kit that includes a Pi board with components and interfaces on it, a power supply, a micro-SD card, some connection leads, and a case that you can fit the board into to protect it.

Setting up a Pi is simple assuming you have bought a micro-SD card that is, either pre-installed with NOOBS, or with the Raspbian Pi operating system pre-installed. NOOBS makes it easy to install (by default) Rasbian (insert the SD card into the Pi and when you power up the Pi, the operating system will be installed during that first boot. If you have a model with an Ethernet connection, and that is connected first, then NOOBS can offer a choice of other operating systems, or you can download another compatible operating system and install it on the micro-SD card.

If you want to be able to use the micro-SD card in other devices (like your PC), you may need to also buy an adapter which allows the micro-SD card to be plugged into a standard SD socket. This may be included if you buy a micro-SD card not sold specifically for the Pi.

The Operating System

As said before, a number of operating systems can be installed on your Pi, even Windows. I leave it to you to read, on-line, how to install the operating system you choose, normally the instructions will be found from where you download the system you have selected.

If you have a micro-SD card pre-installed with an operating system, you are ready to go!

If you have a micro-SD card pre-installed with NOOBS, as described above, the first boot will install the operating system. By default, it will be the latest Raspbian OS when the card was manufactured.

If you want to install Raspbian, and it is not pre-installed, download from https://www.raspberrypi.org/downloads. That imager is run on any device, say your pc, and then you select write to save it onto the micro-SD card (don't forget this overwrites anything already on the card). This should work without a need to format the card first,(but if you do need to format it, do so using a SD card formatter, not the Windows format tool). After this image has been stored it will have created two partitions (one the boot partition can be accessed by Windows, but the larger partition is invisible to Windows).

Pre-configuring the Pi for headless operation

You can store files in the boot partition of the micro-SD card that can be accessed by Windows. When the card is inserted in a Pi, these two files will be removed from that partition, one will switch on a setting that is by default off, the other file will be moved to a new location and enable wireless connection to your local network.

Secure Shell Home

Secure Shell (SSH) is a cryptographic network protocol for operating network services securely over an unsecured network. By default, this optional way for another device to see your PI is turned off. You can turn it on by the configuration application on your Pi, but you may want it turned on as your Pi boots up and you can do that by adding a file to your micro-SD card.

The file must be named "SSH" with those three letters in capitals, but with no file extension and no contents. On a Windows PC, if you right click (while viewing the boot directory on the card) there is an option called New and if you select a text file it will create an empty file with the extension .txt. (In windows there is an option to hide extensions which is on by default, so you may need to deselect this option [New menu -> Options] to see this extension). On Windows you can open the file using Notepad to verify it is empty, if you gave accidentally created a file of another type like word processing it will be full of characters some of which do not display. Anyway, you must remove any extension from the file name so it is really just SSH.

When the card is inserted into your Pi, on boot this file will be removed and the SSH option will be enabled.

Wireless Network

Again, create a new file, again if you are using Windows you can create it as a text file and then replace the .txt extension with a .conf extension, store it in the boot directory on your micro-SD card under the name wpa_supplicant.conf.

Using a text editor that won't add any unwanted control characters, add the following text using UTF-8 encoding:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=GB

network={
    ssid="YourNetwork"
    psk="YourNetworkPassword"
    key_mgmt=WPA-PSK
}

• Obviously, if you are not in United Kingdom, you will replace GB by the country code that applies to you.
• Within the first set of quotes, replace YourNetwork by whatever Service Set Identifier is used for your wireless network. You may have typed this into your mobile phone. It may be shown on a card that slips into a slot on your hub or router. Whatever it is, and it can be up to 32 characters (letters, numbers, and symbols), type it within the double quotes. Some routers come with a default SSID that is the manufacturer's name, and leave it to you to pick a SSID that is unique to you using up to 32 characters to personalise it.
• Within the next set of quotes, which relate to the replace YourNetworkPassword by whatever Pre-Shared-Key (password) is used for your wireless network. You may have typed this into your mobile phone. Hopefully, you have changed it from whatever was shown as the initial password on the card that slips into a slot on your hub or router (even if all you have done is add a prefix or suffix that means something to you).
• Most wireless networks will use Wi-Fi Protected Access (WPA) or (from 2006) Wi-Fi Protected Access II (WPA2) protocols, so WPA-PSK is correct for you. Note that your Pi is only able to use these protocols. The earlier Wired Equivalent Privacy (WEP) was officially withdrawn in 2004 as too easy to crack, so it is not supported on a new Pi.

Should you wish to set up your Pi with several network definitions, please see Notes by ExperiMentor (a contributor to the Cumulus support forum in Switzerland).

Other configuration

There are various other configurations you need to do on your PI. Unlike SSH, these can't be done by storing files on the micro-SD card. You need to use the raspbian configuration tool raspi-config, and this can be accessed on your Pi either in a Graphical User Interface (GUI), or by running a command in Terminal. The same command can be run from a remote device if you successfully have SSH running. On a Windows pc, this will typically involve use of PuTTY software (an SSH client for Windows) downloaded from https://www.putty.org/. In both cases the command to use is sudo raspi-config.

These comprehensive notes describe how to install Cumulus MX on a Pi Zero, using a PC to do some of the work:

Download useful PC software and install on your PC These instructions are for a Windows PC. Steps would be similar on a Mac, but programs and details would differ. Should also be possible with an Android tablet.

• SD Formatter (the Windows Format facility will NOT do)
• balenaEtcher (for unzipping and burning images to SD cards) [Previously named 'Etcher'] https://etcher.io/
• Win32DiskImager (for backup & restore of SD card images) https://sourceforge.net/projects/win32diskimager/
• PuTTY
• FileZilla (an FTP file transfer program for Windows) https://filezilla-project.org/download.php

Download Raspbian Pi Operating System

• Save it on your PC, from https://www.raspberrypi.org/downloads/raspbian/
• "RaspBIAN Buster Lite" is probably OK, but other than small file size it offers no advantage over installing the full version of RaspBIAN Buster. These instructions are being tested using "Raspbian Buster with desktop and recommended software", the largest of all, which could allow you to do other things more easily.
• Just click on “Download Zip” (torrent might be faster if you have the ability, but not worth installing just for this)
• Do not unzip it
• These instructions have been tested with kernel version 4.14, released 18 April 2018 and with kernel version 4.14, released 13 November 2018 [March 2019] and kernel version 4.19 released 10 July 2019

Install Pi Operating System onto Micro SD card

Format the SD card

• Put Micro SD card in PC (use adapter if needed)
  • If re-using a previous Pi SD card, click ‘Cancel’ on the warning about needing to format the card
• Run SD Card Formatter (click Yes to ‘Allow to make changes to your device’).
  • Need to use this program rather than the Format tool in File Explorer, because Pi SD cards end up with a very small ‘Windows accessible’ partition and a large partition containing Linux. SD Card Formatter allows reclaim of the large partition.
• Your SD card should automatically populate in the ‘Drive’ box. In case you have another SD card in your PC, ensure the correct card is selected!
• Click ‘Format’ and check and accept the Warning messages

Copy the Pi Raspbian Operating System onto the card

• Run balenaEtcher on your PC
• Click ‘Select Image’ and choose the ‘Raspbian Buster’ operating system zip file that was downloaded earlier
• SD card should be automatically populated. In case you have another SD card in your PC, ensure the correct card is selected!
• Click ‘Flash!’. The operating system will be copied to the card. This takes about 10 minutes, followed by another 8 minutes to ‘Verify’
• Cancel any messages about needing to Format the card - they are just indicating that Etcher has installed the partition that cannot be read by Windows
• On completion, the card is ‘ejected’ from the PC. Physically remove it and then straight away reinsert it so that the content can be viewed in File Explorer
• TWO drives will now be visible for the SD card. You will likely see a warning that one of the drives needs to be formatted before it can be used. ‘Cancel’ that warning and ignore that drive.
• View the other drive, which is named ‘boot’ in File Explorer
• On the View tab, ensure the ‘File Name extensions’ is ticked
• Right click and select ‘New’, ‘Text document’. Change its name to SSH (deleting the .txt extension; you need to make an empty file called SSH not SSH.txt). Click ‘Yes’ to ‘Are you sure you want to change the extension?’
• Right click and select ‘New’, ‘Text document’. Change its name to wpa_supplicant.conf (deleting the .txt extension; you need to make a file called wpa_supplicant.conf not wpa_supplicant.conf.txt). Click ‘Yes’ to ‘Are you sure you want to change the extension?’
• Right click on this new file and select ‘Open with Notepad’ or ‘Open with …’ then select Notepad. Enter the following content exactly as below (copy and paste) then edit your country code (if needed), WiFi network’s SSID and password: NOTE: Change GB as needed to be the code for your country. The quote marks should appear in the file, that is ssid="YourNetwork" not ssid=YourNetwork . Same for psk.

• Not essential, but I like to keep copies of both those files for future use. They can be on the SD card with different names eg ‘SSH - Copy’ and ‘wpa_supplicant.conf - Copy’ as well as on your PC
• The function of these 2 files is to connect your Raspberry Pi to your network as soon as it boots, and allows you to connect to and control it from your PC by SSH using PuTTY. This avoids needing to connect a keyboard, mouse and monitor to the Raspberry Pi. It is particularly useful for Pi Zero W (or Pi Zero) which hasn’t got enough USB connections and no Ethernet (wired network) connection. This is called ‘Headless operation’.
• Right click on the ‘boot’ SD card in left pane of File Explorer and ‘Eject’ it safely.

Setting up the Raspberry Pi

• With nothing plugged into the Raspberry Pi, take the Micro SD card from your PC and put it in the Pi.
• In a later step, you will need to find out the Raspberry Pi’s IP address by looking at your network router’s web interface. I can’t help you with doing that. If you don’t know how to, an alternative is to connect a keyboard, mouse and monitor to the Raspberry Pi at this stage
• Plug the power supply into the Raspberry Pi. It will boot up (note flashing red and/or green LEDs depending on model).
• On your PC, log into your network router’s web interface and identify the Pi’s IP address, which will be in the form xxx.xxx.xxx.xxx, for example 192.168.1.123
  • NOTE: If you will be switching from a faster “build” Raspberry Pi to a “production” Raspberry Pi Zero W, the IP address will change, so you’ll need to repeat this step later
  • While in your network router for the ‘production’ Pi you will be using, set up some port forwarding that will be needed later.
  • Forward port 8998 to your Pi’s IP address for TCP protocol if you want to be able to access the Cumulus web interface from the external internet (this brings potential security risk though). [Forwarding port 8002 as well was previously needed].
• Start PuTTY on your PC. In the box for ‘Host Name or IP address’, enter the Pi’s IP address from above. In the adjacent ‘Port’ box, enter 22. Connection type should be SSH. Click ‘Open’.
• A window opens. The first time you do this you will probably see a long message asking to confirm it is OK to connect to a not-previously-known device. Click ‘Yes’.
• Login to the Pi. Username is pi [lower case] and password is raspberry [lower case]
• You will see a warning that SSH is enabled but the password has not been changed, which is a security risk. We will change the password in a moment
• Type

sudo raspi-config

• Note, to copy from here (usually need to do 1 line at a time), select it then CTRL-C. To paste into the PuTTY window, right click.
• As needed, adjust the following settings:
  • Change the password to something you will remember. Leaving it at raspberry is a serious security risk – exposes your whole network to hackers
  • In Network Options,
    1. change the name of your pi to ‘Cumulus’ or something you prefer
    2. WiFi network and password have already been set by the wpa-supplicant.conf file added earlier
  • In Boot Options, Desktop / CLI, select ‘Console Autologin’
  • In Localisation Options,
    1. change ‘Locale’ if you need something different to en_GB.UTF-8. [Changing this takes quite a while on a slow Pi]. [As of Sep/Oct 2019, there is some kind of incompatibility between RaspBIAN Buster, mono v6.0.0.314 and locales other that en_GB - so unless you NEED another locale, it would be better to leave it as en_GB. The alternative is to force load an older version of Mono, for example v5.18]
    2. Change Timezone.
    3. Change Keyboard Layout if needed
    4. WiFi country has already been set by the wpa-supplicant.conf file added earlier
  • In Interfacing options, SSH server has already been set to be enabled by the empty SSH file added earlier
  • Select ‘Finish’. There is no need to reboot at this stage. But until you do, you will see messages "sudo: unable to resolve host raspberrypi", but these can be safely ignored (it's just because you renamed the Pi - will disappear after next reboot)

In the steps below, you will need to press y to agree to proceed at various times

If you have been building the Micro SD card on a fast Pi, now is the time to switch to the 'production' Pi, for which a slower Pi Zero W is more than adequate. Shut down the Raspberry Pi safely.

sudo halt

Move the micro SD card to the Pi Zero W. Power on the Pi Zero W. Your SSH (PuTTY) session will close out and you'll need to reconnect after the Pi has rebooted. Use username pi and the new password you chose earlier.

Add the ‘Mono’ package

• Simplification: Mono is a package which allows programs to be written cross-platform so that they will run on Linux (including Raspberry Pi), Windows and Mac OS, similar to the Windows ‘.NET Framework’.
• The previous anomaly with the USB library not working with later versions of mono, affecting Fine Offset stations and the later Oregon Scientific stations (WMR88/100/200 etc) has been fixed (in CumulusMX build 3044 onwards) and these and other stations should now be fine with later/current versions of mono. I am currently using a Fine Offset with mono v5.18
• Process is to install a security certificate, add the mono server to the list of software sources [sources.list] that the Pi searches, then install the mono-complete package:

sudo apt install apt-transport-https dirmngr gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt-get update && sudo apt-get upgrade 
sudo apt-get install -y mono-complete
sudo apt autoremove

At the time of writing (18 Sep 2019), this gets Mono v6.0.0.334, which works with Buster (RaspBIAN 10). However, there have been reports of incompatabilities which require use of an older version of Mono. These may have now been fixed, or alternatively may be related to use of locales other than en_GB.UTF-8 . Please see other threads in Support Forum for discussions. NOTE: 29 Feb 2020: added -y into the line sudo apt-get install -y mono-complete . This makes the install bypass the usual 'Continue Y/n?' prompt¨which was causing strange problems for some, e.g. worked if just pressed 'Enter' to accept default 'Y', but aborted installation if pressed 'Y Enter'. Bizarre. Reboot your Raspberry Pi This would be a reasonable time to reboot your Pi:

sudo reboot

Your SSH (PuTTY) session will close out and you'll need to reconnect after the Pi has rebooted. Use username pi and the new password you chose earlier. Install Cumulus MX on the Raspberry Pi Download it from here to your PC, unzip on your PC which makes a directory named CumulusMX. Remember where that directory is located then on PC run FileZilla

1. In the ‘Host’ box, enter the Raspberry Pi’s IP address eg 192.168.1.123
2. In Username, enter pi
3. In Password enter your pi’s password
4. In Port, enter 22
5. Click ‘Quickconnect’. Raspberry Pi’s directory structure appears on the right and your PC’s directory structure is on the left.
6. In the LEFT window, navigate to where you unzipped the download of Cumulus MX earlier. Ensure can see the folder name ‘CumulusMX’ in the lower left window
7. In the RIGHT window, ensure that the folder /home/pi is shown (see top right window; contents in bottom right window include .cache, .config etc)
8. Drag the folder ‘CumulusMX’ to an empty area in the lower right window (not onto one of the existing directories). Watch progress as this copies the whole CumulusMX folder and contents to directory ~/CumulusMX on the Pi
9. Close FileZilla

On Raspberry Pi PuTTY window:

sudo halt

Plug your USB weather station into the Raspberry Pi – USB cable into the OTG connector (probably via an adaptor lead) if using Raspberry Pi Zero W. If you have an ethernet or WiFi linked weather station then you won't need to do this - I don't have one so I don't know exact details. Steve below says you need to enter the IP address during Cumulus setup, but then also adjust a disconnect period if you are also using Weatherlink software.

Running Cumulus On PC, run PuTTY again and log in to the Pi as before (note you can save the IP address between sessions)

cd ~/CumulusMX
sudo mono CumulusMX.exe

The next thing you will want to do is access Cumulus via its user interface from your PC, so that you can update the settings. Using the IP address for your Pi, in your internet browser, enter: 192.168.y.z:8998 (where y and z are numbers you will need to find from seeing how your router connects to your Pi. You’ll first see a dashboard page, then can access the Settings menu.

To make Cumulus run each time the Pi is rebooted (and force reboot in the early hours each day) On the Pi, type:

sudo crontab -e

On first run select the text editor you prefer (defaults to #1, nano, the easiest) Then add the following lines at the end of the file:

# Start Cumulus as background task 30s after reboot (delay to allow WiFi to startup)
@reboot (sleep 30;cd /home/pi/CumulusMX;sudo mono CumulusMX.exe) &

# Reboot each day at 0253
53 02 * * * sudo reboot

To stop the Pi and restart it without CumulusMX running (eg you need to do that if upgrading the CumulusMX version) type the following

sudo crontab -e

edit to put a # at the start of the line "@reboot..." Ctrl-X to save the change to crontab and reboot using

sudo reboot

When your pi restarts, CumulusMX will no longer be running. You can then do your version upgrade or other task.

To revert to normal auto-running of CumulusMX, go through the same again, but this time edit crontab to remove the # from the start of the line "@reboot...". Save changes and reboot - CumulusMX will be running.

Updating a version of CumulusMX is easily done as follows using this: 1. Stop CumulusMX running (it locks files while it is running) 2. Install the updated CumulusMX version into a new directory - I call mine CumulusMX3xyz (where xyz are the last 3 digits of the build number) so that I can easily see which build it is 3. copy the following from the old CumulusMX directory to the new CumulusMX3xyz directory:

- your CumulusMX/Cumulus.ini file

- your CumulusMX/data directory

- your CumulusMX/twitter.txt file (if you have personalised it)

- your CumulusMX/web directory (if you have personalised any web files)

4. Change your startup instruction to use the version in the new directory eg cd /home/pi/CumulusMX3050;sudo mono CumulusMX.exe

With that method you can easily revert back to the old version if something has gone wrong. If all is well, you can delete the old directory after a few days/weeks/months/if you need the space.

Updating mono version

• First, stop CumulusMX as above by editing crontab.
• Then remove the present version of mono:

sudo apt-get purge libmono* cli-common mono-runtime
sudo apt-get autoremove

• Then install the new version

sudo apt-get install mono-complete

• Finally re-enable auto running by editing crontab to remove the # and finally

sudo reboot

Above Instructions: Last edited by ExperiMentor on Sun 01 Mar 2020 8:17 am,