How to use your Raspberry Pi to monitor broadband speed

How to use your Raspberry Pi to monitor broadband speed

In this article, we’ll show you how to use the Raspberry Pi to monitor broadband speed. This project involves a fair bit of setup, including downloading and installing some packages, writing a Python script, and more. But once we’ve laid the groundwork, we can automate our broadband monitor so that it keeps right on monitoring for as long as we want. The monitor will create a CSV (comma-separated values) file to your Google Drive and update that file once an hour. CSV is a very simple file format that can be opened in Microsoft Excel or imported to Google Sheets. Sound like a handy thing? Then let’s get started. This how-to will assume you already have Raspbian installed on your Pi.

How to use your Raspberry Pi to monitor broadband speed

Step 1: Check for updates

As usual, let’s start with checking for the latest updates. Run the following commands in Terminal:

sudo apt-get update
sudo apt-get upgrade

Step 2: Install speedtest-cli

There are various methods to measure the speed of broadband connection. We’re going to use speedtest-cli, which, in the words of its creator, is “a command line interface for testing internet bandwidth using speedtest.net.”

speedtest-cli isn’t available right away from the Raspbian archives, but we can install it from the Python Package Index (PyPI). That’s easy to do so by using a tool called pip that comes preinstalled on Raspbian Jessie and Stretch. You can make sure that you have pip by running this command:

sudo apt-get install python-pip

If you a message that reads “python-pip is already the newest version,” that means that we’re good to go.

Next, let’s use pip to install speedtest-cli:

sudo pip install speedtest-cli

With speedtest-cli now installed, you can measure your broadband speed easily with this command:

speedtest-cli

Though, for our purposes, using speedtest-cli’s simple mode is more convenient:

speedtest-cli --simple

You should see something like this:

Ping: 47.943 ms
Download: 40.93 Mbit/s
Upload: 2.33 Mbit/s

However, that output doesn’t conform to the CSV syntax. Therefore, we’ll need to parse the data and make it conform.

Step 3: Create a Python script

Let’s create a new Python file:

sudo nano speedtest.py

Here’s what we should have inside the file (you can, of course, copy-paste these lines):

import os
import re
import subprocess
import time

response = subprocess.Popen('speedtest-cli --simple', shell=True, stdout=subprocess.PIPE).stdout.read()

ping = re.findall('Ping:\s(.*?)\s', response, re.MULTILINE)
download = re.findall('Download:\s(.*?)\s', response, re.MULTILINE)
upload = re.findall('Upload:\s(.*?)\s', response, re.MULTILINE)

ping[0] = ping[0].replace(',', '.')
download[0] = download[0].replace(',', '.')
upload[0] = upload[0].replace(',', '.')

try:
    if os.stat('/home/pi/speedtest/speedtest.csv').st_size == 0:
        print 'Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)'
except:
    pass

print '{},{},{},{},{}'.format(time.strftime('%m/%d/%y'), time.strftime('%H:%M'), ping[0], download[0], upload[0])

(I’m assuming that you’re saving the script in the /home/pi/ directory – if not, just change the path here: if os.stat(‘/home/pi/speedtest/speedtest.csv’).st_size == 0:.)

After you have the lines in place, you can save the file and exit the editor by pressing Ctrl+X, Y, and Enter.

The script runs speedtest-cli in its simple mode, parses the output, and prints it out in CSV format. You can run the script with the following command:

python speedtest.py

And if you do, you should see a line like this:

10/26/17,10:18,47.943,40.93,2.33

Alright, we’re ready to move on!

Step 4: Create a directory

Let’s create a directory for our CSV file:

mkdir speedtest

If you’re wondering why we need a directory for a single file, it’s because we’re going to sync that directory to Google Drive. Once synced, everything inside the directory will correspond the contents of our similar Google Drive directory.

Now, if we run our Python script like this:

python speedtest.py >> speedtest/speedtest.csv

…we will have a CSV file with broadband speed data in our new directory.

If you check the contents of the file (for example, cat speedtest/speedtest.csv), you can see lines like these:

Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)
10/26/17,10:18,47.943,40.93,2.33

Great!

Step 5: Integrate the script with Google Drive

In order to integrate our script with Google Drive, we can use GitHub user Petter Rasmussen’s Google Drive CLI Client. Let’s download its Raspberry Pi version by running the following command:

wget -O gdrive https://docs.google.com/uc?id=0B3X9GlR6EmbnVXNLanp4ZFRRbzg&export=download

Next, let’s assign the file executable rights:

chmod +x gdrive

We’ll need to tell Google Drive to allow Google Drive CLI Client to connect to our account. We can do so by running Google Drive CLI Client with any parameter. For example, this command lists the contents of a Google Drive account:

./gdrive list

You should now see an authentication request like this:

gdrive - Authentication request

Just follow the instructions: Go to the URL in your browser, sign in to your Google account, and allow “GDrive (…) to [v]iew and manage the files in your Google Drive.” Then you’re ready to enter the verification code.

Finally, gdrive will list the contents of your Google Drive.

Now that we have our speedtest directory on the Raspberry Pi, let’s create a corresponding directory to our Google Drive:

./gdrive mkdir speedtest

The program returns the ID of our new directory. Copy that, since we’ll need it.

Next, we’re going to sync our two speedtest directories:

./gdrive sync upload speedtest ID

Just replace ID with the ID your speedtest directory.

If everything went as supposed, you’ll now see a directory called speedtest in your Google Drive. In the directory, there’s the file we created earlier (speedtest.csv):

Google Drive - speedtest.csv

Now, if we run the Python script again, new broadband speed data will be appended to the end of the file:

python speedtest.py >> speedtest/speedtest.csv

And if we run the sync command again, we can see our updated file in our Google Drive:

./gdrive sync upload speedtest ID

(Again, remember to replace ID with the ID of your speedtest directory.)

grive - Sync directories

Step 6: Automate everything

We’re almost ready! All we have left to do is to make everything work automatically. For this, we’ll use cron, which makes it possible to schedule commands to run at specified times, like once an hour.

Let’s make a short shell script containing the commands we need to run once an hour:

sudo nano speedtest-cron.sh

Add in the following, familiar commands (I’m assuming that you’ve done everything in the /home/pi/ directory – if not, just change the paths):

sudo python /home/pi/speedtest.py >> /home/pi/speedtest/speedtest.csv
/home/pi/gdrive sync upload speedtest ID

(And, once again, remember to replace ID with the ID of your speedtest directory.)

Then save and exit with Ctrl+X, Y, and Enter.

Let’s assign the script executable rights:

sudo chmod +x speedtest-cron.sh

Now we’re ready to test the script:

./speedtest-cron.sh

Alright, let’s create our cron job:

crontab -e

Type the following line in the editor and save and exit by pressing Ctrl+X, Y, Enter:

0 * * * * /home/pi/speedtest-cron.sh

Now our broadband speed monitor is ready. It will log the results of the speed test once an hour at the top of the hour. You can open the CSV file that’s in your Google Drive using Microsoft Excel or Google Sheets. If you prefer the latter, you’ll have to import the file (File > Import…).

Google Sheets - speedtest.csv

11 Comments on "How to use your Raspberry Pi to monitor broadband speed"

  1. The script itself is great, but the cronjob doesn’t work here.

    The second test of the python speedtest.py is wrong.
    It should be: python speedtest.py >> speedtest/speedtest.csv (CSV, not PY)

    Any tips, why the cronjob doesn’t work?

  2. Great idea, thanks for the post. I am running mine on a Pi with Stretch installed. The cronjob wasn’t working due to this line:

    response = subprocess.Popen(‘speedtest-cli –simple’, shell=True, stdout=subprocess.PIPE).stdout.read()

    I had to add the absolute location of speedtest-cli -> /usr/local/bin/speedtest-cli as such:

    response = subprocess.Popen(‘/usr/local/bin/speedtest-cli –simple’, shell=True, stdout=subprocess.PIPE).stdout.read()

    Cron nominally outputs errors to a user’s mail. Since the pi doesn’t have mail installed by default I installed postfix and checked my mail after the next time the cron was supposed to kick off. From there I was able to see the ‘not found’ error.

  3. I was getting an invalid syntax error on this line:
    print ‘Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)’

    and was able to fix it by changing it to:
    print (‘Date,Time,Ping (ms),Download (Mbit/s),Upload (Mbit/s)’)

    same fix applies to this line as well:
    print ‘{},{},{},{},{}’.format(time.strftime(‘%m/%d/%y’),

    On an unrelated note can anyone explain to me why I might be having syntax errors here:
    time.strftime(‘%H:%M’), ping[0], download[0], upload[0])
    ^

  4. Hello,

    I follow the instructions exactly, but I have two problems.

    1) RSP does not create Google tables (Excel), but the ordinary document (Word) writes the data into a line separated by a comma.

    2) The speedtest task itself will not run… And so the document only contains the first entry.

    Please help me to resolve this issue, or advise what to do. Thank you.

  5. Thank you so much for the post! Got one quick question: Would you happen to know why the data will be uploaded to a Google doc and not a Google sheet?

  6. speedtest-cli now has a –csv option so you can skip all the python code. 🙂

  7. Great tutorial, much appreciated. Works as described. Am now monitoring difference between LAN and WLAN (2 RPIs) to search for dips in my connection at congested hours (evenings).

    Changed the minutes to */15 * * * * to monitor every 15 minutes. Will not leave it running for a long time due to bandwidth consumption.

    Changed the other crontab to 2,17,32,47 * * * * to avoid two speedtests running at the same time.

    Thanks!

  8. Is it possible to sync with Dropbox instead of Google Drive?

  9. Good day,

    I can see my speedtest dir on Google Drive, but when I run ./gdrive sync upload speedtest MY ID. I get the following error: Failed to find root dir: googleapi: Error 404: File not found: MY ID., notFound, can you please assist.

Leave a comment

Your email address will not be published.


*