Wednesday, August 27, 2025

Finishing Up The Color Thermometer

 

Finished Color Thermometer

In the last post I looked at building a thermometer that showed the temperature and humidity of a room through the use of a color display that indicated the relative comfort level of the room, using an Arduino and a couple of extra components. 

While the thermometer worked quite well, it really did just look like just a bunch of wires that glowed. To have it be part of polite society, it needed to be dressed in more appropriate attire. 

Since the circuit is a fairly compact design, the thermometer would lend itself well to being housed a fancy box that could look pretty sitting on a shelf. On a recent clean-up of the shop I noticed that I had few pieces of red cedar and popular sitting in the scrap bin.  I always liked the look (and the smell!) of red cedar so I always try to avoid throwing of it out on the off chance I had use for it. As luck should have it, the scrap pieces looked to be a good size to house the Arduino. 

Designing The Case  

With my source materials acquired, I set about designing my enclosure. Basically the enclosure had to:

  • Of course contain the Arduino, Temperature sensor and the LED strip.
  • Have a way to mount the LED strip so that it seamlessly integrates into the case
    • To do that I decided that the case should have a slot cut into it to mount the LED.
    • To add a bit of a diffusion to the LED's the slot would also be cut to accommodate a plastic strip to mount in front of the LED's 
  • Provide proper air flow so that the temperature sensor can properly detect the room conditions
  • Be simple, basically the case should just be a your basic box.
With the criteria defined and based on the wood I had on hand, I sketched out a quick design



Bill of Materials 

  • Two 1x4x6.5 inch pieces of popular
  • Two 1/4x4x6.5 pieces of red cedar
  • One 1/8x3/4x6.5 piece of clear acrylic 
  • Frosted glass paint
  • Wood glue
  • Furniture felts
  • Small rubber grommet

Tools Required
  • Table saw
  • Router with a 3/4 inch straight bit
  • Wood clamps
  • Drill Press
  • Glue

Case Assembly

I started off by cutting out the 4 pieces of wood that will be used to make up the case

Wood parts

Next I marked out the cavity that will be used to house the Arduino and sensor on the two popular pieces and routed out the cavities

Marking out the cavity

Cutting out the cavity

Cut out cavities

Once everything was cut out, I then glued the two Poplar pieces together along with one of the Red Cedar pieces (which will serve as the top for our case)

Gluing things together

Gluing the top

Gluing things together

 Once the glue hard dried. I next had to put in a grove near the top of the case. To do that I took a 3/4 inch straight bit on my router and I cut a 1/2 inch deep groove about one inch down from the top of the case along the front. This groove needs to be deep enough to accommodate the LED strip and the 1/8 inch thick acrylic strip. 

Routing the strip

Routing the strip

LED Test Fit

Completed Groove

One problem that I found with the groove is that I did have it come out of the sides of the case which didn't really look very good as far as I was concerned so I really wanted to have it look a little bit better.

To solve that I cut out and glued in a couple of red cedar "plugs" to fill the ends of the grooves but still left some space on the end for the acrylic strip to be flush against the front of the case. 

Acrylic strip test fit

Cutting out plugs

Plugs installed

At this point, the main part of the case is now assembled. 

Perfect time to give every a quick sanding to neat everything up. 

Quick Sanding

With the groove in place I next cut a 3/4 inch wide strip of acrylic and cut it to the length of the case.

After a quick test fit to the case (and some trimming to make it fit better) I then gave the strip a coat of glass frosting paint to get a more diffused look when the LED's are lit. 

Cutting  acrylic

Doing a test fit

painting the strip

To complete the construction of the top part of the case I finished off by drilling in a few vent holes so that I could have some air flow for the temperature sensor.

Top of the case

Drilling out the vent



Installing the Innards 


The next step in the process was to put all the electronic bits into the case. I started off by looking at what would be best way to mount the Arduino.  To do this, I decided on mounting it to the remaining piece of red cedar that I hadn't yet glued to the case. 

After doing a test fit of the Arduino to the bottom plate I determined what would be the best place to mount the Arduino I then cut out and glued a couple of wooden rails to the red cedar, which would serve as a mounting point for the Arduino. 

Mounting the Arduino

Gluing in the Rails

Once the glue dried, I then mounted the Arduino to the base with a few screws.

Mounting the Arduino

The next thing to do was to mount the temperature sensor into the case. Since I drilled the ventilation holes into the top of the case, it makes the most sense to have the sensor mounted as close as possible to the vent. To do this I glued another small wood block to the top of the case and attached the sensor to the block.

Sensor Mount

Sensor Mounted


Finally I mounted the LED's into the groove that I had earlier cut, luckily the groove also opened up an access to the internals of the case, so the wiring for the LED was pretty easy to feed through 

I glued the LED strip in place followed by the acrylic strip 

Installing the LED strip

Installing the LED strip

Installing the acrylic strip



With all the various components mounted the next step was to wire everything up and buttoning up the case. 
Wiring up the circuit

Wiring up the circuit


One thing I failed to note was how I planned to power up the Arduino once it was sealed up. 

To power the Arduino, I chose to power it through the Arduino's USB connector, which will also give me an opportunity to do any tweaks to the programming that I might want to do in the future. 

To provide power access, I usually would drill a half inch hole into the back of the case, but as a lucky coincidence, one of my popular pieces had a knot that popped out that was the ideal size. So I fed the USB cable through the knot hole and used a rubber grommet to neatly cover the hole.

Installing the USB cable

Finishing the wiring

 Finally I glued the rest of the case together

Gluing it all together

Gluing it all together

I finished up the job with some varnish and after installing some felt pads on the bottom, the project is done!

Varnishing

Felt pads

At the moment the thermometer is still on my desk here at work giving my a nice visual interpretation on the office climate.  It's certainly been a bit of a conversation piece. 

Next month, I going to revisit a topic that I first posted about a few years ago. 


Tuesday, July 15, 2025

Building A Color Thermometer



This month I wanted to get back onto more "hands on" stuff that was more based in the real world than the virtual one. When looking around for ideas, I found a small stash of Arduino Uno's that I got from my Creation Crate subscription a while back (by the way - I highly recommend them - the neat little projects I got each month were a lot of fun to put together - and I learn quite a lot on what the Arduino is capable of) so I wanted to find a unique project that I could make with and Arduino and whatever components I could find lurking around the workbench.

Rooting around, I came across a roll of NeoPixel programable color LEDs. These are pretty neat since you can do a lot of different light effects and you could program it so that individual LED's on the strip to light up. The original reason I had that roll was for a clock project that didn't quite pan out the way I wanted it to go. 

I also came across a DHT11 module, which is a small temperature/humidity sensor that feeds its data through a single wire data feed. 

So, with those 3 items, the obvious solution is to put together a thermometer that displays the temperature and humidity in the room as a color display. 

Since the LED's and the sensor are digitally controlled, it really makes for a simple circuit to put together - basically I just need to plug the components directly to the Arduino's power outputs and digital inputs and some fairly straight forward Arduino code should handle the rest. 

The basic premise of the project is to be a smart weather display that automatically changes light patterns based on the temperature and humidity in the room. The temperature sensor continuously monitors the air conditions and then creates different colorful lighting effects on the LED strip to show what the weather felt like.



Showing The Room Conditions

To properly visualize the room conditions I really needed to come up with an appropriate color effect that reflected how it felt in the room. 

After some playing around with the NeoPixel, I came up with the following patterns: 

  • Cold and Damp (Under 15°C with high humidity over 40%)
    • Effect: Cool Blue Pulse
      • The lights glow in cool blue tones that slowly pulse brighter and dimmer
      • Represents the feeling of being underwater or in a cold, misty environment
      • Creates a calming, cool atmosphere

  • Hot and Dry (Over 25°C with humidity under 70%)
    • Effect: Fire Flicker
      • The lights flicker in red, orange, and yellow colors like real flames
      • Each light changes randomly to create a realistic fire effect
      • Represents the dry heat of a desert or fireplace

  • Comfortable Conditions (15-25°C with moderate humidity 40-70%)
    • Effect: Calm Green Breathing
      • Soft green lights that slowly fade in and out like gentle breathing
      • Creates a peaceful, natural feeling
      • Indicates ideal living conditions

  • Hot and Humid (Over 30°C with high humidity over 70%)
    • Effect: Storm Swirl
      • Purple and magenta colors that swirl and move along the strip
      • Creates a chaotic, energetic pattern
      • Represents the unsettled feeling of hot, sticky weather

  • All Other Conditions
    • Effect: Rainbow Cycle
      • A smooth rainbow pattern that continuously moves along the strip
      • Shows all colors in a pleasing sequence
      • Acts as the default display when conditions don't match the other categories

The lighting effects are intended to be intuitive - cooler colors for cooler conditions, warmer colors for hotter conditions, and special effects for extreme or unusual combinations of temperature and humidity.


Now that I have the effects that I wanted to have to represent the conditions in the room, that next step is to start putting things together



Components

To start the project, I collected the following times




Temperature and Humidity Sensor (DHT11)
  • A small electronic sensor that measures how hot or cold it is and how much moisture is in the air
  • Takes readings every few seconds automatically
  • Connected to your Arduino microcontroller

LED Strip (NeoPixels)
  • A strip of  colorful lights that can display any color
  • For this project I'll use a string of 7 LED's
  • Each light can be controlled individually
  • Can create smooth animations and effects

Arduino Uno Microcontroller
  • The "brain" that reads the sensor data and controls the lights



Technical Details

  • Temperature Range: Celsius measurements are to be used
  • Humidity Range: Measured as relative humidity percentage (0-100%)
  • Update Speed: The lights on the NeoPixel update every 20-80 milliseconds depending on the effect
  • Power: The thermometer will runs on standard Arduino power (5V). In particular I plan on running the Arduino off the USB connector, which will allow me to apply future software updates. 
  • Installation: I am planning on enclosing the thermometer in some sort of case, but it will need to be able to have access to power and good air circulation for accurate readings.



Wiring It Up

With the components all collected, I looked at how to connect everything up.  

In reality, connecting all the components together are pretty straightforward. 

Connections to the Arduino are:
  • The positive connection from the LED was connected to the +5 port
  • Data line from the LED was inserted to the D6 socket
  • Ground on the LED was inserted into one of the Ground sockets
  • Positive connection from the DHT11 was put into the  Vcc socket
If you're more picture oriented, I've also included a handy schematic diagram below



Writing The Code 

With everything all connected, the next step is to tell the Arduino to start monitoring the output of the DHT11, check the readings it got against the display rules that I defined earlier and tell the NeoPixel LEDs to display the appropriate color display. 

The Arduino checks the temperature and humidity continuously and updates the light display in real-time. The effects needed to be smooth and change gradually as the environmental conditions changed.

To get everything to work, I put together the following program and uploaded it up to the Arduino:


#include <Adafruit_NeoPixel.h>
#include <DHT.h>

#define DHTPIN 6
#define DHTTYPE DHT11
#define LED_PIN 2
#define NUM_LEDS 7

DHT dht(DHTPIN, DHTTYPE);
Adafruit_NeoPixel strip(NUM_LEDS, LED_PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  dht.begin();
  strip.begin();
  strip.show();
}

void loop() {
  float temp = dht.readTemperature();
  float hum = dht.readHumidity();
  Serial.begin(9600);

  if (temp < 15 && hum > 40) {
    coolBluePulse();
  } else if (temp > 25 && hum < 70) {
    fireFlicker();
  } else if (temp >= 15 && temp <= 25 && hum >= 40 && hum <= 70) {
    calmBreathGreen();
    } else if (temp > 30 && hum > 70) {
    stormSwirl();
  } else {
    rainbowCycle();
  }
}

void calmBreathGreen() {
  static float brightness = 0;
  static int fadeDirection = 1;
  static unsigned long lastUpdate = 0;

  const unsigned long interval = 100;    // Slower fade timing
  const float step = 0.8;                // Smaller step for smoothness

  if (millis() - lastUpdate >= interval) {
    lastUpdate = millis();

    brightness += fadeDirection * step;

    // Reverse direction at full brightness and full darkness
    if (brightness >= 150) {
      brightness = 150;
      fadeDirection = -1;
    } else if (brightness <= 0) {
      brightness = 0;
      fadeDirection = 1;
    }

    // Set all LEDs to green with current brightness
    for (int i = 0; i < NUM_LEDS; i++) {
      strip.setPixelColor(i, strip.Color(0, (int)brightness, 0));
    }
    strip.show();
  }
}

// Effect: underwater blue pulse
void coolBluePulse() {
  static uint8_t brightness = 0;
  static int fadeDirection = 1;

  for (int i = 0; i < NUM_LEDS; i++) {
    strip.setPixelColor(i, strip.Color(0, brightness, brightness + 40));
  }
  strip.show();

  brightness += fadeDirection * 3;
  if (brightness <= 10 || brightness >= 120) fadeDirection *= -1;

  delay(40);
}

// Effect: flickering fire
void fireFlicker() {
  for (int i = 0; i < NUM_LEDS; i++) {
    int r = random(180, 255);
    int g = random(50, 100);
    strip.setPixelColor(i, strip.Color(r, g, 0));
  }
  strip.show();
  delay(80);
}

// Effect: chaotic swirl (hot & humid)
void stormSwirl() {
  static int pos = 0;
  for (int i = 0; i < NUM_LEDS; i++) {
    int val = (i + pos) % 256;
    strip.setPixelColor(i, strip.Color(val, 0, 255 - val));
  }
  strip.show();
  pos++;
  delay(50);
}

// Backup effect
void rainbowCycle() {
  static uint16_t j = 0;
  for (int i = 0; i < NUM_LEDS; i++) {
    strip.setPixelColor(i, Wheel((i * 256 / NUM_LEDS + j) & 255));
  }
  strip.show();
  j++;
  delay(20);
}

uint32_t Wheel(byte WheelPos) {
  WheelPos = 255 - WheelPos;
  if (WheelPos < 85)
    return strip.Color(255 - WheelPos * 3, 0, WheelPos * 3);
  if (WheelPos < 170) {
    WheelPos -= 85;
    return strip.Color(0, WheelPos * 3, 255 - WheelPos * 3);
  }
  WheelPos -= 170;
  return strip.Color(WheelPos * 3, 255 - WheelPos * 3, 0);
}


Once the code was uploaded, the LED's started doing their thing. To get a sense of what the effects look like, I've attached a quick video that showed all the available light effects.




At this point I had a working prototype of a color thermometer and I really like the effects that are being produced. I did have to tweak the settings a little to remove some of the gaps in the temperature and humidity rules in order to cut down the cases where it defaulted to the rainbow effect. 

As of now, while the lights are pretty cool, it's basically just a lump of glowing wires, so to make it really practical I need to properly house it. 

Next - my thermometer gets a proper home. 

Tuesday, June 17, 2025

Posting ACARS Messages Online



After all the work done to set up a radio monitor to scan the ACARS frequenices, figuring out what kind of messages I wanted to see, and finally building a way to easily view the messages, the final  part of the project is to now post these messages up somewhere where I can view them almost anywhere.

 As mentioned in the previous post, I had looked into building a feed to the usual social media applications, but usually these types of feeds usually required the use of API connections to make things work. While this is pretty common within the IT world where I spend my working hours, I also realize that a good part of the world may not be familiar with the voodoo that define electronic data interchange standards, so I wanted to develop a solution that would be pretty simple to implement by someone who wasn't a tech guru. 

After some experiementing, I settled on a solution that would post my daily ACARS feed up to a fairly simple Blogger web site. The main reason for using Blogger is that it's a platform that is built to allow people to post to a personal webpage via a relativley simple user interface.  Blogger also has the ability to customize the look and feel of your webpage, so you can be creative and have your webpage suit your personal taste.  The third (and probably the most important) reason I chose Blogger was that it was free. 

Since Blogger utilizes a graphical interface for creating posts, I thought it might be interesting to try and create a solution that minics a live person creating a post. Granted, this would probably be seen as sacraligous by my IT colleagues to consider this sort of solution, but like I said earlier, I wanted to build something that a "non-techy" person might be able to build with a minimal amount of fuss.    

 With those goals in mind, I started work by creating a Blogger site called "ACARS Radio Log" <Click To Visit Site> and after doing some tinkering on my site's look and feel, I had somethng that allowed me to easily see the current messages.

With the website set up, I next looked into how to build the process of loading up ACARS messages to the site.

Obviously I wanted to look at some sort of programmed solution and since I've had pretty good luck with Python so far I wanted to still some how be able to use it for the solution's foundation. 

In sketching out a logic flow, I can up with this high level process:

  • Open a browser (I chose Chrome as the browser for my solution)
  • Log onto the Google account (since Blogger is owned by Google)
  • Open the blogger session for my ACARS Log site and click on the New Post button
  • Set the Edit window that opens up to "HTML" mode
  • Copy the contents of the "blog_ready.html" file that was created in the previous process into the Blogger Edit window
  • Save and Post the Blog Post.    

An Interactive Session

To make this work, I needed to program something that was going to work in interactive mode, which as it suggests, means that the program needs to "interact" with the computer's GUI environment, which is a bit different for Python program, which usually likes to work behnd the scenes. 

The one thing that I like the most about Python is that there seems to be a really extensive library of companion tools that seems to give it the ability do almost anything. After some research, I came across a plug in called Playwright, which is a general purpose browser automation tool. 

Doing some experimenting with Playwright, I found that it would do a great job of opening up a browser and logging onto my Blogger page. However it was having troubles navigating the editor on Blogger. Doing some investigation showed that Playwright was looking for specific html code snippets within the Blogger page itself that didn't exist in the current Blogger site. What this told me was that Blogger would occasionally make some internal changes to their site, which even if I could get it to evetually work with Playwright, it was likely a matter of time before it broke again. 

To get around that I needed to find a method that avoided the need for looking at the internal programming of the site.  After some poking around, I decided I needed to resort to a good old fashion key logger. 

Key Logging

Again, looking at the Python libraries, I did find a plugin that would play back recorded key stokes. I ended up using a plugin called Pynput. This tool will allow you to record and playback any keystrokes on a computer, which is then stored as a json file, making editting of your keystrokes pretty easy if you need to modify anything. 

The first step was to get my Blogger session to the point where I was starting to have problems with Playwright. To create the keylogger file, I needed to create a "throw away" Python to create the log:

# keylogger.py
from pynput import keyboard
import json

keystrokes = []

def on_press(key):
    try:
        keystrokes.append(key.char)
    except AttributeError:
        keystrokes.append(str(key))

def on_release(key):
    if key == keyboard.Key.esc:
        # Save to file on ESC key press
        with open("keystrokes+2.json", "w") as f:
            json.dump(keystrokes, f)
        print("Keystrokes saved. Exiting...")
        return False  # Stop listener

with keyboard.Listener(on_press=on_press, on_release=on_release) as listener:
    print("Recording... Press ESC to stop.")
    listener.join()

Executing this program turns the recorder on and I can start recording keystrokes (I noticed that it won't record any mouse activity). After I finished my keystrokes, I simply turned the recorder off by pressing the Escape key. 

After recording, I got this as a json file:

["Key.tab", "Key.tab", "Key.tab", "Key.tab", "Key.tab", "Key.enter", "Key.down", "Key.up", "Key.enter"]

After putting together some Pynput playback code, I gave it a quick test 


It certainly will do what I want it to do. 

Posting To The Blog

Now that I've sorted out the mechanics how things should work, I threw together a small Python program that follows the logic flow that I've highlighted earlier

import os
import time
from playwright.sync_api import sync_playwright
from pynput.keyboard import Controller, Key
import json
import pyperclip


# === CONFIGURATION ===
BLOGGER_URL = "https://www.blogger.com"
HTML_FILE_PATH = "blog_ready.html"
PASTE_DELAY = 1.0

# === Credentials ===
EMAIL = "<My Google Account>"
PASSWORD = "<My Password>"

if not EMAIL or not PASSWORD:
    raise ValueError("Missing EMAIL or PASSWORD")

# === Load HTML content ===
with open(HTML_FILE_PATH, 'r', encoding='utf-8') as file:
    html_content = file.read()

# === Copy to clipboard ===
pyperclip.copy(html_content)

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False, args=[
        '--disable-blink-features=AutomationControlled',
        '--incognito',
        '--disable-extensions',
        '--start-maximized'
    ])
    context = browser.new_context()
    page = context.new_page()

    # Stealth
    page.add_init_script("""
        Object.defineProperty(navigator, 'webdriver', {get: () => undefined});
    """)

    # Step 1: Google login
    page.goto("https://accounts.google.com/")
    page.wait_for_selector("input[type='email']", timeout=10000)
    page.fill("input[type='email']", EMAIL)
    page.click("#identifierNext")

    page.wait_for_selector("input[type='password']", timeout=10000)
    page.fill("input[type='password']", PASSWORD)
    page.click("#passwordNext")

    page.wait_for_timeout(5000)  # Let login settle

    # Step 2: Go to Blogger dashboard
    page.goto(BLOGGER_URL)
    page.wait_for_load_state("load", timeout=15000)
    page.wait_for_timeout(5000)

    # Click New Post
    print("Clicking NEW POST button...")
    try:
        new_post_button = page.get_by_role("button", name="New Post").first
        new_post_button.wait_for(state="visible", timeout=10000)
        new_post_button.click()
    except Exception as e:
        print(f"Failed to click NEW POST button: {e}")
    # Do not close the browser; just wait for user input
        input("Press Enter to exit and close the browser...")
        exit(1)

    keyboard_controller = Controller()

    # Load keystrokes from file
    with open("keystrokes_1.json", "r") as f:
        keystrokes = json.load(f)
 
    print("Replaying keystrokes in 3 seconds...")
    time.sleep(3)

    for key in keystrokes:
        if key.startswith("Key."):
            # Convert string back to actual Key
            try:
                k = getattr(Key, key.split(".")[1])
                keyboard_controller.press(k)
                keyboard_controller.release(k)
            except AttributeError:
                pass  # Unknown key
        else:
            keyboard_controller.press(key)
            keyboard_controller.release(key)
        time.sleep(0.05)  # slight delay to mimic real typing

    # === Wait for user to focus editor ===
    print("📋 HTML content copied to clipboard.")
    print("Switch to the editor window in the next few seconds...")
    time.sleep(PASTE_DELAY)

    # === Paste using keyboard (Ctrl+V) ===
    keyboard = Controller()
    keyboard.press(Key.ctrl)
    keyboard.press('v')
    keyboard.release('v')
    keyboard.release(Key.ctrl)

    print("✅ Content pasted.")
  
     # === Publish post ===
    try:
        print("🚀 Attempting to publish post...")

    # Look for 'Publish' button
        publish_button = page.locator('div[role="button"] span:has-text("Publish")').first
        publish_button.wait_for(state="visible", timeout=10000)
        publish_button.click()
        print("🟡 Clicked Publish button...")
        
    except Exception as e:
        print("❌ Could not publish post:", e)
    

    # Confirm Publish
    
    with open("keystrokes_2.json", "r") as f:
        keystrokes = json.load(f)
 
    print("Replaying keystrokes in 3 seconds...")
    time.sleep(3)

    for key in keystrokes:
        if key.startswith("Key."):
            # Convert string back to actual Key
            try:
                k = getattr(Key, key.split(".")[1])
                keyboard_controller.press(k)
                keyboard_controller.release(k)
            except AttributeError:
                pass  # Unknown key
        else:
            keyboard_controller.press(key)
            keyboard_controller.release(key)
        time.sleep(0.05)  # slight delay to mimic real typing

    print("✅ Confirmed Publish.")

    time.sleep(3)
    browser.close()

The program follows this flow:
  • It first logs onto my Blogger page through the associated Google account. 
  • Next it navigates to the Blogger Editor by simulating the pressing of the New Post button.
  • Once the Editor screen is open, the Keylogger playback navigates the editor by first setting the editor to html mode, and tabbing over to the edit screen. 
  • The html code that has been created in the ACARS_to_htm program is pasted to the Editor
  • The post is then published 

Putting It All Together


Now that I've got a full end to end process to collect, filter and post the ACARS messages, the last step is to put the whole thing together as a consolidated script that will run everything at once.  

To do this I created a script file called Create_post.sh on my ACARS computer:

# Run file_rename.py
python3 file_rename.py

# Check if the first script succeeded
if [ $? -ne 0 ]; then
    echo "file_rename.py failed. Aborting."
    exit 1
fi
# Run ACARS_to_html.py
python3 ACARS_to_html.py
# Check if the second script succeeded
if [ $? -ne 0 ]; then
    echo "ACARS_to_html.py failed."
    exit 1
fi
# Run ACARS_to_html.py
python3 Post_Blog.py
# Check if the second script succeeded
if [ $? -ne 0 ]; then
    echo "Post_Blog.py failed."
    exit 1
fi

echo "All scripts executed successfully."

To finish things up I then set up a crontab schedule to run this script at midnight every day. 

So far it's been running like clockwork everyday, and I love really being able to pop in anytime to check out the activity. 

So that completes the ACARS project. I'll admit that there was probably a bit more software with this one. I promise that the next project will be a bit more hands on! 

Thursday, May 22, 2025

Reading ACARS Messages



Continuing on from last month's post, now that I've set up my Linux computer and my Software Defined Radio to monitor ACARS frequencies and send the messages received every day to a log file that I can read and maybe do something a little more interesting with it.


When reviewing the daily log files, I can see that there can be literally hundreds of messages being received during a tpical 24 hour period, which is hardly surprising considering I am fairly close to a  large international airport. 

One immediate draw back is the shear volume of rather cryptic messages that I needed to wade through. 
  

A lot of it is really pretty mundane stuff like position reports, clearance approvals and system status updates with the odd free text messages.

What I would like to do is to come up with some way to filter through a day's worth of messages, filter out only the messages I care about, and post it up to a place where I can check in to see what\s been happening during the day, regardless where I may in the world - meaning I wanted to have the messages posted someplace where I can look at them online.  

To do this I initially looked at maybe posting things up to a social media site like X (i.e. Twitter). The issue with that idea is that I would need to hook into them with an API connection, which can be a bit of a hassle to set up, and depending on the service involved, may also require a paid subscription. After some playing around I determined that the best option was to set up a pretty simple blog site, which wuld allow me to use a non-API based interface, with the added bonus of being able to fully customize the look and feel of the messages.

I also didn't want to post up in "real time" just in case there may be some sort of security concerns that I might not be aware of. Based on that concern I wanted to delay any posts until at least 2 days had past since the transmission. 

So, with those ideas in mind, I needed to come up with an automated process that would:
  1. Look for the log file that my SDR process created 2 days ago
  2. Look at the file selected and extract only the messages I care about
  3. Create html code to format the messages in a format that makes them easy to read
  4. Post the html code up to a blog page.  
Since there is a fair bit of  logic needed here to make this all happen, I needed some sort of programmed solution here. In short I needed to write some code. 

Recently, I've started using Python as part of my day to day work. While I am nowhere near what I would consider proficient in it, I know enough to fully appreciate what it is able to do and it seemed to be the perfect tool to do what I want to do. 

To make it happen, I needed to break the process into steps.



Step 1 - Identifying and Selecting The Correct Message File


The first step in the process is to look for a Daily log file that was created by ACARSDEC two days ago for converting it into a temporary file that can be used for further processing. 

The basic logic for this is to look at the date tag in the Daily log file name. ACARSDEC creates the daily log files as a "Daily_YYYYMMDD.log" naming convention. For pulling the needed file, I needed my program to do the following:

  • Scan the directory that contains my log files and look for a file that has the date labelled in it's name that equals to 2 days previous than today's date. 
  • Once the file is found, create a copy of that file called "Daily.log" that we will use for further processing. 
In Python code, it looks like this:

import os
import shutil
from datetime import datetime, timedelta

def copy_log_file_if_two_days_old():
    # Get the date two days ago
    two_days_ago = datetime.now() - timedelta(days=2)
    date_str = two_days_ago.strftime('%Y%m%d')
    
    # Construct the expected filename
    source_filename = f"Daily_{date_str}.log"
    
    # Check if the file exists in the current directory
    if os.path.exists(source_filename):
        # Copy the file to 'daily.log'
        shutil.copyfile(source_filename, "Daily.log")
        print(f"Copied '{source_filename}' to 'Daily.log'.")
    else:
        print(f"File '{source_filename}' does not exist.")

if __name__ == "__main__":
    copy_log_file_if_two_days_old()


With the execution of this program, I now had a working copy of the raw ACARS data from 2 days ago. What I needed to do now was to filter out the data that I didn't want to look at and create an html file that I then see the data that I was interested in a fairly easy to read format



Step 2 - Extracting The Information and Building the Webpage


Now comes the heavy lifting.

As I mentioned at the start of this post, there is a really large volume of data that's being transmitted on a daily basis. While all interesting stuff in of itself, I was really interested in looking at any messages that were likely to be human generated. 

To figure out what sorts of messages I wanted to look at, I first captured several days worth of transmissions and tried to find some common message labels that were most likely to have been maually created. 

From my analysis I determined that the following message labels were my best candidates:
  • 84 - labelled as "S.A.S. Free Text Message"
  • 87 - labelled as "Airline Defined" - Likely Air Canada based on the aircraft tail numbers
  • 85 - labelled as "Airline Defined" - Likely Air Canada based on the aircraft tail numbers
  • 5Z (with the words "FRM ENTRY " or "DISP MSG" in the message text.  - labelled as "Airline designated downlink" - Primarily United Airlines based on the aircraft tail numbers
Once I had defined the messages I wanted to look at, it should be fairly straightforward to parse the file to pull out the required messages. 

In order to increase the readability of the message, I decided that I really only wanted to see:
  • Message Label
  • Message Number
  • Tail Number
  • Message Text 
Once I had figured out what I wanted to see and how I wanted to see it, the final step was to put the information in a format that would make it readable as a webpage - in preparation for eventually posting it to a website - which meant converting the data to html code.  

While it looks like there's a lot going, it actually translated into a fairly compact Python program where the program reads in the Daily.log file that I crreated in the last program, pulls out the records that I mentioned above and threw some html code around it before spitting it all out as a html file. 

As a result I ended up with a Python program that looked like this:

import json
from datetime import datetime, timedelta

phrase = "FRM ENTRY"
phrase2 = "DISP MSG"

# Get the date two days ago
two_days_ago = datetime.now() - timedelta(days=2)
date_str = two_days_ago.strftime("%B %d, %Y")

def read_json_records(filename, fields):
    records = []
    with open(filename, 'r', encoding='utf-8') as file:
        for line in file:
            try:
                record = json.loads(line.strip())
                if (
                    "label" in record and
                    (
                        record["label"] in ["84", "87", "85"] or
                        (record["label"] == '5Z' and phrase in record["text"]) or
                        (record["label"] == '5Z' and phrase2 in record["text"])
                    )
                ):
                    records.append({field: record.get(field, "") for field in fields})
            except json.JSONDecodeError:
                continue
    return records

def records_to_html(records, fields):
    html = f"<html><head><meta charset='UTF-8'><title>Filtered Records</title></head><body>"
    html += f"<h1>ACARS Messages {date_str}</h1>"
    for record in records:
        html += "<div style='margin-bottom: 20px; padding: 10px; border-bottom: 1px solid #ccc;'>"
        for field in fields:
            html += f"<p><strong>{field}:</strong> {record.get(field, '')}</p>"
        html += "</div>"
    html += "</body></html>"
    return html

# Example usage
if __name__ == "__main__":
    filename = 'Daily.log'
    fields = ["label", "msgno", "tail", "text"]
    json_records = read_json_records(filename, fields)

    # Save to HTML
    html_output = records_to_html(json_records, fields)
    with open("blog_ready.html", "w", encoding="utf-8") as f:
        f.write(html_output)

    print("HTML file created: blog_ready.html")

Executing the program gives me something that looks like this:


Now I am able to convert the huge mass of cryptic messages from an airplanes ACARS terminal into something that is pretty easy to understand. 

The final step in the process is to now post the day's activity up to a website that I can view anytime I want.

As I mentioned at the start of this post, I wanted to post this up to a blog page, without the need to utilize any API connections. That proved to be more of a complex process than I had expected. So because of that, I think this may be a good time to wrap it up for this month and describe how I sorted that out in my next post.

Stay tuned next month for the finale of this project. 
 

Tuesday, April 15, 2025

Receiving ACARS Signals

 

ACARS Receiver

Last month I talked a little bit about the secret world of aviation communication that involved radio signals coming from the little screen that you typically find in the middle of the cockpit of commercial aircraft. 

Now that I learned a bit about what comes out of that box, it was only natural that I should try to listen in on the transmissions that are coming from the aircraft flying nearby.  

As I also mentioned last month, I had a few Software Defined Radio (SDR) dongles sitting around that I really wanted to find a use for.

For those who may not be familiar with them, a Software Defined Radio (SDR) is a radio communication system where the components that you would have traditionally equated to physical electronic circuits and interfaces  (like mixers, filters, amplifiers, modulators/demodulators, detectors, dials, switches, etc.) are instead implemented by software on a computer. 

The big thing is that traditional radios are usually built for a specific purpose with fixed hardware tuned for specific frequencies and modulation types - which is why for example, you can't listen to cell phone conversation through your bedroom clock radio. SDRs on the other hand, digitize the signal as early as possible (usually right after the antenna), and then do all the signal processing through software.

As a result, your typical SDR looks like a very small brick that plugs into the side of your computer

Software Defined Radio
Software Defined Radio



The beauty of this arrangement is that this really opens up what you can listen to over the airwaves, your only restriction is purely based on the software that you are running on your computer. 

A while back I "inherited" a Chromebook computer. While I have nothing personally against the Chrome operating system in of itself, I found that the amount of SDR tools for Chrome a bit lacking. 

Since the Chromebook was fairly new and had some pretty powerful specs, I was wondering about perhaps installing an operating system that had a bit more of a software selection available. Since installing a version of windows looked very much like a non-starter, it wasn't until I came across the  MrChromebox.tech website where I found out that I could install Linux on my Chromebook with relatively little effort. Walking through how I did the install will probably be covered in a future post, but for now, I successfully installed a copy of MX Linux on the Chromebook and I was now ready for setting up the radio,  

Once I had my computer all set up and ready to roll, my next area of focus was to figure out what sort of software I want to use to pull in the ACARS transmissions. 

The beauty of  Linux is that there is a large library of open source software out there, a lot of which is supported by a very active community of enthusiasts. 

So it wasn't much a surprise to find a decent number of applications out there that could work with my SDR to snoop on aircraft. 

With all the choices available, I had to winnow down the list to one that had the following criteria that I was looking for:

  • Can run as a command line script (the idea here is that I could schedule my Chromebook to automatically start scanning as soon as it's booted up). 
  • Easily customizable to scan for certain ACARS message labels (this way I can only pull in the messages that I am only interested in) 
  • Can export the results of the scan out to a text file, which I can use to feed in another application for further processing   

After reviewing the options, I settled on ACARSDEC which was written by Thierry Leconte and seems to have a fairly robust support community behind it. 

This application looks to be very powerful, with the ability to monitor and report on multiple SDR's at once (a bit overkill for me, but good to know that I have lots of room for future expansion), and of course more importantly, it ticked all the boxes on what I was looking for, 

The bulk of the information out there for installing ACARSDEC seems to be primarily tailored for installation on a Raspberry Pi. That would certainly make sense since I would fully expect this to be the main platform for doing this sort of scanning, however, since I had a bit of a different set up so I found that the published instructions on installing ACARSDEC weren't working very well on my MX Live installation. 

After some playing around, I ended up with a process that worked for my particular set-up.  For my particular case I used RTL_SDR as the background SDR tool (ACARSDEC can also support Airspy and SDRPlay too) 

To install ACARSDEC on my Chromebook, I opened up a terminal session and entered the following commands:

git clone GitHub - TLeconte/acarsdec 

cd arsdec

mkdir build

cd build

cmake .. -Drtl=ON

make

sudo make install

With that done, I plugged in my SDR dongle into the Chromebook and did a quick test of the dongle by executing the RTL_TEST function at the terminal command line. Once I got the message back that the dongle was on line and receiving, I was ready to listen to airplanes.  

The next step was to craft the command that I needed to execute in order to gather the specific messages that I wanted to capture.

To do this I needed to find what the most active VHF frequencies are in my area. Luckily I am fairly close to a few airport, with the closest major one being Pearson airport near Toronto, so I should have a healthy amount of traffic to listen too. 

To determine what frequencies to use, I plugged in the known ACARS frequencies (which are pretty easy to find online) into a radio scanner that had a nice feature of counting the number of transmissions that happen on a frequency. After running the scanner for 24 hours I got a very good sense what ACARS frequencies were the most active.

To do this I had to run the ACARSDEC command at the terminal window in MX Linux. 

In a broad term - ACARSDEC is activated using this command structure:

acarsdec  [-o lv] [-t time] [-A] [-b filter ] [-e] [-n|N|j ipaddr:port] [-i stationid] [-l logfile [-H|-D]] -r rtldevicenumber  f1 [f2] [... fN] | -s f1 [f2] [... fN]

Granted this looks pretty cryptic, but these are the mainly used to define the options you want to use so that that does give you a bit of an idea that is within the application. 

For my initial test, I wanted to focus on what I could actually hear on the frequencies that I've identified from my scanner as the most active. I wanted to store what was picked up in a Daily Log file in a JSON file format. 

Basically I need to set the appropriate parameter value for:

  • Setting my output to the JSON format (-o)
  • I also don't really care about empty messages (-e)  
  • I want to output the data to a file called Daily.log
  • I also want to create a new log file daily (-D) 
  • I also need to define the where to find my SDR to scan (-r)
  • Once I know what particular messages I wanted to gather, I can define them with a filter list (-b) 
  • And I can also list the frequencies that I've found the most activity to scan
With that I entered the following command to start monitoring:

acarsdec -o 4 -e -l Daily.log -D -r 0 130.450 131.550 131.725 130.025 131.125

Since I wanted to get all of the messages at the moment, I didn't define any filters. 

After executing the command, ACARSDEC started to monitor away with no issues and I was able to collect a significant amount of aircraft data. 

Sample Output

There was one small quirk that I did uncover which usually manifested itself after the application had been running a while. Every once in a while I found that the application stopped recognizing the SDR and errored out the process. 

I wasn't able to really get to the bottom of the problem, however I found that by adding the ACASDEC command as part of the start up processes when the Chromebook reboots along with a scheduled task on the Chromebook to have it reboot every hour seemed to be a pretty good workaround. 

Now that I have establish a way to monitor and log all the activity in my area, the next step is to collect some data for a while and take a look at what kind of interesting stuff could I use for my next phase of the project.