Brian's Blog

New Network Host Alert

Mon, 17 Apr 2017 20:00:00 +0000

For a while now I’ve been wanting to know when a new host joins my home network. I’ve (rightly) been accused of being a bit paranoid at times but as the sole administrator of our family’s network really I just like to know what is going on so that I’m prepared when I get complaints about e.g. the Netflix not working. In any case, a couple weeks ago I decided to spend an hour whipping up a script to help me do this.

The Script

Here is the script.

Why Ruby?

First you’ll probably notice that the script is written in Ruby. Lately for any non-trivial script I’ve had to write I’ve chosen Ruby over Bash. It’s what I know best and almost any system I’m going to use comes with a relatively modern version out of the box. I find Ruby much more readable than Bash and then when I have to come back to tweak it I’m not wasting time brushing up on obscure Bash syntax.

Requirements

Modern Ruby version
arp-scan and nslookup installed
A working local mail server listening on port 25
Some kind of scheduling software (e.g. cron)
Root privileges

How it works

The heavy lifting is handled by arp-scan. This utility scans the local network and reports any hosts that it finds. The result is parsed by the script and a set of known MAC addresses is serialized to a YAML file so that the script can remember hosts that have been seen before. When a new host is found, nslookup is used to try to find the hostname and then the script sends an email alert to a predefined email address.

How to run it

First, the script needs to be edited to add the email address. Then the script just needs to be scheduled (e.g. via cron) and run as root because arp-scan needs to run as root. It could probably be pretty easily setup to run it as a normal user with some sudoers tricks etc. but I chose to keep it simple. Here is my crontab entry which runs the script once an hour (notice how I had to tweak the PATH so that arp-scan could be found):

# m h  dom mon dow   command
0 * * * * PATH=/usr/bin:$PATH /path/to/notify-new-hosts

That’s it!

I’ve been running this script for a couple weeks now. It’s caught me by surprise a couple times, usually when some device that hasn’t been used in a long time is powered up and joins the network. No bad surprises yet though, thankfully! Enjoy!

Forwarding Ports in OS X/MacOS

Sat, 10 Oct 2015 17:00:00 +0000

Update: I have tested this with MacOS Sierra and it works for me!

I forward local ports on my OS X machines using pfctl so that I can use traditional ports for HTTP and HTTPS with Vagrant without having to start vagrant with root privleges. Recently when I updated to OS X El Capitan (10.11) I noticed that my port forwards stopped working.

Previously I had followed some of the steps in this gist, but all of those changes had been wiped out with the upgrade. When I tried to re-apply them, a new feature called System Integrity Protection (SIP) prevented me from editing some necessary files. Also, since all of my changes had been wiped out with this upgrade, I wanted to try to keep my changes out of existing system files as much as possible in the hopes that they won’t be wiped out with the next upgrade.

Creating an anchor file

The first file we need to add is an anchor file. This defines the ports we want to forward. Create the file in /etc/pf.anchors/<CUSTOM NAME>. You can add one or many lines of the following format:

rdr pass on lo0 inet proto tcp from any to any port <SOURCE PORT> -> 127.0.0.1 port <DESTINATION PORT>

Testing the anchor file

To test the anchor file, run the following command.

sudo pfctl -vnf /etc/pf.anchors/<CUSTOM NAME>

The ports won’t actually be forwarded yet, this just checks the validity of your anchor file. If you see output that looks something like the below, with no errors, you’re good.

fctl: Use of -f option, could result in flushing of rules
present in the main ruleset added by the system at startup.
See /etc/pf.conf for further details.

rdr pass on lo0 inet proto tcp from any to any port = <SOURCE PORT> -> 127.0.0.1 port <DESTINATION PORT>

Creating a pfctl config file

Once your anchor file checks out, you need to add a pfctl config file. Create this file under /etc/pf-<CUSTOM NAME>.conf and add the following contents.

rdr-anchor "forwarding"
load anchor "forwarding" from "/etc/pf.anchors/<CUSTOM NAME>"

Testing the config file

You can start pfctl using the below command. This will forward the ports according to your rules.

sudo pfctl -ef /etc/pf-<CUSTOM NAME>.conf

To stop forwarding ports run the same command, replacing the e option with d.

sudo pfctl -df /etc/pf-<CUSTOM NAME>.conf

Forwarding ports at startup

You can use the commands above to start port forwarding on demand if you wish, otherwise if (like me) you want to forward ports automatically at startup you can create a launchctl plist file. Create a file under /Library/LaunchDaemons/com.apple.pfctl-<CUSTOM NAME>.plist with the following contents:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
     <key>Label</key>
     <string>com.apple.pfctl-<CUSTOM NAME></string>
     <key>Program</key>
     <string>/sbin/pfctl</string>
     <key>ProgramArguments</key>
     <array>
          <string>pfctl</string>
          <string>-e</string>
          <string>-f</string>
          <string>/etc/pf-<CUSTOM NAME>.conf</string>
     </array>
     <key>RunAtLoad</key>
     <true/>
     <key>KeepAlive</key>
     <false/>
</dict>
</plist>

Add the file to startup using the following command:

sudo launchctl load -w /Library/LaunchDaemons/com.apple.pfctl-<CUSTOM NAME>.plist

Example

You can find an example here that forwards port 80 to 4000 and 443 to 4001.

Credits

Hopefully this was helpful. Thanks to kujohn for creating the excellent gist that worked so well for me previously.

Automating backups of VMware Fusion VMs

Sun, 11 Jan 2015 17:00:00 +0000

I use VMware Fusion VMs for a bunch of stuff at home: Vagrant VMs, running a MythTV backend, random Linux hacking, anything you might need a Windows machine for, etc. Snapshots work great for quick, local online backups, but sometimes you want a backup that you can store offsite. This post will walk you through how to script and automate the backup of a VMware Fusion Virtual Machine to a file that can be easily transferred. Before I get into this post I want to make it clear that while I am an employee of VMware, this is a personal blog and this post is being written on my behalf, not that of my employer.

Shutting down the Virtual Machine

Per the official documentation, to take a full backup of a Fusion VM the .vmwarevm bundle must be copied while the VM is shut down. Since directory copies are non-atomic, this ensures that the VM is in a valid state. To do this, the vmrun command can be used. You can find this command inside the VMware Fusion.app bundle, typically at /Applications/VMware Fusion.app/Contents/Library/vmrun. You will also need to know the path to the VM’s vmx file.

'/Applications/VMware Fusion.app/Contents/Library/vmrun' stop '/path/to/file.vmx' soft

This command will fail if the VM is already shut down, so when putting this in a script (which we’ll be doing) it helps to check to make sure the VM is running before issuing it.

VMRUN='/Applications/VMware Fusion.app/Contents/Library/vmrun'
VMX='/path/to/file.vmx'
if "$VMRUN" list | grep --quiet "$VMX"; then
  "$VMRUN" stop "$VMX" soft
fi

Creating the backup file

Once the VM is backed up, the .vmwarevm bundle is ready to be copied. This is a good time to compress the bundle to a file to make it more portable. Note: I typically like to have my backup directory on a network volume to keep me from losing both the backup and the VM if the main disk has issues.

cd '/path/to/bundle.vmwarevm'
tar -czpf "/path/to/backup/directory/fusion-backup-`date +%Y%m%d-%H%M%S`.tar.gz" *

Since the VM must remain shut down until these files are done being copied, another option to minimize VM downtime would be to rsync the .vmwarevm bundle to an intermediate directory, start the VM, then compress the bundle.

# The following command must be run as root
rsync --archive --delete '/path/to/bundle.vmwarevm/' '/path/to/intermediate/directory' 
# Start the VM (we’ll cover this in the next section)
cd '/path/to/intermediate/directory'
tar -czpf "/path/to/backup/directory/fusion-backup-`date +%Y%m%d-%H%M%S`.tar.gz" *

The downside of this is that it can be a lot harder on your disks - since I use SSDs and I don’t really care if my VM is down for a few extra minutes I stick with the former.

Starting the VM

Similar to shutting down, we can start the VM with the vmrun utility.

'/Applications/VMware Fusion.app/Contents/Library/vmrun' start '/path/to/file.vmx'

It can be helpful to use the trap function to ensure that the VM gets started up even if something else in the script causes it to exit.

trap startvm EXIT
function startvm {
  '/Applications/VMware Fusion.app/Contents/Library/vmrun' start '/path/to/file.vmx'
}

Pruning stale backups

In order to preserve space, it is a good idea to prune old backups that are no longer needed. Note: the below assumes that the backup directory is used exclusively for backups of this VM. If for whatever reason you have other files in the backup directory this will not work as expected and may delete files that you don’t want deleted.

cd '/path/to/backup/directory'
# Delete all but the 3 most recent backups
(ls -t | head -n 3; ls) | sort | uniq -u | xargs rm

Wrapping it up into a script

Here is the full script for the above. Note: you’ll need to make the script executable to run it (e.g. chmod +x /path/to/backup-fusion.sh).

Scheduling backups

There are 2 built-in options for scheduling jobs in OS X, launchd and cron. Per the documentation, cron is deprecated so we’ll use launchd. To schedule a launchd task, create a .plist file in ~/Library/LaunchAgents/, e.g. ~/Library/LaunchAgents/com.vmware.backupfusion.plist. Check the documentation for more information on the format. The following example schedules a task to run every Monday at 3am.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
  <dict>
    <key>Label</key>
    <string>com.vmware.backupfusion</string>

    <key>ProgramArguments</key>
    <array>
        <string>/path/to/backup-fusion.sh</string>
    </array>

    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>3</integer>

        <key>Minute</key>
        <integer>0</integer>

        <key>Weekday</key>
        <integer>1</integer>
    </dict>
  </dict>
</plist>

Run the following command to add the .plist file to the scheduler.

launchctl load ~/Library/LaunchAgents/com.vmware.backupfusion.plist

Restoring from backup

Restoring from backup is pretty easy. These are just files so you can extract them wherever you want, then point Fusion at them via File --> Open....

cd /path/to/target/directory
tar -xzvf '/path/to/backup/file.tar.gz'

Setting up my blog using Jekyll and GitHub Pages

Thu, 01 Jan 2015 18:04:49 +0000

Last week I was listening to a recent Ruby Rogues podcast on Marketing Yourself as a Software Developer. During this episode John Sonmez, who runs the site Simple Programmer gives some great reasons as to why you should market yourself as a software developer, as well as some excellent ways to do that. One of the ideas was to create a blog. Sounds like a great idea to me. So, predictably, my first post is about creating my blog.

GitHub Pages

GitHub Pages is a good choice for hosting a blog for a couple of reasons:

Many developers (myself included) use GitHub daily and they will host your blog for free, so no messing around with a VPS or WordPress or anything like that
They have built-in Jekyll support, which is a great platform for blogging

The initial setup with Pages was pretty simple and only took a few minutes. You can read the guide here so I won’t bore you with the details. After pushing the initial commit to my repository and waiting a few minutes for the changes to show up, I had a working blog at f1sherman.github.io.

Custom Domain

Hosting your blog on your personal domain is a nice way to further personalize your blog. This is pretty easy and you can find instructions here. Basically it involves dropping a CNAME file into your repository and setting up the CNAME with your DNS provider.

Jekyll

Jekyll seemed like a good choice for a blogging platform because it is supported natively by GitHub Pages and supports Markdown, which I already use often. Initial setup with Jekyll was a combination of the steps listed in the GitHub Pages documentation as well as the Jekyll documentation. After a few minutes I had some scaffolding to build on.

Theme

I decided to stick with the base Jekyll theme for now, mostly because I wanted to start generating content as soon as possible and this seemed like it could be a rabbit hole. At some point I’ll probably rip it out and try something more fancy.

Google Analytics

Google Analytics is a great, free way to track the popularity of your site. I created a _includes/google-analytics.html fragment with the GA tracking code and added that to _layouts/head.html.

<head>
  <SNIP>
  {% include google-analytics.html %}
</head>

I wanted to add some social media links to my footer (in addition to the github and twitter links that are included with jekyll), but had a tough time finding SVGs that would fit in with the default theme. Eventually I stumbled upon this set of svgs which was helpful not only for the SVGs themselves, but also for the idea to get icons from iconmonstr and compress them using this SVG Editor for icons that weren’t included. See the markup on github for the code.

Comments

I decided to use disqus for commenting because it was familiar and easy to setup. I used the wizard on the disqus site to set up the commenting account, then added the code that it provided to _includes/disqus.html and included it in _layouts/post.html.

About Page

I removed the About page that Jekyll creates because I wasn’t ready to tackle this yet and I wanted to get the site up as soon as possible. I may add this back later.

Tweet Button

Adding the Tweet button resource from Twitter to the post layout will make it easy for readers to share my posts.

Sitemap/robots.txt

It’s a good idea to have a Sitemap and robots.txt to make it easier for search engines to crawl your site. I followed the Github Pages Sitemap instructions to get a sitemap setup, then added a robots.txt file pointing to it.

User-agent: *
Disallow:
Sitemap: http://blog.brianjohn.com/sitemap.xml

Submit to Google index

I used Google Webmaster Tools to submit my site to Google’s search index to make sure my site shows up in Google searches. I’ve seen this take up to a week to take affect, so don’t expect it to show up right away.

Redirect from brianjohn.com and www

I wasn’t using my domain for anything important, so I set up brianjohn.com and www.brianjohn.com to redirect to blog.brianjohn.com in my DNS provider.

Many social media sites allow you to list your site in your profile. I went through as many sites as I could think of and added my blog. Here is a list of the ones I could think of:

Facebook
Github
Google+
LinkedIn
Stack Overflow
Twitter

That’s It!

Wow, that felt like a lot of work but I had a lot of fun putting this together. Hopefully other folks with blogs (or that want to set one up) find this useful. Questions? Any other related tips or info? Leave a comment below! Thanks for reading!