If you're a regular .NET Core user and follow Microsoft's official docs on installing .NET 5 in Fedora, you might end up running into some problems with Targeting Packs for earlier .NET Core versions.

In particular, you might get the infamous:

The framework 'Microsoft.NETCore.App', version '3.1.0' was not found.
  - The following frameworks were found:
      5.0.1 at [/usr/share/dotnet/shared/Microsoft.NETCore.App]

You can resolve the problem by installing the specified framework and/or SDK.

even if you already have all of dotnet-sdk-3.1, dotnet-runtime-3.1 and dotnet-targeting-pack-3.1 installed. This (usually) happens when your installation ends up as a mix of Fedora and Microsoft-sourced packages.

What's happening

What's causing this is essentially just different installation paths: the packages in the Microsoft repo will install the runtime and all its moving parts into /usr/share/dotnet which is also where the dotnet CLI will look if it comes from a Microsoft-installed package.

On the other hand, Fedora installs its dotnet-* packages to /usr/lib64/dotnet and will look for installed SDKs and runtimes there instead. As such, the Fedora packages will only "see" runtimes/SDKs from the Fedora packages, not the Microsoft ones.

How to fix it

There's two options here: wait or reinstall from one source. At the time of writing, .NET 5 packages aren't available in the official repos for Fedora 32, so if you only want to use 3.1 you can just hang on until .NET 5 packages land in the official repos.

If you want to play with the fun stuff in .NET 5 right away though, you can install both 3.1 and 5 from the Microsoft packages and they should see each other. First, remove your existing packages:

sudo dnf remove dotnet-*

Then, install all the packages you need, only from the Microsoft repo, using dnf's disablerepo option:

sudo dnf install --enablerepo=packages-microsoft-com-prod --disablerepo=updates --disablerepo=fedora dotnet-sdk-3.1 dotnet-sdk-5.0

You should then be able to run dotnet --info and see both 3.1 and 5.0 SDKs/runtimes installed and available.

This should also resolve problems with global tools not running after upgrading to .NET 5

I was recently trying to work with a (questionably designed) REST API that had its CORS configuration set up to only allow client-side requests from their own domain, effectively limiting any API integrations to being server-side.

Fortunately, CORS is basically just HTTP response headers so if you're willing to run a proxy server (and the API owners/maintainers don't prevent it), you can always set up a reverse proxy to alter the response headers from the API and allow your own domain.

While you can use pretty much any proxy-capable web server here, I'm going to demonstrate how you can do this with YARP, Microsoft's new ASP.NET Core-based reverse proxy server. You could do this with your own server, but I'm going to use my own pre-built YARP server image.

Rather than having to build this in code, we're just going to use YARP's configuration system. In short, you just need to add the following configuration to your server's configuration file:

{
    "ReverseProxy": {
        "Routes": [{
            "RouteId": "proxy-route",
            "ClusterId": "upstream-api",
            "Match": {
                "Path": "{**catch-all}"
            },
            "Transforms": [{
                "ResponseHeader": "Access-Control-Allow-Origin",
                "Set": "https://yourchosendomain.com"
            }]
        }],
        "Clusters": {
            "upstream-api": {
                "Destinations": {
                    "upstream-api/public": {
                        "Address": "https://api.upstream.com/"
                    }
                }
            }
        }
    }
}

Obviously replace your own domain (line 11) and your upstream API address (line 18).

You can also just set the ResponseHeader value to * to allow all, but I don't recommend that!

You can then just use whatever path you were using with the upstream API (/v1/users/info or whatever) with your own domain and YARP will automatically replace the Access-Control-Allow-Origin header in the response, allowing your client.

If you're using my Docker image, just bind-mount that JSON into the container's /app/reverseProxy.json

Author's Note: This post has been sitting in draft state since my original post was published, so some details might be a little out of date. I'm publishing it anyway and hoping its helpful for someone!

I recently published a post outlining how to use QEMU to emulate a Raspberry Pi. In that post, we got things working, but not much more than that. Today, I'm covering a few post-install tweaks you might want to work on.

Display

When you run the commands from the previous post you'll see QEMU pop up a UI window with the Pi's display output. While this can be handy if you're constantly interacting with the UI, I find it quite irritating and would prefer an option to disconnect and reconnect to the UI without resetting the Pi. To that end, we can tell QEMU to serve the Pi's raw display output over a VNC connection instead!

By adding a simple -vnc :5 to the command, we tell QEMU to serve the vPi's display over a local-only VNC server on port 5905 (port 5900 + :5 = 5905). You can connect to the session with any VNC-compatible client (such as GNOME Remote Desktop Viewer). Note that since QEMU is the server here (not the vPi), the server is active during boot as well!

qemu-system-arm \
## trimmed for brevity
-net tap,ifname=vnet0,script=no,downscript=no \
-vnc :5 # <- this is the important part!

Scripting

The main thing is that every time you wanted to start the virtual Pi (vPi) you would need to run the same lengthy qemu-system-arm command, and it would block the prompt while doing so). Fortunately, there's a better way!

In my case, I went with a combination of a simple bash script and GNU Screen, a handy utility for running detachable terminal sessions in the background.

So, I created a small bash script (called vpi):

#!/usr/bin/env bash
cd /path/to/work-folder # where your kernel and disk image are!
qemu-system-arm \
## TRIMMED FOR BREVITY

and added it to my PATH. Combined with the trick above, running this script (you may need chmod +x first) will give you a fully backgrounded running vPi.

While this is effective, it still blocks the current console and will kill the vPi if we close it. Instead, I'll preface the whole command with screen -dmS vpi. Thus, the full script will be:

#!/usr/bin/env bash
cd /path/to/work-folder
screen -dmS vpi qemu-system-arm \
-kernel ./kernel-qemu-4.4.34-jessie \
-cpu arm1176 -m 256 -M versatilepb \
-no-reboot -serial stdio \
-append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
-hda raspbian.qcow \
-net nic -net user \
-net tap,ifname=vnet0,script=no,downscript=no \
-vnc :5

Now, run chmod +x vpi and run vpi. You'll get a screen session called vpi started in the background that spins up the machine. You can then attach (with screen -r vpi) and detach (with Ctrl-A D) at will to get the serial console, and connect via VNC (on port 5905) at any time.

If you followed the instructions in the last post, you can also run ssh pi@192.168.122.200 to quickly SSH into your new Pi.

Permissions

Note that you might run into some wonky permissions around the /dev/net/tun device if you try and run these scripts not as root. The details of fixing this are a bit complex to get into here, but essentially come down to:

• Give your user access to or ownership of /dev/net/tun (I'd recommend a udev rules file)
• Add your user/group to the device with ip using sudo ip tuntap add dev vnet0 mode tap group <group-here>
• Run the machine, making sure to include -net user

This process can be a little glitchy but does take out the need to run as root.

TL;DR: I published a simple CLI tool to manage switching between complex Git configs. You can find all the docs and details here, the source here and download it here

I've recently published a new command line utility: Git Profile Manager.

Git Profile Manager (GPM) was originally intended purely as a demo for .NET Core native publishing during a conference presentation. As it happened, the tool worked a little better than I thought so I kept hacking on it so I published it earlier this year for anyone who finds it useful!

What does it do?

Git Profile Manager, as the name suggests, manages Git profiles. What that means is collections of Git configuration options. As a practical example, I work on a lot of different Git projects, and many of them have different configuration requirements:

• Cake: Sign all commits with Keybase key, commit with cakebuild.net address
• Other GitHub projects: sign with Keybase key, commit with personal email, whole suite of aliases
• Work projects: sign with internal key and internal address, no auto CRLF detection, custom commit message template

Likewise, I use different diff tools for different projects, I only use aliases for some complex projects, and projects using submodules need some extra config.

Now, I could sit there and run git config commands for ages (used to do that), or write up project-specific shell scripts (recently did that), but now I can simply create "profiles" with GPM that contain a set of configuration options for each project or type of project. Now, when I clone a new work project, I just run:

gpm activate work-profile

and the repo is immediately configured with my work keys and our team's customised Git configs.

On the same workstation, I can start a new Cake project, run gpm activate cake and now I'm using my Cake configuration (including my Keybase key for signing commits).

Plus, since you can activate multiple profiles at the same time, I can quickly run gpm activate submodules and add submodule-specific config without affecting the rest of the repo. I can also take my existing shell scripts and use gpm profile import /path/to/script.sh and convert them easily into profiles that are stored in my home folder.

GPM supports a whole host of commands for easily activating, deactivating and manipulating profiles, and they're all detailed in the documentation.

How do I get it?

The easiest way (in my opinion) is as a .NET Core global tool: just run dotnet tool install -g git-profile-manager and gpm should be ready to go.

Native packages are also available for Windows as well as a range of Linux distributions (Ubuntu, Debian, RHEL, CentOS).

You can also download the latest release and manually extract it to your PC, if you don't want to install it!

This post was written on, and tested with, Fedora 30. Other distributions or releases might be a little different.

While plenty of users probably haven't used SELinux much, most will only know it as a security framework to prevent misbehaving apps and enforcing access control in Linux. If you get into the meat of it though, you can use its sweet security controls to do all sorts of Fun™ things.

Today I'm going to show you how to use SELinux to run a clean Firefox session in a sandbox.

The boring bits

SELinux is a maddeningly complex bit of technology. It's really powerful, boasts some really awesome design and is something you'll really want to learn if you want to up your security (or Linux) skills. That being said, the finer details of users, roles, types, sensitivities and categories are way beyond this post, and you don't really need to understand everything to get started!

Playing with your sandbox

The crucial command behind all of this is the sandbox command. On most SELinux-enabled systems (and certainly on Fedora), sandbox will be installed by default.

As the name implies sandbox is used to run any command in a fully user-controllable SELinux-enforced sandbox. This allows you to isolate commands/applications from the rest of your system and only grant specific permissions and capabilities. Even better, SELinux includes a handful of prebuilt sandbox types that allow access to certain critical resources.

After you install one dependency (policycoreutils-sandbox), you can even run X apps in an SELinux sandbox!

The good bit

With that context, out of the way, here's the magic command for running Firefox in an SELinux sandbox on Fedora:

sandbox -X -t sandbox_net_t -t sandbox_web_t -w 1280x1024 firefox

This runs a sandbox with its own X server (-X), allows ports required for web browsing and general network access (-t sandbox_web_t and -t sandbox_net_t) and launches firefox in a 1280x1024 window (-w 1280x1024).

This will open up a new window with a completely clean instance of Firefox that is isolated from the rest of your processes by SELinux. Note that this also means you won't be able to access any of your files (including your Firefox profile) so you will get a completely fresh session every time.

Next steps

If you find this handy, you might want to try extending this to other apps that you use where you might want to test things in clean environments or are handling files you don't entirely trust. For example, here's how to open a report.pdf from my home directory in a sandboxed PDF viewer:

sandbox -X -w 1280x1024 -i ~/report.pdf evince report.pdf

Since these commands are unwieldy and the options are app-specific, I recommend setting up some aliases for anything you use often.

There's quite a few ways to achieve sandboxed X apps these days! If SELinux/sandbox isn't right for you, you might want to check out LXC/LXD, Docker (especially with subuser) or Podman.

Now go forth and sandbox all the things!

You should also be aware that sand is coarse and rough and irritating and it gets everywhere.

Recently at work, I had a very strange problem with an NGINX reverse proxy I was using: it was truncating static files at seemingly arbitrary places. To this day, I don't know what caused it.

On the plus side, a colleague of mine (thanks Mark!) pointed me towards an alternative I could test with: nghttp2.

To be clear: nghttp2 is not a complete NGINX replacement. It does, however, make for a stunningly easy HTTP/2 compliant reverse proxy alternative.

The nghttp2 project actually includes quite a few components: a client (nghttp), server (nghttpd) and a proxy (nghttpx). I'm only looking at the proxy today!

Previously, using NGINX as a reverse proxy required a slightly obtuse (but very powerful) config file and restarting the nginx server. Using nghttpx allows you to directly run the server on-demand, then easily swap it to a system service later.

Running the proxy

Once you've got it installed (packages are available in EPEL for RHEL/CentOS, or in the mainline repos for Fedora), you can run the proxy component directly with the nghttpx command. The syntax can be a little dizzying at first, but is super-easy to tweak and adjust. For example, here's the command my app ended up needing:

nghttpx -s -f'*,443' -b127.0.0.1,9442 /etc/ssl/cert.key /etc/ssl/cert.pem

That starts up a HTTP/2 proxy (-s) on port 443 (-f'*,443') proxying a local HTTP app on port 9442 (-b127.0.0.1,9442) using the cert and key from /etc/ssl. If your upstream is HTTPS-enabled, just change it up slightly:

nghttpx -s -f'*,443' -b'127.0.0.1,9442;;tls' /etc/ssl/cert.key /etc/ssl/cert.pem

The man page includes a lot of info on the various ways to use the -b/--backend option!

It also supports the same -k behaviour as curl if your upstream is using a self-signed cert.

Running as a service

To run it as a service, you'll want to use a config file. Like many other tools nghttpx uses a key-value config file that reflects the same options as the command line switches. Here's the above command as a config file:

frontend=0.0.0.0,443
backend=127.0.0.1,9443;/;tls
insecure=yes
private-key-file=/etc/ssl/cert.key
certificate-file=/etc/ssl/cert.pem

add-x-forwarded-for=yes
accesslog-file=/var/log/nghttpx/access.log
errorlog-file=/var/log/nghttpx/error.log

You may have to create the directories for your log files before starting the service.

Put that config file in the default location of /etc/nghttpx/nghttpx.conf and you're one easy command away from running proxy:

systemctl start nghttpx.service
# or for a start-at-boot server:
systemctl enable --now nghttpx.service

Seriously, if you need to use a reverse proxy for anything you're running (non-production-grade apps, containers, older servers etc), take a look at nghttp2: there's plenty of docs on GitHub and the man page is really detailed and easy to understand.

For anyone who doesn't realise: HTTP/2 is SSL/TLS-only so make sure you've got a cert before you try and jump on the new hotness!

This post is just a quick documentation of a fix I found the hard way.

TL;DR:
SDK Resolver folder exists but without an SDK Resolver DLL or manifest file. This may indicate a corrupt or invalid installation of MSBuild

might be fixed by

sudo rm -r /usr/lib/mono/msbuild/15.0/bin/SdkResolvers/NuGet.MSBuildSdkResolver

(or equivalent mv). YMMV.

After updating both VS Code and the .NET Core SDK on my Fedora 29 machine, VS Code couldn't start OmniSharp correctly and wouldn't load .NET Core projects properly. The error code was a a rather cryptic one:

SDK Resolver folder exists but without an SDK Resolver DLL or manifest file. This may indicate a corrupt or invalid installation of MSBuild

This was pretty surprising given its a "standard" dnf install from the official MS repos and to be honest, I still don't actually know for sure what caused this issue (but you might see my guess if you keep reading)

Regardless, the VS Code logs weren't giving me any information on my resolvers for some reason. I could see the problem was not specific to VS Code and was actually coming from MSBuild for some reason.

Thankfully the Internet delivered in the form of two GitHub issues and a Developer Community post from which I could piece together the following:

1. that it could be fixed by reinstalling the VS Build Tools package
2. that the problem file (on Windows) was C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\15.0\Bin\SdkResolvers\NuGet.MSBuildSdkResolver\NuGet.MSBuildSdkResolver.dll.new

Given #1 is completely irrelevant on Linux, I followed the rabbit hole of #2 and one simple locate command later I could see that there were two different MSBuild resolver locations: /usr/share/dotnet/sdk/<version>/SdkResolvers/NuGet.MSBuildSdkResolver and /usr/lib/mono/msbuild/15.0/bin/SdkResolvers/NuGet.MSBuildSdkResolver.

I don't know enough about the internals of MSBuild to tell you why but simply removing the entire /usr/lib/mono/msbuild/15.0/bin/SdkResolvers/NuGet.MSBuildSdkResolver directory and restarting OmniSharp led to a successful project load!

If you're new to systemd you're probably thinking that the title sounds like the sort of thing that should be easy, and you're right! If you're used to systemd, you'll already know that it's probably not easy, and you're also right!

On the surface, you should be able to just add your entries to /etc/fstab as you've been doing up until now and if that's working for you: stick to it!

If, however, you either a) are having problems with mounts being attempted before your network is up (common with WiFi systems) or b) want to do everything the full systemd way, then read on!

The Mount File

Systemd includes a helpful mount unit type, handled as a .mount unit file (just like .service etc). As of the time of writing, this file actually just gets 'executed' through the usual mount command anyway, but let's have a look at what a mount file looks like.

Check out James Oguya's excellent post on this topic for the full details.

[Unit]
Description = Mount NFS Share

[Mount]
What=172.24.0.5:/srv/backups
Where=/mnt/backups
Type=nfs
Options=defaults
# Uncomment the below if your server is real slow
# TimeoutSec=10

[Install]
WantedBy=multi-user.target

and that's it! Sort of.

Put this file in /etc/systemd/system like your other system units, but be warned: the file must be named exactly for its target mount point. In the example above, the file would be /etc/systemd/system/mnt-backups.mount.

Don't ask me why, I don't know either.

Then run systemctl daemon-reload and systemctl start mnt-backups.mount to mount your filesystem. Now to mount at boot, you would think you can just systemctl enable ... but unfortunately it's not that simple.

Putting a boot in it

Now if you just run systemctl enable ... with your unit as it stands, it will almost certainly fail since systemd won't know to wait for the network before it actually runs the mount.

You can control this using the After= option in the [Unit] section:

[Unit]
Description = Mount Server Private Directory
After=## this bit here ##

[Mount]
...

Now your first thought will be to use the "special" remote-fs.target here and this might work in simple setups. If your system is using Network Manager however, the accepted wisdom seems to be to use After=NetworkManager-wait-online.service as your dependency. This will usually work for wired connections, so try this first.

There's a bit of a caveat here though:

NetworkManager doesn't actually wait

If you check the /usr/lib/systemd/system/NetworkManager-wait-online.service file yourself, you'll notice the service is just using nm-online to wait for the network. However, if you check the arguments being used, you'll see that using the -s flag here only waits for NetworkManager itself to be ready, not the actual connection to be connected and ready. While you can simply remove the -s flag here (see notes below), I find it better to create a new service.

Creating a service that actually waits

In my case, I created the following file at /etc/systemd/system/network-online.service:

[Unit]
Description=Wait until NM actually online
Requires=NetworkManager-wait-online.service
After=NetworkManager-wait-online.service

[Service]
Type=oneshot
ExecStart=/usr/bin/nm-online -q --timeout=120
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

This will run after the existing NetworkManager-wait-online.service but will only successfully start (and stay active) once Network Manager has actually connected to the network.

Run systemctl daemon-reload and systemctl enable --now network-online.service to confirm your service is working and enable the check to run at boot

Adding this to your mount

Now you have this service, you can slightly tweak your existing .mount file with the following lines in the [Unit] section:

Requires=network-online.service
After=network-online.service

Now your mount won't be run until NetworkManager reports your network as actually connected.

Putting it together

You can actually use this new network-online.service as a After= criteria on any other services you have that should wait until the network is actually connected, instead of just having the stack up.

Notes

What about if I'm not using NetworkManager?

Fair point actually. The overall principles at work remain the same: create a service with a command that will only return a success code when the network is available and have your mounts depend on that. You'll just have to find the right system service to use as an After= substitute for NetworkManager-wait-online.service and find a command you can use to replace nm-online.

What about removing the -s flag in the builtin unit?

So this is a valid approach with two important caveats:

• You might break existing stuff. In my non-rigorous testing, boot behaviour actually got worse when I made this change and there were a few services that seemed to have trouble starting/waiting for the modified service.
• DO NOT modify the builtin service file. If you're on a vaguely recent release, you should be able to create a /etc/systemd/system/NetworkManager-wait-online.service.d/ directory and put .conf files in there to override the built-in one. Check the docs or the better unofficial docs

Can't you do this in /etc/fstab?

Yes and no, in my experience. Yes, /etc/fstab is both the easiest and most Linux-y way of doing mounts, but it's also unintuitive, error-prone and doesn't integrate well with things like NetworkManager/netctl/whatever. Personally, I'd prefer to whip up an easy .mount file then try and decipher whatever x-systemd.automount means, and which column of this plaintext file it should be in.

Why not use automount?

This is the big one: the answer to the vast majority of problems with NetworkManager/systemd/etc and mount delay is to use "automounts" which are essentially lazy filesystem mounts, that are auto-mounted the first time they're accessed. For the most part, these do solve a few of the problems from above. That being said, if you want your boot to actually wait for the filesystem rather than leave it until first access, you will need to use this method instead.

As great as the ANAVI Light Controller is out-of-the-box, you might want to either customise the firmware that's factory-loaded or add your own behaviour to the Light Controller.

Note that these instructions are using the Arduino IDE on Linux. You can also use PlatformIO, but that's not covered here. This guide is basically a prologue for Leon's excellent guide on flashing the Controller.

Background and Setup

Before you get too far, make sure you have a feel for what you're working with. The Light Controller is an ESP-12E module wired up with 3 pins for each RGB channel, one pin for the button (marked SW1) and one for the LED (marked D1). You'll find the 3 UART pins just below the 3 I2C sensor slots, and the acrylic case even includes convenient cutouts for both the button and UART pins.

Setting up your environment is pretty easy. First, check your USB UART adapter is correctly recognised (as /dev/ttyUSB0 on my Fedora system) and it's strongly recommended you allow your user access to the device without sudo.

On Fedora, this can be done by adding your user to the dialout group (with sudo usermod -aG dialout $USER) then logging out and back in again.

Finally, make sure you have installed the Arduino IDE. On most Linux distros (especially convenient on Fedora) you can use the Flatpak package from Flathub or you can manually install it.

Get the code

If you're building your own firmware from scratch, just use your own sketch file and skip this section. For everyone else, you can use either the stock firmware file or the much simpler blinking LED demo as a starting point. Either download the file directly or fork and clone the repo with Git to get the .ino file.

Finally, open it up in Arduino IDE!

Set up Arduino IDE

To build and flash the ANAVI Light Controller, you will need to set up the Arduino IDE to suit the Controller.

Add the board

1. First, open the File -> Preferences menu and add the following URL to the "Additional Boards Manager URLs" field: http://arduino.esp8266.com/stable/package_esp8266com_index.json

2. Close the Preferences window and open the Tools > Board > Boards Manager... menu to open the Boards Manager. Once the index has updated, use the filter field to find the esp8266 board and install the latest version.

Enable the flash storage

If you want the onboard flash storage to be available, you also need to use the Tools > Flash Size menu to add SPIFFS. Personally, I use the 4M (1M SPIFFS) option and haven't had any issues.

Check the port

Finally, check the port in the Tools > Port menu to make sure you're using your USB UART adapter (usually /dev/ttyUSB0)

Build and Verify

Install the dependencies

Firstly, you need to install your sketch's dependencies. This will be different if you're using your own sketch! If you're using the blinking LED sample, you can skip this step entirely, but if you're building from the stock firmware, keep reading.

Use the Sketch > Include Library > Manage Libraries... menu to open the Library Manager. You will need to install several libraries, so work down the following list, installing the version shown:

• DNSServer (1.1.0)
• ESP8266WebServer (1.0.0)
• WiFiManager (by Tzapu, 0.14.0)
• ArduinoJson (5.13.2, not 6.x)
• PubSubClient (by Nick O'Leary, 2.6.0)
• Adafruit APDS9960 Library (search for adafruit_apds, 1.0.5)
• Adafruit HTU21DF Library (search for adafruit_htu, 1.0.1)

Build the sketch

Once you have the dependencies installed and your sketch is ready, just click the check mark button in the top left to build and verify your sketch. Watch the status window at the bottom to make sure there's no errors in building your sketch.

You can increase the verbosity of the build window using the Show verbose output during: option in the Preferences window

Flash the board

This bit is much better explained by Leon, so watch the nice and easy-to-follow video from Leon himself below:

Monitoring your Controller

If your sketch prints to the serial console at all (both of the Anavi-provided sketches do out-of-the-box), you can monitor this by opening a terminal window and running screen /dev/ttyUSB0 115200.

Note that screen will own the port while this is running so you will need to close the session (press Ctrl-A and type :quit) before you can upload another sketch using the IDE.

TL:DR;

light:
  - platform: mqtt_json
    command_topic: "cmnd/<device-id>/color"
    state_topic: "stat/<device-id>/color"
    brightness: true
    rgb: true

Background

I recently received my ANAVI Light Controller in the mail and was excited to try it out. The easiest way to get set up is to follow Leon's excellent intro video (included below).

While you're connecting to your WiFi network, you'll have the option of setting your own MQTT broker, and here make sure here you change it from the default eclipse.org broker to the broker your Home Assistant install is using.

As the interface warns you, make certain that you grab the device ID (for future reference, the device ID is the MD5 sum of your Chip ID).

Checking your controller

If your MQTT broker also supports a Websockets connection, you can also use the demo.anavi.technology website, but if you're using a local/self-hosted MQTT broker, that's probably not the case. If you really want to make sure, you can always fire up MQTTSpy, connect to your broker and watch the messages. You can add a subscription for # to see all messages on the broker.

Controlling your Controller

If you just want to add your Controller to Home Assistant, jump down to the next topic.

The Controller uses the following MQTT topics:

Command Topics

There are two command topics:

cmnd/<device-id>/power: This is the simple on/off switch topic. Publishing a message with a payload of either "ON" or "OFF" will turn the strip on or off.

cmnd/<device-id>/color: This is the more advanced topic and can even replace the normal /power topic. This topic expects a simple payload with the following structure:

{
    "state": "ON",
    "brightness": 255
    "color": {"r": 255, "g": 0, "b": 0}
}

You can quickly set the state, brightness and color in one hit using this topic, or any of the three keys individually (which is also what Home Assistant will do).

State Topics

The controller will automatically publish a status message to the two topics below whenever it acts on an incoming MQTT message.

stat/<device-id>/power: Just like its cmnd equivalent, messages on this topic will have their payloads set to the string "ON" or "OFF" to reflect the current state.

stat/<device-id>/color: Again, this topic works much like its cmnd equivalent, publishing the same JSON payload (with one exception) to reflect the current LED state.

Adding the Controller to Home Assistant

To add your controller to Home Assistant, just add the following YAML to your configuration.yaml, substituting your device ID from the earlier step (and changing the name to match your lights):

light:
  - platform: mqtt_json
    name: Lounge LEDs
    command_topic: "cmnd/<device-id>/color"
    state_topic: "stat/<device-id>/color"
    brightness: true
    rgb: true

You should then be able to restart your Home Assistant and you will have a new entry in your Home Assistant "Lights" panel.

Bonus Round

Every single MQTT payload, complete with which topic it's published on, is actually also written directly to the serial console, so even if you are having trouble with your broker, you can watch the console. This output even includes all the configuration details such as the broker details, and any user/password configured (if you bought the Starter or Advanced Kits, you will have received a USB UART adapter in the kit).

If you're building software for the Raspberry Pi (like I sometimes do), it can be a pain to have to constantly keep Pi hardware around and spotting Pi-specific problems can be difficult until too late.

One option (and the one I most like) is to emulate a Raspberry Pi locally before ever hitting the device. Why?

• Works anywhere you can install QEMU
• No hardware setup needed (no more scratching around for a power supply)
• Faster feedback cycle compared to hardware
• I can use Pi software (like Raspbian) in a virtual context
• I can prep my "virtual Pi" with all the tools I need regardless of my physical Pi's use case

Given I'm next-to-useless at Python, that last one is pretty important as it allows me to install every Python debugging and testing tool known to man on my virtual Pi while my end-product hardware stays comparatively pristine.

Getting started

First, you'll need a few prerequisites:

QEMU (more specifically qemu-system-arm)

You can find all the packages for your chosen platform on the QEMU website and is installable across Linux, macOS and even Windows.

Raspbian

Simply download the copy of Raspbian you need from the official site. Personally, I used the 2017-08-16 version of Raspbian Lite, since I don't need an X server.

Kernel

Since the standard RPi kernel can't be booted out of the box on QEMU, we'll need a custom kernel. We'll cover that in the next step.

Preparing

Get your kernel

First, you'll need to download a kernel. Personally, I (along with most people) use the dhruvvyas90/qemu-rpi-kernel repository's kernels. Either clone the repo:

git clone https://github.com/dhruvvyas90/qemu-rpi-kernel.git

or download a kernel directly:

curl https://github.com/dhruvvyas90/qemu-rpi-kernel/raw/master/kernel-qemu-4.4.34-jessie

For the rest of these steps I'm going to be using the kernel-qemu-4.4.34-jessie kernel, so update the commands as needed if you're using another version.

Filesystem image

This step is optional, but recommended

When you download the Raspbian image it will be in the raw format, a plain disk image (generally with an .img extension).

A more efficient option is to convert this to a qcow2 image first. Use the qemu-img command to do this:

qemu-img convert -f raw -O qcow2 2017-08-16-raspbian-stretch-lite.img raspbian-stretch-lite.qcow

Now we can also easily expand the image:

qemu-img resize raspbian-stretch-lite.qcow +6G

You can check on your image using the qemu-img info command

Starting

You've got everything you need now: a kernel, a disk image, and QEMU!

Actually running the virtual Pi is done using the qemu-system-arm command and it can be quite complicated. The full command is this (don't worry it's explained below):

sudo qemu-system-arm \
-kernel ./kernel-qemu-4.4.34-jessie \
-append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
-hda raspbian-stretch-lite.qcow \
-cpu arm1176 -m 256 \
-M versatilepb \
-no-reboot \
-serial stdio \
-net nic -net user \
-net tap,ifname=vnet0,script=no,downscript=no

So, in order:

• sudo qemu-system-arm: you need to run QEMU as root
• -kernel: this is the path to the QEMU kernel we downloaded in the previous step
• -append: here we are providing the boot args direct to the kernel, telling it where to find it's root filesytem and what type it is
• -hda: here we're attaching the disk image itself
• -cpu/-m: this sets the CPU type and RAM limit to match a Raspberry Pi
• -M: this sets the machine we are emulating. versatilepb is the 'ARM Versatile/PB' machine
• -no-reboot: just tells QEMU to exit rather than rebooting the machine
• -serial: redirects the machine's virtual serial port to our host's stdio
• -net: this configures the machine's network stack to attach a NIC, use the user-mode stack, connect the host's vnet0 TAP device to the new NIC and don't use config scripts.

If it's all gone well, you should now have a QEMU window pop up and you should see the familiar Raspberry Pi boot screen show up.

Now, go get yourself a drink to celebrate, because it might take a little while.

Networking

Now, that's all well and good, but without networking, we may as well be back on hardware. When the machine started, it will have attached a NIC and connected it to the host's vnet0 TAP device. If we configure that device with an IP and add it to a bridge on our host, you should be able to reliably access it like any other virtual machine.

(on host) Find a bridge and address

This will vary by host, but on my Fedora machine, for example, there is a pre-configured virbr0 bridge interface with an address in the 192.168.122.0/24 space:

virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
        ether 00:00:00:1e:77:43  txqueuelen 1000  (Ethernet)

I'm going to use this bridge and just pick a static address for my Pi: 192.168.122.200

Reusing an existing (pre-configured) bridge means you won't need to sort your own routing

(in guest) Configure interface

NOTE: I'm assuming Stretch here.

Open /etc/dhcpcd.conf in your new virtual Pi and configure the eth0 interface with a static address in your bridge's subnet. For example, for my bridge:

# in /etc/dhcpcd.conf
interface eth0
static ip_address=192.168.122.200/24
static routers=192.168.122.254
static domain_name_servers=8.8.8.8 8.8.4.4

You may need to reboot for this to take effect

(in host) Add TAP to bridge

Finally, add the machine's TAP interface to your chosen bridge with the brctl command:

sudo brctl addif virbr0 vnet0

Now, on your host, you should be able to ping 192.168.122.200 (or your Pi's address).

Set up SSH

Now, in your machine, you can run sudo raspi-config and enable the SSH server (in the "Interfacing Options" menu at time of writing).

Make sure you change the password from default while you're there!

Finally, on your host, run ssh-copy-id pi@192.168.122.200 to copy your SSH key into the Pi's pi user and you can now SSH directly into your Pi without a password prompt.

This is part of a series of posts on the new features added in Downlink 0.2! Check out this post for more or hit the docs.

So all week we've been looking at all these cool new features available in Downlink like all the new extension points or the flexible routing behaviour, but a lot of it has required getting your hands dirty in your app's startup code and handling things like dependency injection in multiple places.

If that all seems a bit messy to you, you're quite right. To that end, Downlink has it's own powerful plugin system!

What's in a plugin?

So a plugin, in Downlink-land, is just a type (often in it's own assembly) that can add new features or behaviours to Downlink in one easy motion. In fact, plugins in Downlink are essentially just a package/wrapper around all the cool new features we've been discussing this week.

Plugins allow you to ship, for example, support for a whole new storage service, including it's own storage backend, scheme client, and supported pattern matchers, all in one easy-to-consume package.

Plugins are so handy that most of the moving parts included in Downlink out of the box are implemented as plugins!

Using plugins in your app

Since plugins are a package of extension points, they need just one change to add to your app:

// in Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().AddDownlink(d => d.UsePlugin<MyPlugin>());
}

The MyPlugin plugin will then be loaded, added to your app, and any services it provides will be available in the app.

Plugins in the Docker image

If you want to use plugins without running your own host, you can also take advantage of highly experimental support for loading plugins from the local folder!

You can see how to get started with local plugin discovery in the online docs.

Building plugins

If you want to build a plugin for Downlink, it's pretty simple:

• Optionally, create and reference a class library
• Add a class that implements IDownlinkPlugin
• Add it to your startup code

That's it! Restart the app and your plugin will get invoked, your services will be added to the app container and you're good to go!

The technical details

Downlink's plugin system is essentially a wrapper over the existing DI-powered extension points. The IDownlinkPlugin interface just provides a single "hook" method to add and configure services with direct access to the DI container (via the IDownlinkBuilder) that makes registering services easier.

Since they're much simpler, quite a few of Downlink's existing moving parts like the routing, context and all of the default services are implemented as plugins that get loaded first.

Since plugins themselves are resolved out of DI, you can also use your plugin's constructor to import required services (example). It's the IPluginLoader/PluginLoader that does the actual loading.

Summary

That's it for this week's journey through Downlink 0.2's new features! Hopefully you'll find some cool new uses for Downlink and get a chance to try out all it's new features.

As always, you can find help in the docs, on GitHub, or on Gitter.

This is part of a series of posts on the new features added in Downlink 0.2! Check out this post for more or hit the docs.

On Tuesday, we saw how you can now self-host Downlink in an existing (or new) ASP.NET MVC Core app. However, Downlink's routing pattern is quite aggressive and you might find it kicking in for routes you'd prefer it didn't.

While before the rebuild, you had very little control over Downlink's internals, if you're self-hosting, you have much more power!

In fact, you can now easily add a route prefix to Downlink through no less than 3 different methods:

1. Configuration

The easiest method is to define the DownlinkPrefix configuration key (using a config file, or environment variable). When you do so, that prefix will be automatically added to Downlink's routes, no further changes required!

Since this is part of the default setup, this configuration also works the same for the Docker image!

Add the prefix to your configuration, such as 'download':

{
  "DownlinkPrefix": "download"
}

Now, instead of a request to yourapp.com/v1.2/windows/x64 triggering your download, you'll want yourapp.com/download/v1.2/windows/x64. It's that simple!

2. Code

If you're self-hosting you can also easily add a new route prefix in the startup code of your app. Use the fluent overload of the AddDownlink() method to call the UseRoutePrefix method. For example, to add the same 'download' prefix we used above:

// in Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().AddDownlink(d => d.UseRoutePrefix("download"));
}

Now this works pretty well for simple "add a string to the start" routing scenarios, but there's another much more powerful option!

3. Extension point

We saw yesterday that Downlink 0.2 now supports extensibility for most of it's core and internal functionality. Fortunately, that even extends to the routing behaviour!

Internally, Downlink attempts to resolve an instance of IRoutePrefixBuilder to determine what route prefix (if any) to apply to Downlink routes when the app starts. In fact, the two methods above are using those same extension points (they're ConfigurationRoutePrefixBuilder and StaticRoutePrefixBuilder, respectively).

So, to control the routing prefix more directly, you can also use the builder to add your own prefix implementation with whatever logic you like. For example:

// in Startup.cs
public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().AddDownlink(d => d.UseRouteBuilder<MyAwesomeBuilder>());
}

Then, Downlink will call your builder's GetPrefix method when the app starts to get the prefix to prepend to Downlink's routes

The technical details

The IRoutePrefixBuilder that we use to resolve the route prefix (see above) is a very simple interface just used to get a prefix. The actual routing changes are done in the DownlinkRouteConvention. This type makes use of ASP.NET Core's new IActionModelConvention to apply an app-wide "convention" to any actions.

The default convention (which you can replace if you're feeling very adventurous) just goes through the actions in the DownlinkController and merges the prefix we resolved into the existing attribute routes to create our new route table entries.

As you can see, Downlink's routing is now much more flexible and puts the control back in your hands! Check back tomorrow for more posts on new features in the rebuilt Downlink 0.2!

This is part of a series of posts on the new features added in Downlink 0.2! Check out this post for more or hit the docs.

While hosting Downlink (as we discussed yesterday) might be the big-ticket change in the rebuilt version, it wouldn't have been possible without this change: Downlink is now completely extensible!

That's right, you can now control and modify Downlink on the fly in your own code using any of a myriad of extension points. In fact, when self-hosting you get direct access to the services container it uses (shared with the hosting app) so you can register and resolve any service.

That's all good, but what does that look like in practice? Let's say you wanted to change how Downlink matches patterns in your app: simply implement the IPatternMatcher interface in a class (that we'll call MyAwesomeMatcher), and then in your startup code:

services.AddMvc().AddDownlink(d => d.AddPatternMatcher<MyAwesomeMatcher>());

Now, you can update your configuration and Downlink will automatically make sure your pattern matcher is available.

Dependency Injection

Since Downlink's extensibility is built around dependency injection, you can also use injection in your own extensions. Let's say that awesome pattern matcher from above needs access to the app configuration for some reason. Well, just add it to your constructor and Downlink will resolve it for you:

public class MyAwesomeMatcher : IPatternMatcher {
  public MyAwesomeMatcher(IConfiguration config) {
    // ...
  }
  // TRIMMED
}

If you need something that's not loaded by default, just add it in the startup code:

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc().AddDownlink(d => {
        d.Services.AddSingleton<MyExtraService>();
        d.AddPatternMatcher<MyAwesomeMatcher>();
    });
}

Now you can resolve MyExtraService in your constructor!

Disabling built-ins

Downlink out of the box includes a lot of "default" implementations for moving parts like storage backends, scheme clients and pattern matchers. If you don't want those active in your app, you can use the new extensibility to also disable them in your startup code:

services.AddMvc().AddDownlink(
  d => d.AddPatternMatcher<MyAwesomeMatcher>(),
  DownlinkBuilderOptions.SkipDefaultPatterns
);
// now only your pattern matcher is active!

Use Cases

In reality, almost every moving part of Downlink can be replaced or controlled using the extensibility points in Downlink, but the main ones (which have helpers provided) include:

• Storage backends
• Pattern matchers
• Scheme clients
• Route builders
• Plugins (more on that on Friday!)

Note that the pre-built app (as included in Docker) has experimental plugin support for extensibility instead, but you'll need to wait for the post on plugins to see that in action!

The technical details

Okay, so for all the fancy talk this really comes down to a particularly heavy refactoring of the app code to be interface-driven and almost completely reliant on DI and IoC.

Basically every moving part in Downlink now implements an interface that gets resolved out of the DI container (currently using Microsoft.Extensions.DependencyInjection) and the app startup code registers all the sane defaults using a plugin. This way, self-hosting scenarios can replace any Downlink component by registering it with the DI container in the AddDownlink method!

You can find detailed documentation on Downlink's new extensibility in the developer guide (or check out all the online docs)

This is part of a series of posts on the new features added in Downlink 0.2! Check out this post for more or hit the docs.

So one of the biggest cornerstones of the rebuilt Downlink is that it is no longer a monolithic Web API! Downlink is now a modular MVC-based component that you can add to any ASP.NET Core app (targeting .NET Core 2.0).

Using the pre-built version

Just because we've made all these changes doesn't make Downlink any harder to get started with!

If you've already got a config file (like config.yml), you can run it just like this:

docker run -it --rm -p 80:80 -v $PWD/config.yml:/downlink/config.yml agc93/downlink:latest

Even if you don't use a config file, you can use environment variables instead:

docker run -it --rm \
-p 80:80 \
-e DOWNLINK:Storage=GitHub \
-e DOWNLINK:GitHubStorage:Repository=gohugoio/hugo \
agc93/downlink:latest

(we're just using Hugo as an example here)

The trick is, you might not want to run a whole separate app/image just for your downloads. What if you want to add Downlink's awesome magic to your existing apps?

Hosting Downlink in another app

Downlink 0.2 has now been re-built as an MVC component, so you can install it into an existing app much more easily.

In fact, when you run the Docker image above, you're actually running a super-lightweight ASP.NET Core app with Downlink pre-installed!

So, how do you host it yourself? Just create a new app (say, with dotnet new api for example) and follow the steps below:

1. Install Downlink

Just run dotnet add package Downlink or add the PackageReference to your csproj:

<PackageReference Include="Downlink" Version="0.2.0" />

2. Add the configuration to your Program.cs

In Program.cs, just update your BuildWebHost to add the required configuration:

WebHost.CreateDefaultBuilder(args)
            .ConfigureDownlink() // <-- add this line!
            .UseStartup<Startup>()
            .Build();

You'll need to provide configuration such as from environment variables or config files, just like when running directly.

3. Add Downlink to your app (in Startup.cs)

Now in Startup.cs, just add a single call to your ConfigureServices method:

services.AddMvc().AddDownlink();

That's it! Downlink is now installed and when you run your app (such as using dotnet run), Downlink will register itself with MVC and be listening for requests.

A note of caution

It's worth noting that Downlink's routing is quite aggressive so I recommend using Downlink 0.2's new routing support! Check back on Thursday for the full details on controlling your routing.

The technical details

Since Downlink uses a whole bunch of the built-in MVC features, it's not as easy as adding a new middleware and standing back. Instead, Downlink uses MVC's new Application Part abstraction.

You can see here that we only need to register the hosting assembly as an application part and MVC will use it to pull resources (controllers etc) and add them to the hosting app. Downlink's internal DownlinkBuilder type is then responsible for configuration and extensions (see tomorrow's post for more on that!).

The only other important part is that ConfigureDownlink() method that we added earlier. That just adds in the configuration that Downlink expects!

And just to prove the point you can see that the app we package in the Docker image is built like any other hosting app, no magic here!

You can find detailed documentation on hosting Downlink here (or check out all the online docs)