Configuring Solarized Colours in Termonad

Posted by Craige McWhirter on

I'm currently using Termonad as my terminal of choice. What is Termonad?

Termonad is a terminal emulator configurable in Haskell. It is extremely customizable and provides hooks to modify the default behavior. It can be thought of as the "XMonad" of terminal emulators.

As a long time Xmonad user, this is a rather appealing description as well as a fairly lofty and worthy goal. It's also a niche not currently filled, one that I'm pretty happy to see being filled. By default, Termonad looks like this.

Default Termonad palette

Which is pretty standard as far as terminal defaults go but a long way from the eye-soothing grace of the Solarized palette which I essentially won't work without.

The Solarized dark output looks like this (Haskell in vim):

Solarized (dark) Termonad palette

This is the function controlling the dark palette:

    solarizedDark1 :: Vec N8 (Colour Double)
    solarizedDark1 =
         sRGB24   0  43  54 -- base03, background
      :* sRGB24 220  50  47 -- red
      :* sRGB24 133 153   0 -- green
      :* sRGB24 181 137   0 -- yellow
      :* sRGB24  38 139 210 -- blue
      :* sRGB24 211  54 130 -- magenta
      :* sRGB24  42 161 152 -- cyan
      :* sRGB24 238 232 213 -- base2
      :* EmptyVec

    solarizedDark2 :: Vec N8 (Colour Double)
    solarizedDark2 =
         sRGB24   7  54  66 -- base02, background highlights
      :* sRGB24 203  75  22 -- orange
      :* sRGB24  88 110 117 -- base01, comments / secondary text
      :* sRGB24 131 148 150 -- base0, body text / default code / primary content
      :* sRGB24 147 161 161 -- base1, optional emphasised content
      :* sRGB24 108 113 196 -- violet
      :* sRGB24 101 123 131 -- base00
      :* sRGB24 253 246 227 -- base3
      :* EmptyVec

The Solarized light output looks like this (Haskell in vim):

Solarized (light) Termonad palette

This is the function controlling the light palette:

    solarizedLight1 :: Vec N8 (Colour Double)
    solarizedLight1 =
         sRGB24 238 232 213 -- base2, background highlights
      :* sRGB24 220  50  47 -- red
      :* sRGB24 133 153   0 -- green
      :* sRGB24 181 137   0 -- yellow
      :* sRGB24  38 139 210 -- blue
      :* sRGB24 211  54 130 -- magenta
      :* sRGB24  42 161 152 -- cyan
      :* sRGB24   7  54  66 -- base02
      :* EmptyVec

    solarizedLight2 :: Vec N8 (Colour Double)
    solarizedLight2 =
         sRGB24 253 246 227 -- base3, background
      :* sRGB24 203  75  22 -- orange
      :* sRGB24 147 161 161 -- base1, comments / secondary text
      :* sRGB24 101 123 131 -- base00, body text / default code / primary content
      :* sRGB24  88 110 117 -- base01, optional emphasised content
      :* sRGB24 108 113 196 -- violet
      :* sRGB24 131 148 150 -- base0
      :* sRGB24   0  43  54 -- base03
      :* EmptyVec

You can see how I've applied these options in this commit.

Update: My complete example is now included in the termonad examples.

Finding Bugs in an Aercus WS3083

Posted by Craige McWhirter on

Aercus WS3083

While we're not drought declared, there's been precious little rain over the last 6 months or so, which made it hard to work out if I really had a problem or not.

The rainfall had been light and infrequent while each time the weather station recorded 0mm rainfall, which didn't seem unreasonable but was dubious nonetheless.

All the sensors appeared to be working OK. I re-seated, reconnected and reset everything to make sure that the system was connected and working fine but with no change in the result.

I eventually was driven to digging out the manual gauge which recorded 18mm the next time it rained while the weather station recorded 0mm.

No choice this time but to get up the ladder and disassemble the rain gauge.

I removed the cover and everything passed an initial eye balling.

Aercus WS3083 rain gauge

The Aercus WS3083 uses a lever arm to measure and discard collected rain. Being a simple primate, I attempted toggled it with my index finger, instantly noted it wasn't moving freely before a massive grasshopper|cricket flew out from under the arm and into my face. It had made a nice home under the arm, preventing it from dipping down and counting rain.

Unsurprisingly it moved freely after that and has been accurately measuring rain since.

Postscript: I think the chickens are still upset at me for failing to catch the grasshopper|cricket and feed it to them.

Querying Installed Package Versions Across An Openstack Cloud

Posted by Craige McWhirter on

AKA: The Joy of juju run

Package upgrades across an OpenStack cloud do not always happen at the same time. In most cases they may happen within an hour or so across your cloud but for a variety reasons, some upgrades may be applied inconsistently, delayed or blocked on some servers.

As these packages may be rolling out a much needed patch or perhaps carrying a bug, you may wish to know which services are impacted in fairly short order.

If your OpenStack cloud is running Ubuntu and managed by Juju and MAAS, here's where juju run can come to the rescue.

For example, perhaps there's an update to the Corosync library libcpg4 and you wish to know which of your HA clusters have what version installed.

From your Juju controller, create a list of servers managed by Juju:

Juju 1.x:

$ juju stat --format tabular > jsft.out

Now you could fashion a query like this, utilising juju run:

$ for i in $(egrep -o '[a-z]+-hacluster/[0-9]+' jsft.out | cut -d/ -f1 | sort -u);
do juju run --timeout 30s --service $i "dpkg-query -W -f='\${Version}' libcpg4" | \
python -c 'import yaml,sys;print("\n".join(["{} == {}".format(y["Stdout"], y["UnitId"]) for y in yaml.safe_load(sys.stdin)]))';
done

The output returned will look something like this:

2.3.3-1ubuntu4 == ceilometer-hacluster/1
2.3.3-1ubuntu4 == ceilometer-hacluster/0
2.3.3-1ubuntu4 == ceilometer-hacluster/2
2.3.3-1ubuntu4 == cinder-hacluster/0
2.3.3-1ubuntu4 == cinder-hacluster/1
2.3.3-1ubuntu4 == cinder-hacluster/2
2.3.3-1ubuntu4 == glance-hacluster/3
2.3.3-1ubuntu4 == glance-hacluster/4
2.3.3-1ubuntu4 == glance-hacluster/5
2.3.3-1ubuntu4 == keystone-hacluster/1
2.3.3-1ubuntu4 == keystone-hacluster/0
2.3.3-1ubuntu4 == keystone-hacluster/2
2.3.3-1ubuntu4 == mysql-hacluster/1
2.3.3-1ubuntu4 == mysql-hacluster/2
2.3.3-1ubuntu4 == mysql-hacluster/0
2.3.3-1ubuntu4 == ncc-hacluster/1
2.3.3-1ubuntu4 == ncc-hacluster/0
2.3.3-1ubuntu4 == ncc-hacluster/2
2.3.3-1ubuntu4 == neutron-hacluster/2
2.3.3-1ubuntu4 == neutron-hacluster/1
2.3.3-1ubuntu4 == neutron-hacluster/0
2.3.3-1ubuntu4 == osd-hacluster/0
2.3.3-1ubuntu4 == osd-hacluster/1
2.3.3-1ubuntu4 == osd-hacluster/2
2.3.3-1ubuntu4 == swift-hacluster/1
2.3.3-1ubuntu4 == swift-hacluster/0
2.3.3-1ubuntu4 == swift-hacluster/2

Juju 2.x:

$ juju status > jsft.out

Now you could fashion a query like this:

$ for i in $(egrep -o 'hacluster-[a-z]+/[0-9]+' jsft.out | cut -d/ -f1 |sort -u);
do juju run --timeout 30s --application $i "dpkg-query -W -f='\${Version}' libcpg4" | \
python -c 'import yaml,sys;print("\n".join(["{} == {}".format(y["Stdout"], y["UnitId"]) for y in yaml.safe_load(sys.stdin)]))';
done

The output returned will look something like this:

2.3.5-3ubuntu2 == hacluster-ceilometer/1
2.3.5-3ubuntu2 == hacluster-ceilometer/0
2.3.5-3ubuntu2 == hacluster-ceilometer/2
2.3.5-3ubuntu2 == hacluster-cinder/1
2.3.5-3ubuntu2 == hacluster-cinder/0
2.3.5-3ubuntu2 == hacluster-cinder/2
2.3.5-3ubuntu2 == hacluster-glance/0
2.3.5-3ubuntu2 == hacluster-glance/1
2.3.5-3ubuntu2 == hacluster-glance/2
2.3.5-3ubuntu2 == hacluster-heat/0
2.3.5-3ubuntu2 == hacluster-heat/1
2.3.5-3ubuntu2 == hacluster-heat/2
2.3.5-3ubuntu2 == hacluster-horizon/0
2.3.5-3ubuntu2 == hacluster-horizon/1
2.3.5-3ubuntu2 == hacluster-horizon/2
2.3.5-3ubuntu2 == hacluster-keystone/0
2.3.5-3ubuntu2 == hacluster-keystone/1
2.3.5-3ubuntu2 == hacluster-keystone/2
2.3.5-3ubuntu2 == hacluster-mysql/0
2.3.5-3ubuntu2 == hacluster-mysql/1
2.3.5-3ubuntu2 == hacluster-mysql/2
2.3.5-3ubuntu2 == hacluster-neutron/0
2.3.5-3ubuntu2 == hacluster-neutron/2
2.3.5-3ubuntu2 == hacluster-neutron/1
2.3.5-3ubuntu2 == hacluster-nova/1
2.3.5-3ubuntu2 == hacluster-nova/2
2.3.5-3ubuntu2 == hacluster-nova/0

You can of course substitute libcpg4 in the above query for any package that you need to check.

By far and away my most favourite feature of Juju at present, juju run reminds me of knife ssh, which is unsurprisingly one of my favourite features of Chef.

Resolving a Partitioned RabbitMQ Cluster with JuJu

Posted by Craige McWhirter on

On occasion, a RabbitMQ cluster may partition itself. In a OpenStack environment this can often first present itself as nova-compute services stopping with errors such as these:

ERROR nova.openstack.common.periodic_task [-] Error during ComputeManager._sync_power_states: Timed out waiting for a reply to message ID 8fc8ea15c5d445f983fba98664b53d0c
...
TRACE nova.openstack.common.periodic_task self._raise_timeout_exception(msg_id)
TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.7/dist-packages/oslo/messaging/_drivers/amqpdriver.py", line 218, in _raise_timeout_exception
TRACE nova.openstack.common.periodic_task 'Timed out waiting for a reply to message ID %s' % msg_id)
TRACE nova.openstack.common.periodic_task MessagingTimeout: Timed out waiting for a reply to message ID 8fc8ea15c5d445f983fba98664b53d0c

Merely restarting the stopped nova-compute services will not resolve this issue.

You may also find that querying the rabbitmq service may either not return or take an awful long time to return:

$ sudo rabbitmqctl -p openstack list_queues name messages consumers status

...and in an environment managed by juju, you could also see JuJu trying to correct the RabbitMQ but failing:

$ juju stat --format tabular | grep rabbit
rabbitmq-server                       false local:trusty/rabbitmq-server-128
rabbitmq-server/0           idle   1.25.13.1 0/lxc/12 5672/tcp 192.168.7.148
rabbitmq-server/1   error   idle   1.25.13.1 1/lxc/8  5672/tcp 192.168.7.163   hook failed: "config-changed"
rabbitmq-server/2   error   idle   1.25.13.1 2/lxc/10 5672/tcp 192.168.7.174   hook failed: "config-changed"

You should now run rabbitmqctl cluster_status on each of your rabbit instances and review the output. If the cluster is partitioned, you will see something like the below:

ubuntu@my_juju_lxc:~$ sudo rabbitmqctl cluster_status
Cluster status of node 'rabbit@192-168-7-148' ...
[{nodes,[{disc,['rabbit@192-168-7-148','rabbit@192-168-7-163',
                'rabbit@192-168-7-174']}]},
 {running_nodes,['rabbit@192-168-7-174','rabbit@192-168-7-148']},
 {partitions,[{'rabbit@192-168-7-174',['rabbit@192-168-7-163']},
               {'rabbit@192-168-7-148',['rabbit@192-168-7-163']}]}]
...done.

You can clearly see from the above that there are two partitions for RabbitMQ. We need to now identify which of these is considered the leader:

maas-my_cloud:~$ juju run --service rabbitmq-server "is-leader"
- MachineId: 0/lxc/12
  Stderr: |
  Stdout: |
    True
  UnitId: rabbitmq-server/0
- MachineId: 1/lxc/8
  Stderr: |
  Stdout: |
    False
  UnitId: rabbitmq-server/1
- MachineId: 2/lxc/10
  Stderr: |
  Stdout: |
    False
  UnitId: rabbitmq-server/2

As you see above, in this example machine 0/lxc/12 is the leader, via it's status of "True". Now we need to hit the other two servers and shut down RabbitMQ:

# service rabbitmq-server stop

Once both services have completed shutting down, we can resolve the partitioning by running:

$ juju resolved -r rabbitmq-server/<whichever is leader>

Substituting <whichever is leader> for the machine ID identified earlier.

Once that has completed, you can start the previously stopped services with the below on each host:

# service rabbitmq-server start

and verify the result with:

$ sudo rabbitmqctl cluster_status
Cluster status of node 'rabbit@192-168-7-148' ...
[{nodes,[{disc,['rabbit@192-168-7-148','rabbit@192-168-7-163',
                'rabbit@192-168-7-174']}]},
 {running_nodes,['rabbit@192-168-7-163','rabbit@192-168-7-174',
                 'rabbit@192-168-7-148']},
 {partitions,[]}]
...done.

No partitions \o/

The JuJu errors for RabbitMQ should clear within a few minutes:

$ juju stat --format tabular | grep rabbit
rabbitmq-server                       false local:trusty/rabbitmq-server-128
rabbitmq-server/0             idle   1.25.13.1 0/lxc/12 5672/tcp 19 2.168.1.148
rabbitmq-server/1   unknown   idle   1.25.13.1 1/lxc/8  5672/tcp 19 2.168.1.163
rabbitmq-server/2   unknown   idle   1.25.13.1 2/lxc/10 5672/tcp 192.168.1.174

You should also find the nova-compute instances starting up fine.

First Look at Snaps

Posted by Craige McWhirter on

I've belatedly come to have a close up look at both Ubuntu Core (Snappy), Snaps and the Snappy package manager.

The first pass was to rebuild my rack of Raspberry Pi's from Debian armhf to Ubuntu Core for the Raspberry Pi.

Rack'o'Pi's)

This proved to be the most graceful install I've ever had on any hardware, ever. No hyperbole: boot, authenticate, done. I repeated this for all six Pi's in such a short time frame that I was concerned I'd done something wrong. Your SSH keys are already installed, you can log in immediately and just get on with it.

Which is where snaps come into play.

Back on my laptop, I followed the tutorial Create Your First Snap which uses GNU Hello as an example snap build and finishes with a push to the snap store at snapcraft.io.

I then created a Launchpad Repo, related a snap package, told it to build for armhf and amd64 and before long, I could install this snap on both my laptop and the Pi's.

Overall this was a pretty impressive and graceful process.

Japanese House in Python for Minecraft

Posted by Craige McWhirter on

I have kids that I'm teaching to hack. They started of on Scratch (which is excellent) and are ready to move up to Python. They also happen to be mad Minecraft fans, so now they're making their way through Adventures in Minecraft.

As I used Scratch when they were, I'm also hacking in Python & Minecraft as they are. It helps if I hit the bumps and hurdles before they do, as well as have a sound handle on the problems they're solving.

Now I've branched out from the tutorial and I'm just having fun with it and leaving behind code the kids can use, hack whatever. This code is in my minecraft-tools repo (for want of a better name). It's just a collection of random tools I've written for Minecraft aren't quite up to being their own thing. I expect this will mostly be a collection of python programs to construct things inside Minecraft via CanaryMod and CanaryRaspberryJuicePlugin.

The first bit of code to be shaken out of the tree is japanese_house.py which produces a Minecraft interpretation of a classic Japanese house. Presently it only produces the single configuration that is little more than an empty shell.

Japanese House (day) Japanese House
(night)

I intend to add an interior fit out plus a whole bunch of optional configurations you can set at run time but for now it is what it is, as I'm going to move onto writing geodesic domes and transport | teleport rings (as per the Expanse, which will lead to eventually coding a TARDIS, that will you know, be actually bigger on the inside ;-)

Solarized xmobar, trayer and dmenu for xmonad

Posted by Craige McWhirter on

Like many other people, Ethan Schoonover's precision colours for machines and people, known as Solarized is my preferred colour pallet, especially as I spend most of my time in terminals.

I've "Solarized" most things in my work environment except my window manager, xmonad. Well to be more specific, the xmobar, trayer and dmenu applications that I use within xmonad.

xmobar screenshot

The commits:

Modifications were made to xmonad.hs, .xmobarrc and trayer in .xsession that switches out the existing and default colours for those I've selected from the Solarized palette. I now have a fully Solarized window manager that fits in much better with the rest of my workspace.

Relevant code snippets:

xmonad.hs

                , logHook = dynamicLogWithPP $ xmobarPP
                    { ppOutput = hPutStrLn xmproc
                    , ppCurrent = xmobarColor "#859900" "" . wrap "[" "]"
                    , ppVisible = xmobarColor "#2aa198" "" . wrap "(" ")"
                    , ppLayout = xmobarColor "#2aa198" ""
                    , ppTitle = xmobarColor "#859900" "" . shorten 50
                    }
            , modMask = mod4Mask -- Rebind Mod to the Windows key
            --, borderWidth = 1
            } `additionalKeys`
                    -- Custom dmenu launcher
                    [ ((mod4Mask, xK_p ), spawn "exe=`dmenu_path | dmenu -fn \"Open Sans-10\" -p \"λ:\" -nb \"#073642\" -nf \"#93a1a1\" -sb \"#002b36\" -sf \"#859900\"` && eval \"exec $exe\"")

.xmobarrc

-- Appearance
  font = "xft:OpenSans:size=10:antialias=true"
, bgColor = "#073642"
, fgColor = "#93a1a1"

...

--Plugins
, commands =
    -- CPU Activity Monitor
    ...
                         , "--low"      , "#2aa198"
                         , "--normal"   , "#859900"
                         , "--high"     , "#dc322f"
    ...
    -- cpu core temperature monitor
    ...
                         , "--low"      , "#2aa198"
                         , "--normal"   , "#859900"
                         , "--high"     , "#dc322f"
    ...
    -- Memory Usage Monitor
    ...
                         , "--low"      , "#2aa198"
                         , "--normal"   , "#859900"
                         , "--high"     , "#dc322f"
    ...
    -- Battery Monitor
    ...
                         , "--low"      , "#dc322f"
                         , "--normal"   , "#859900"
                         , "--high"     , "#2aa198"

                         , "--" -- battery specific options
                                   -- discharging status
                                   , "-o"   , "<left>% (<timeleft>)"
                                   -- AC "on" status
                                   , "-O" , "<fc=#2aa198>Charging</fc>"
                                   -- charged status
                                   , "-i"   , "<fc=#859900>Charged</fc>"
    ...
    -- Weather Monitor
    ...
                         , "--low"      , "#2aa198"
                         , "--normal"   , "#859900"
                         , "--high"     , "#dc322f"
    ...
    -- Network Activity Monitor
    ...
                         , "--low"      , "#2aa198"
                         , "--normal"   , "#859900"
                         , "--high"     , "#dc322f"
    ...
    -- Time and Date Display
    , Run Date           "<fc=#268bd2>%a %b %_d %H:%M</fc>" "date" 10
    ...

.xsession

exec /usr/bin/trayer --edge top --align right --SetDockType true --SetPartialStrut true --expand true --width 10 --transparent true --alpha 0 --tint 0x073642 --height 22 &

That at least implements the colours I selected from Solarized and should be a good starting point for anyone else.

Adding a Docker Runner to GitLab

Posted by Craige McWhirter on

In my particular scenario, I need to run both docker and docker-compose to test and build our changes. The first step to achieving this is to add an appropriate GitLab runner.

We especially need to run a privileged runner to make this happen.

Assuming that GitLab Runner has already been successfully installed, head to Admin -> Runner in the webUI of your GitLab instance and note your Registration token.

From a suitable account on your GitLab instance register a shared runner:

% sudo /usr/bin/gitlab-ci-multi-runner register --docker-privileged \
    --url https://gitlab.my.domain/ci \
    --registration-token REGISTRATION_TOKEN \
    --executor docker \
    --description "My Docker Runner" \
    --docker-image "docker:latest" \

Your shared runner should now be ready to run.

This applies to self-hosting a GitLab instance. If you are using the gitlab.com hosted services, a suitable runner is already supplied.

There are many types of executors for runners, suiting a variety of scenarios. This example's scenario is that both GitLab and the desired runner are on the same instance.

Have It Your Way: Maximizing Drive-Thru Contributions - PyConAu 2016

Posted by Craige McWhirter on

by VM (Vicky) Brasseur.

Slides.

Vicky talked about the importance non-committing contributors but the primary focus is on committing contributors due to time limits.

Covered the different types of drive-thru contributors and why they show up.

  • Scratching an itch.
  • Unwilling / Unable to find an alternative to this project
  • They like you.

Why do they leave?

  • Itch has been sratched.
  • Not enough time.
  • No longer using the project.
  • Often a high barrier to contribution.
  • Absence of appreciation.
  • Unpleasant people.
  • Inappropriate attribution.

Disadvantages

  • It takes more time to help them land patches
    • Reluctance to help them "as they're not community".

It appears to be that many project see community as the foundation but Vicky contended it is contributors.

More drive-thru contributors are a sign of a healthy project and can lead to a larger community.

Advantages:

  • Have better processes in place.
  • Faster patch and release times.
  • More eyes and shallower bugs
  • Better community, code and project reputation.

Leads to a healthier overall project.

Methods for Maxmising drive-thru contributions:

Documentation!

  • give your project super powers.
  • Scales!
  • Ensures efficient and successful contributions.
  • Minimises questions.
  • Standardises processes.
  • Vicky provided a documentation quick start guide.

Mentoring!

  • Code review.
  • "Office hours" for communication.
  • Hackfests.
  • New contributor events.

Process improvements!

  • Tag starter bugs
  • Contributor SLA
  • Use containers / VM of dev environment

Culture!

  • Value contributions and contributors
  • Culture of documentation
  • Default to assistance

Outreach! * Gratitude * Recognition * Follow-up!

Institute the "No Asshole" rule.

PyConAu 2016

Keynote - Python All the Things - PyConAu 2016

Posted by Craige McWhirter on

by Russell Keith-Magee.

Russell spoke about porting Python to mobile devices. CPython being written in C enables it to leverage the supported platforms of the C language and be compiled a wide range of platforms.

There was a deep dive in the options and pitfalls when selecting a method to and implementing Python on Android phones.

Ouroboros is a pure Python implementation of the Python standard library.

Most of the tools discussed are at an early stage of development.

Why?

  • Being able to run on new or mobile platforms addresses an existential threat.
  • The threat also presents an opportunity to grown, broaden and improve Python.
  • Wants Python to be a "first contact" language, like (Visual) Basic once was.
  • Unlike Basic, Python also support very complex concepts and operations.
  • Presents an opportunity to encourage broader usage by otherwise passive users.
  • Technical superiority is rarely enough to guarantee success.
  • A breadth of technical domains is required for Python to become this choice.
  • Technical problems are the easiest to solve.
  • Te most difficult problems are social and community and require more attention.

Russell's will be putting his focus into BeeWare and related projects.

Fortune favours the prepared mind

(Louis Pasteur)

PyConAu 2016