E pur si muove

Setting up a Python development environment on Solaris 11

Sunday, December 08, 2013

Solaris is one of those things which tends to be a bit un-loved, it being baught by Oracle probably didn't help either. But regardless they've made a rather nice and modern OS out of it and it is used in a fair few places so you may need to support it. Here I'll describe how to easily setup a Python development environment on Solaris 11 x86. This will use Vagrant and VirtualBox for virtualisation and bootstrap the development environment using the excellent OpenCSW packages.

Building a Solaris Vagrant box

Vagrant is a fairly nice tool to manage local development virtual machines. It's automation of the ssh setup and project folder sharing are very nice. However all docs encourage you to find a "box" image somewhere on the net and simply update it. This won't work for Solaris as there's no box image, and I'm not providing one either as it'd probably break the Solaris license. Also, if you created the box from scratch at least you know what's in it. There are a few tools that try to automate building boxes, but they're more layers of complexity for little gain (at least for this scenario). And as ususal you can bet their Solaris support is much less smooth then say their Ubuntu support.

There's very little information on how to build a box from scratch however. But it turns out it's not that hard. Before you get started however make sure you have Vagrant 1.3 or above as Vagrant isn't quite completely OS-agnostic and learned how to handle Solaris in 1.3.

Firstly, download the install ISO from Oracle, the text installer is my preferred one as there is really no need for all the UI stuff which just takes space in your final box. I'm not providing a direct link as Oracle URLs change more often then the phase of the moon, but you should be able to find the Solaris 11 download links somewhere. There should even be no need to sign up or register at all, just agree to the license IIRC.

Now create a new VirtualBox VM as normal, but when walking through the installer setup screens there's a few points to observe:

  • Use something like "vagrant-solaris11" as hostname. I'm not sure how important this is but I think it's a vagrant convention.
  • Solaris has default password restrictions. Something like "vagrant1" worked for me but just make something up, we'll change it later.
  • Use "vagrant" as the username of the primary account. Each box needs this user.

Once installed a few things need to be setup for vagrant to fully work. This is all insecure, but this is a local box anyway so no need to worry (right?).

  • Edit /etc/default/passwd to set NAMECHECK=NO and MINNONALPHA=0. This will allow us to create insecure passwords.
  • Now change the root password to "vagrant" using passwd, do the same for the "vagrant" account.
  • Allow passwordless use of sudo: visudo -f /etc/sudoers.d/svc-system-config-user and edit this so that the entry looks like: vagrant ALL=(ALL)NOPASSWD: ALL.
  • Vagrant uses a known ssh key to log into the VMs. Again, this is insecure so do not expose this box to anything but yourself. Do something like this as the vagrant user to install this key:
            $ cd ~
            $ mkdir .ssh
            $ wget -O .ssh/authorized_keys \
            $ chmod 700 .ssh
            $ chmod 600 .ssh/authoriszed_keys

Finally you need to install the VirtualBox guest additions. After clicking the menu item to install them Solaris should auto-mount the guest additions ISO somewhere in /media/VBOXADDITIONS_.... Install the additions with

    pkgadd -G -d /media/VBOXADDITIONS_.../VBoxSolarisAddtions.pkg

Time to create a vagrant box out of this virtual machine. Shut it down and find either it's name or UUID using vboxmanage list vms. Now you can use vagrant to build the box:

    vagrant package --base $name_or_uuid --output

This was a fair amount of manual work, but the good news is that you'll never have to do this again. Save this box somewhere safe and it can be used as the base of any new boxes you want for Solaris 11.

Creating the Python development environment

Now the manual way of installing a Python environment is somehow install a compiler, hunt down all Python's dependencies, build them from source and then build Python. Followed by figuring out what is still missing and repeating the whole process.

Fortunately the good people at OpenCSW have created an entire APT-like package repository based on the GNU toolchain. And even better then that they are pretty good at keeping it updated. So in order to setup a Python environment we just need to install a few of their packages. And even if you wanted to compile Python from source I'd reccomend to use OpenCSW packages for the dependencies.

There are detailed instructions on OpenCSW's site, but I'll continue our earlier example here.

  • First create a new Vagrant project:

            $ mkdir myproj
            $ cd myproj
            $ vagrant init solaris11 /path/to/

    This creates a Vagrantfile in the current directory, which is the per-project Vagrant configuration. You can have a look, but mostly don't need to change anything for this simple scenario.

  • Time to startup the box and ssh into it:

            $ vagrant up
            $ vagrant ssh
  • Once the virtual machine is running and you are logged into it it's time to install some more packages. Firstly bootstrap OpenCSW:

            # pkgadd -d

    (Note how this is utterly insecure)

  • This installed pkgutil which is a package manger for the old SVR4 packages, that is they are the dpkg/rpm equivalent and pkgutil was an apt/yum layer on top of that which Solaris was missing before Solaris 11. OpenCSW is working on supporting Solaris 11 with the native IPS system, but until then pkgutil still works a treat.

    Before doing anything else you want to configure the mirror and release stream you want to use. Edit /etc/opt/csw/pkgutil.conf to set the mirror to or any other unstable mirror. Other release streams will have a lot less recent software available and this is a development box after all

  • Now I prefer to setup at least some form of package verification. So lets set up the key infrastucture for this:

            # pkgutil -U
            # pkgutil -i cswpki

    Once this is finished you'll want to enable this in the configuration. Edit /etc/opt/csw/pkgutil.conf again but this time set use_gpg=true and use_md5=true. Unless you like seeing warnings you may also want to configure the OpenCSW key to be ultimately trusted. It's not like you use this GPG account for anything else.

Now you just need to install the required packages using pkgutil. The following sets you up with Python 2.7:

    # pkgutil -i python27 py_setuptools py_pip

If you want you can also install python33 for Python 3.

If you'd also like to play with extension modules you need a few more things. This is the package list needed for using cffi:

  • gcc4core
  • python27_dev
  • libffi_dev

However before you can install cffi there's one more thing you need to do. The default installation when using the text installer is missing some system headers you need. Luckly since Solaris 11 there is the very powerful Image Packagin System (IPS) to manage Solaris packages (OpenCSW is still working on supporting this natively, pkgutil was made at a time IPS did not exist in Solaris). This is a rather impressive package manager if you feel like digging into it, it's kind of to apt/yum like what ZFS was to ext3 or like SMF was to sysv-init, if that makes sense. And what's even better is that Oracle is hosting the basic Solaris 11 package repository for free so you can use it without any extra setup to install the missing headers just like you've been able to do with apt for the last 20 years:

    pkg install [-nv] system/header

Here the -nv options are to show you a preview before actually doing things, which is a good habit to get into I guess.

And at this point you should be able to install cffi using pip just like you're used too: pip install --user cffi.

Re-packaging the vagrant box

Once you have setup the development environment you want it might be worth to re-package the box again before you start on your project. This will allow you to use it as the base of another project later. It will also allow you to simply throw away the box using vagrant destroy if you managed to somehow break it after which a simple vagrant up would initialise you a new VM with the python environment ready to use.

This is very similar as creating the first box. The only difference is that you want to run vagrant in the directory of your project and not specify the base. This will make vagrant use the VM in the project as the base of the new box:

    vagrant package --output /path/to/boxes/dir/

Now the last thing you may want to do is update the Vagrantfile of you project to point to this new box.

Subscribe to: Posts (Atom)