Keith Sterling

Its a mixed bag of stuff, but mainly tech, music and programming, oh and just a bit of social comment !!!

Keith Sterling - Its a mixed bag of stuff, but mainly tech, music and programming, oh and just a bit of social comment !!!

Working with INI files in Python

Quite often you want to extract you configuration data into a config file, the Windows INI is still a popular format, well understood by most people who like to get their hands dirty on the command line.
When using ini files, you basically want to do 3 things

  • Pass the name of the config file as a command line arg, printing something useful if the arg is missing
  • Load the contents of the init file and use them

This little code snippet does the lot in one go

import configparser
import os.path
from optparse import OptionParser

config  = ["-c", "--config", "config", "Configuration File, INI format"]
parser = OptionParser()
parser.add_option(config[0], config[1], dest=config[2], help=config[3])
(options, args) = parser.parse_args()
if None == options.config:
    parser.print_help ()
    exit(0)
if not os.path.isfile(options.config):
    exit(0)
    parser.print_help ()

config_file = configparser.ConfigParser(interpolation=None)
config_file.read(options.config)

Basically, import the necessary libraries, then set up the Command Line arg, in this instance -c, but you can add more by calling add_option() with new values

Next call parser.parse_args() to parse the command line args passed to you app. If nothing is there, or ‘config’ is missing, or not a valid file, then bomb out print useful help text

Finally load the ini file by using ConfigParser. I use ‘interpolation=None’ so that the parser does not substitute variables inside the ini allowing me to use $ and other non alpha numeric characters where needed

Simple Client/Server RESTful Services with Python

In this article we look at creating the basics of a RESTful service and associated client, that we’ll build on later to create our home automation server

First we need to make sure we have a couple of libraries installed, specifically httplib2 and Flask. Install them using pip

sudo pip install httplib2
sudo pip install Flask

We can check they are installed with

pip list

which should display a list which includes the following

$ pip list
Flask (0.10.1)
httplib2 (0.8)

RESTFul Server

First the server, which we build using Flask

from flask import Flask, request, json
from pymongo import MongoClient

app = Flask(__name__)
mongo_client = MongoClient()
mongo_client['HomeControl'].temperature.drop ()

@app.route("/homecontrol/api/v1.0/temperature", methods=['POST'])
def api_temperature():
    if request.headers['Content-Type'] == 'text/plain':
        print (request.data)
        return 'OK', 200
    elif request.headers['Content-Type'] == 'application/json':
        print (json.dumps(request.json))
        mongo_client['HomeControl']['temperature'].insert(request.json)
        print (mongo_client['HomeControl']['temperature'].count ())
        return "OK", 200
    else:
        return "Unsupported Media Type", 415

if __name__ == '__main__':
    app.run(debug = False)

For a description of how to create a RESTFul server in Flask, check out the excellent blog by Miguel Grinberg

RESTFul Client

Then a basic client that emulates our temp server

import httplib2
import json
import time
import datetime

if __name__ == '__main__':

    httplib2.debuglevel     = 0
    http                    = httplib2.Http()
    content_type_header     = "application/json"

    url = "http://127.0.0.1:5000/homecontrol/api/v1.0/temperature"

    data = {    'room':         "Living Room",
                'temp':         23.45,
                'humidity':     50.00,
                'timestamp':    str(datetime.datetime.now())
           }

    headers = {'Content-Type': content_type_header}
    print ("Posting %s" % data)

    while True:
        response, content = http.request( url,
                                          'POST',
                                          json.dumps(data),
                                          headers=headers)
        print (response)
        print (content)
        time.sleep(3)

This simple client sits in a loop posting a JSON object to our server every 3 seconds
We’ll fill out this client as time goes by, hooking it up to real temp sensors on the pi

Room Temperature Sensor With Raspberry Pi

This article is taken from the following web pages, with a few modifications by me.

The intention is to create a Raspberry monitoring station for each room in the house, each broadcasting temperature and other information back to a central server in my office.

To start with I am using DS18B20 temperature sensor, which you can get from Amazon and eBay for about 50p-£1 each.

Along with the sensor you are going to need a breadboard, ribbon cable, some wires and 4.7k resistor

breadboard-ic.png

Once you have this wired up, you need to activate the sensor

$ sudo modprobe w1-gpio
$ sudo modprobe w1-therm
$ cd /sys/bus/w1/devices
$ ls

This will give you the id of the device starting 28- and looking something like

28-xxxxxxxxxxxxxxx

Now create a shell script we can query the sensor

nano ~/temp.sh

And add the following lines

#! /bin/bash
sudo modprobe w1-gpio
sudo modprobe w1-therm
roomtemp=$(cat /sys/bus/w1/devices/28-xxxxxxxxxxxx/w1_slave | grep  -E -o ".{0,0}t=.{0,5}" | cut -c 3-)
echo "Temperature: $roomtemp"

To make it executable, give it +x permissions

chmod +x ~/temp.sh

And it will print out the temp, device / 1000 for ‘C

$ ~./temp.sh
186903

Which is 18.69′C

Now lets create a little Python script that parses the data and loops printing out the value every 10 secs

nano ~/temp.py

And add the following lines

import time
while True:
        # Open the file that we viewed earlier so that python 
        # can see what is in it. Replace the serial number as before.
        tfile = open("/sys/bus/w1/devices/28-xxxxxxxxxx/w1_slave")
        # Read all of the text in the file.
        text = tfile.read()
        # Close the file now that the text has been read.
        tfile.close()
        # Split the text with new lines (\n) and select the second line.
        secondline = text.split("\n")[1]
        # Split the line into words, referring to the spaces, 
        # and select the 10th word (counting from 0).
        temperaturedata = secondline.split(" ")[9]
        # The first two characters are "t=", so get rid of those and 
        # convert the temperature from a string to a number.
        temperature = float(temperaturedata[2:])
        # Put the decimal point in the right place and display it.
        temperature = temperature / 1000
        print ("%2.2f'C" % temperature)
        time.sleep(10)

To run the script, use

python temp.py

Mine is currently showing a balmy 16-19′C in my office

19.00'C
18.94'C
19.06'C
18.81'C
18.12'C
18.12'C
18.50'C
18.56'C
18.06'C
18.06'C
18.25'C
18.06'C
17.69'C
17.81'C
17.25'C
17.12'C
16.69'C
16.31'C
16.50'C
16.81'C
17.12'C
17.38'C
17.56'C
17.56'C

Loading Drivers on Startup

The following lines are taken from raspberry.znix.com

sudo nano /etc/modprobe.d/raspi-blacklist.conf and comment out the following 2 lines

blacklist spi-bcm2708
blacklist i2c-bcm2708

sudo nano /etc/modules

w1-gpio
w1-therm

Next steps

Now that we have a temperature sensor schematic and working prototype, need to build 3 or 4 more, and then hook them up to the central server. So once they are built, I need a Restful service that each of the Pi’s can call with their temperature data. The easiest way in Python is using the Flask framework backed with a Mongo DB. Watch this space for….

Building a RESTful web service using Pythong & Flask

 

Changing the hostname on a Raspberry Pi

Changing your hostname on a Raspi is pretty straight forward,

To get a host name use

keith@fuzepi ~ $ hostname
fuzepi

We now need to change the host name for the local address 127.0.0.1

keith@fuzepi ~ $ sudo vi /etc/hosts
[sudo] password for keith:

Change the name associated with 127.0.01, then we need to change the actual name of the host

keith@fuzepi ~ $ sudo nano /etc/hostname

Change the name in this file and save
One more step is to update the system

keith@fuzepi ~ $ sudo /etc/init.d/hostname.sh

Finally, reboot the Pi

keith@fuzepi ~ $ reboot

and the new name should be available with the command ‘hostname’

keith@gertpi ~ $ hostname
gertpi

Raspberry Pi MongoDB Installation

Full instructions for install mongodb on a Pi can be found here

However on my 256M Model B, I have to change the memory split with the GPU and just allocated it 16M. This allows the command

git clone https://github.com/skrabban/mongo-nonx86

To not exit with “out of memory”. You can do this by running

sudo raspi-config

And selecting advanced options, then memory split

 

 

Running GGTS on OSX under Java 7

I had a look at this and it looks to be a bug somehow related to the way that RJB on OSX works. RJB is the library that Buildr uses to interact with the JVM. Essentially RJB does not seem to allow the configuration of the JVM without setting some environment variables (possibly at build time?). Seehttp://rjb.rubyforge.org/.

There are two main ways to work around this;

  • use the external compiler by adding something like “compile.using(:externaljavac)” into your buildfile.
  • use JRuby.

I will look into what is required to solve this correctly for the next release (1.4.8).

Update

Fixed in 1.4.8 – a work around for 1.4.7 is to set the JVM_LIB environment variable like

export JVM_LIB=$JAVA_HOME/jre/lib/server/libjvm.dylib

Install Python 3 Raspberry Pi

Its pretty simple to get Python 3.2 ( the latest for Pi ) installed, just a couple of apt-get commands and a little shell hacking will do it

First lets get Python 3 install

sudo apt-get install python3

Next get setup tools

sudo apt-get install python-setuptools

And finally we want Pip installing but ensuring it works for Python 3

sudo apt-get install python3-pip

Then modify you shell script to alias ‘python’ to run ‘python3′

alias vi='vim'
alias python='python3.2'
alias pip='pip-3.2'

Jobs a good’un !

Install Python 3 on OSX

Even the the latest version of OSX ships with Python 2.7 which is getting a bit long in the tooth and Python 3.x is where its at these days. Although I wish most Python library developers would update their systems. Its not that hard

Anyway, download the latest version, in this instance Python 3.3

http://www.python.org/getit/

I went with the 64 bit version as default. Run the installer once downloaded and you have 3.3 installed but from the command line, it still shows 2.7, so we need to tell the shell to use Python 3.

Edit ~./bash_rc and add the following 2 lines

alias python=python3
PATH="/Library/Frameworks/Python.framework/Versions/3.3/bin:${PATH}" export PATH

If we want to do anything interesting with Python we will need pip, the pack installer. Get this with the following command

curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python3

You can check its installed and working with Python 3 with the following

pip --version

Which should should something line the following

pip 1.5.2 from /Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages (python 3.3)

Which shows you are running pip on the latest version of Python you just installed

Adventures in Natural Language Processing – No 1. Setup

Natural Language Processing

So this will be a series of articles on natural language processing, this is based on Python 3, and the NL Toolkit, NLTK 3.0, which can be found at http://www.nltk.org

 

The current version of NLTK is based on Python 2,8, As we are using Python 3, and there is an alpha version available, but requires a bit of a manual process. First we need to install PyYaml 3.10 as NLTK requires a version greater than 3.9 and 3.10 is the latest version.

This can be installed at the command line using pip

> sudo pip install pyyaml

We can then check its installed with ‘pip list’

> pip list

Next head over to http://nltk.org/nltk3-alpha/, download one of the distributions, then unzip and open command line in root of unzipped files

> sudo python setup.py install

This will install NLTK into your current version of Python. You can test the install by starting the Python command line and importing the nltk module

> python
 Python 2.7.5 (default, Oct 11 2013, 15:52:19)
 [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.75)] on darwin
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import nltk

If this failed or throws an error, you install failed.

If there are no errors, then download the corpas ( this also tests nltk is install ), remaining in the Python command line.

>>> import nltk
>>> ntlk.download ()

A dialog will appear, select all to download all packages and then click Download

Any errors in the above and its likely nltk installation failed

We are ready to begin….. watch out for No 2. NLP Basics