FT has selected Martin Fords book The Rise of The Robots, as the most important business book of 2015 and poses the question “Robots are going to take our jobs, so what do we do about it?”
I think the answer is simple, learn to build robots. New economies have to embrace automation in all its many forms and as with any industrial revolution, if we don’t embrace the change you’ll get left behind.
Nice documentary on IBM Watson, and a good introduction to Natural Language Processing
We’ve been exploring and using NLP at my work for the past year and seeing huge benefits
This is the sources and references I used while learning Clojure
Clojure Programming – Wikibooks
Clojure for the Brave and True
Clojure – Functional Programming for the JVM
Ok, we can use base types know the difference between collections and sequences ( if you do be sure to tell me ! ) and we can write some basic functions and use map/reduce/filter to play around with collections.
Hopefully you have also upgraded from the basic Clojure REPL to Leiningen one, and are reaping the benefits of uber productivity
Now its time to get real and start writing some real functions, more than a single line and with ability to control flow through the function
;; if statement
(defn is_true? [x] (if (true? x) "yes" "no")) ;; Example of if statement
(is_true? false) ;; results in "no"
;; cond statements
(defn is_positive? [x] ;; Define a function that gives a string value for a numbers positivity
(cond
(> x 0) "positive"
(< x 0) "negative"
:else "zero" )) ;; Optional default catch all others
(is_positive? 3) ;; Results in "positive"
;; case statements
(defn number_str [x]
(case x
1 "One"
2 "Two"
3 "Three"
4 "Four"
5 "Five"
"Too big"))
(number_str 1) ;; Results in "One"
(number_str 4) ;; Results in "Four"
(number_str 10) ;; Results in "Too big"
;; do statements
;; Do evaluates a sequence of expressions in order, the result of the expression is the value of the last
;; expression in the list
(def x 3)
(do
(println "Starting")
(+ x 1)
(inc x)
(println "Stopping")
x )
;; Results in the following output, but why ?
Starting
Stopping
3
;; This is a classic example of immutable data, while 2 statements in the middle apply operations to the variable x
;; but these operations only return values, they do not change the value of x in the expressions
;; This is one of the key facets of functional programming
;; loop/recur
(loop [x 10] ;; Entry point
(when (> x 1)
(println x)
(recur (- x 2)))) ;; Recursive call into the call stack with updated
;; parameters
;; List Comprehension
(for [x (range 100)] x) ;; Create a list 0 to 99
(for [x (range 100)] (* x x)) ;; Create a list by iterating through each number
;; 0 - 99 and multiplying it by itself
(for [x (range 100)] (even? x)) ;; Create a list of booleans representing whether
;; each number between 0 and 99 is even or odd
(for [a (range 100) ;; Nested loop that creates a list of vectors
b (range 100 200) ;; each vector is created from a nest loop
[a b] ;; For each a 0 - 99, iterate through b 100-199
)
(for [a (range 100) ;; Nested loop that creates a list if vectors
b (range 100 200) ;; as above, but exits when a * b > 1000
:while (< (* a b) 1000)]
[a b]
)
So far we have being using the default REPL ( Read-Evalulate-Print Loop ) for entering our Clojure code and seeing how it works.
We are now going to take a look at a much better REPL, one built into another tool called Leiningen. Leiningen is much bigger than an improved REPL, it is a complete project configuration and automation tool, much like Maven or Ant, which all Clojure developers come to rely on.
Head over to leiningen.org and follow the basic instructions for installation onto your platform. We’ll touch on the power of Leiningen in further parts, but for now we are only interested in the REPL
Once you have it installed, to get a new and improve REPL, enter
lein repl
This will then load a REPL which will look something like
REPL server started on port 63316 on host 127.0.0.1 - nrepl://127.0.0.1:63316
REPL-y 0.3.5, nREPL 0.2.6
Clojure 1.6.0
Java HotSpot(TM) 64-Bit Server VM 1.7.0_71-b14
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=>
The key features we are interested in are
Some other useful commands which help your use of REPL
We have to use functions at some point, Clojure is after all a Functional programming language.
(def hello (fn [] "Hello World")) ;; Defines a function, no parameters
(hello) ;; call a function
(defn hello2 [] "hello world") ;; Use a macro to define a function
(hello2) ;; No difference in way to call the function
(defn hello-to [name] (println "Hello" name)) ;; Define a function with a parameter
(hello-to "Keith") ;; Call a function
(defn hello3-to [name, age] ;; Define a function with multiple parameters
(println "Hello" name "you are " age))
(hello2-to "keith" 21) ;; Call a function with multiple parameters
(defn hello-to-some ;; Define a function which has 3 polymorphic methods
([] (println "Hello to no one")) ;; No parameters
([x] (println "Hello to " x)) ;; One parameter
([x y] (println "Hello to " x " and " y))) ;; Two parameters
(hello-to-some)
(hello-to-some "keith")
(hello-to-some "keith" "cat")
Now we know how to creation functions, we can use this new knowledge to do some basic manipulation of collections
(defn squareit [x] (* x x)) ;; Define a method to square a number
(squareit 3) ;; Test it, should return 9
(map squareit [1 2 3 4 5]) ;; Applies squareit to every item of the collection
;; returning a new collection with new values
;; Should return (1 4 9 16 25)
(map squareit '(1 2 3 4 5)) ;; Same as above, this time against a list
(defn more-than-one-char [x] (> (count x) 1)) ;; Filter takes a function that returns a boolean result
;; Called a predicate in Clojure land
;; This filter returns true if the string is > 1 character
(filter more-than-one-char ["a" "ab" "abc"]) ;; Applies the filter to a string, should return ("ab" "abc")
;; More often than not, there is already a function that can be used. In fact in <a href="http://www.keithsterling.com/?p=518" title="Head into Clojure – Part 1: The Basics">Part 1</a> we saw lots of mathematical and
;; boolean functions +, -, *, /, mod, inc dec etc
(map inc [1 2 3]) ;; Increments each element of the collection by 1
(map dec [1 2 3]) ;; Decreases each element by 1
(filter even? [1 2 3 4 5 6 7 8 9 10]) ;; Filters out all odd numbers, returning a collection of evens
(filter float? [1 2.0 3 4 5.0]) ;; Creates a collection of only floats
;; A slightly more advance, but equally important function is reduce. This takes the first item in the list, and applies
;; the define function to it and the next item. The result is them applied to the 3rd item and on and on through to the end
;; of the list. This is best explained with a basic example using the + function
(reduce + [5 8 14 21]) ;; Here reduce takes 5 and applies + to it and 8, result 13
;; 13 then forms the left hand side of the method call,
;; 14 on the right hand side, the result being 27
;; 27 is then the left and 21 the right, result 48
(+ (+ (+ 5 8) 14) 21) ;; The equivalent if you wrote it out long hand
;; but this would be impossible if you didn't know the length
;; of the collection in the first place
More Clojure-ness code examples. This time some basic types, collections and sequences
; Basic Types
;; Chars
;; Clojure represents all chars in with a leading \
(char 97) ;; Returns \a, 97 being ASCII a
(char? \a) ;; true
(char? (char 97)) ;; true
;; Strings
“This is a string” ;; Define a string, uses double quotes “…”
(string? "Are you a string") ;; Check if a string
(println "Hello world") ;; Print a string out
print-str
println-str
pr-str
prn-str
(with-out-str (println "this should return as a string"))
(subs "1234" 1)
(subs “abcdef” 1 2)
(count “1 2 3 4”) ;; Counts the number of chars in a string
;; Collections and Sequences
;; For a good description about the difference ( mainly in your head ) between Clojure collections
;; and sequences, head over to http://www.brainonfire.net/files/seqs-and-colls/main.html
;; What you need to get your head round is every sequence is a collection, but not every collection is a sequence.
;;; Lists ;; Linked List
(+ 1 2 3) ;; List, function as first item, evaluates to 6
‘( 1 2 3 4 ) ;; List, made up of 4 items
(list 1 3 4 5) ;; Another way to produce a list
(def my-list ‘(1 2 3)) ;; List, assigned to variable my-list
;;; Vectors
[1 2 3] ;; Vector of 3 items
[[1 2] [3 4] [5 6]] ;; Vector of 3 Vectors
(vector 1 2 3) ;; Another way to create a vector
(def my-vector [1 2 3 4]) ;; Assign vector of 4 times to variable my-vector
(my-vector 2) ;; Access 3rd element ( vectors zero based )
(get my-vector 0) ;; 1
(get my-vector 2) ;; 3
(get my-vector 10) ;; nil
;;; Maps ;; value to value
{ 1 2 3 4 } ;; Create a vector or 2 elements, 1 => 2 and 3 => 4
;; Creating maps always needs an even number of elements
(def my-map {1 2 3 4})
(my-map 1) ;; Returns value with key 1
(my-map 5) ;; Returns nil
(def my-map {:name "fred" :age 32}) ;; Map with keywords
(my-map :name) ;; Access map via keyword
(get my-map :name) ;; fred
(def my-map2 {“name” "fred" “age” 32}) ;; Map with strings as keys
(my-map2 “name”) ;; Access map via str key
;;; Sets ;; Sets - List of unique values, no duplicates
#{1 2 3 4} ;; Defined using #{..}
(set [1 2 3 4 5 6]) ;; Results in a set of 6 items 1, 2, 3, 4, 5 and 6
(set [1 2 2 1 1 2]) ;; Results in a set of 2 items 1 and 2
(my-set 1)
(get my-set 1)
;; Manipulation Collections
(first '(1 2 3)) ;; Get the first element of a list
(first [1 2 3]) ;; Get the first element of a Vector
(first {1 2 3 4}) ;; Get the first element of a value map
(first {:first 1 :second 2}) ;; Get first element of keyword map, but which gets
;; returned is indeterminate because you don’t know
;; internally how the map is stored in order
(rest '(1 2 3)) ;; Get all but the first element of a list
(rest [1 2 3]) ;; Get all but the first element of a Vector
(rest {1 2 3 4}) ;; Get all but the first element of a value map
(rest {:first 1 :second 2}) ;; Get all but the first element of keyword map,
;; but which gets returned is indeterminate because
;; you don’t know internally how the map is stored in order
(cons 1 '(2 3 4)) ;; Add element to front of list
(cons 1 [2 3 4]) ;; Add element to front of Vector
I love reading books and I love learning programming languages, but I’ve found that I don’t like reading books to learning programming languages. This is especially true for some of the new languages, specifically Scala and Clojure. Yes you can read a multitude of acronym vebosities as an author tries and shows you just how many words they can type to help explain a feature, but I would rather just have the core basics, explained via lots of examples.
If you have read any of the the Little Schemer books, which are just a set of examples getting increasingly more complex, or have checked out any of the learnxinyminutes.com pages, then you will find this article ( and the subsequent ones ), very similar. I found I learn by example, and as I learn a new language I write down examples of each of the features as I learn them.
These notes are how I learnt Clojure and built my first apps with it.
First lets get some basic terminology out of the way, specifically atoms, s-expressions and forms. In Clojure code is made up of s-expressions, and s-expressions are recursively made up of either an s-expression or an atom
An atom is the most basic of types, an immutable object that always resolves to itself, typically a scala type such as an integer, float, string or char
1 ;; Integer atom 2.3 ;; Float atom "keith" ;; String atom \b ;; Char atom
An s-expressions is sequence of data surrounded by parantheses that is evaluated by the Clojure interpreter. For a generic definition see Wikipedia, which states
In computing, s-expressions, sexprs or sexps (for “symbolic expression”) are a notation for nested list (tree-structured) data, invented for and popularized by the programming language Lisp, which uses them for source code as well as data.
And from Clojure.org itself, they are defined as:
Clojure programs are composed of expressions. Every form not handled specially by a special form or macro is considered by the compiler to be an expression, which is evaluated to yield a value. There are no declarations or statements, although sometimes expressions may be evaluated for their side-effects and their values ignored.
Clojure works by taking an s-expressions, a list of data surrounded by () and assumes the first element in the list is a function. It then applies the function the remaining items of data in the list. In Clojure this is called a form. The only deviation from this rule is for either special forms and macros, both we’ll talk about later as we come across them and now neccassary at this stage
; This is a comment, in Clojure all single line comments start with a ';'
;; However the Clojure community seems to prefer the use of double ';;' instead
;;; However this is a comment
;;;; And so is this
;;;;;;;;; Clojure basically ignores everything after the first ;
;; For more details on a community attempt to document Clojure code, head over
;; to https://clojuredocs.org/
;; There are other ways to comments code out and we'll come across that as we
;; move through this tutorial
;; Before you can even start using Clojure you need to get an instance running
;; Clojure runs on top of the Java virtual machine, so first you need to install
;; an appropriate JVM.
;; This can be downloaded from Oracle https://www.java.com/en/download/
;; Clojure needs atleast Java 6 SDK to run
;; Once installed and running you then need to download the latest Clojure from
;; http://clojure.org/downloads
;; The first way we will use Clojure is through the native REPL, which stands for
;; Read, Evalutate, Print, Loop
;; This provides a command line interface that you can execute Clojure commands from
;; To run the REPL, execute the following command from the command line
;; java -cp /Clojure-1.6.0/clojure-1.6.0.jar clojure.main
; Get Help
;; Before we get started you are going to want more help that these files produce
;; The primary source of all Clojure info is at
;; http://clojure.org/
;; This tutorial is heavily based on the learn x in y minutes pages at
;; http://learnxinyminutes.com/docs/clojure/
;; The primary difference being its structure the way I think and learnt Clojure
;; which builds from first principles as a pure Lisp language and then introduces
;; the additional complexity added by he Java Virtual Machine
;; When you really just need help on syntax, or just need to remember the name
;; of a command then your first stop should be
;; http://clojure.org/cheatsheet
; Getting started
;; Clojure is a Lisp, a member of a family of languages, including Scheme, Common Lisp
;; Lisp ( and therefore Clojure ) are built around the basic concept of a list of things
;; more commonly called forms
;; Forms are bounded by brackets either () or [] and contain zero or more items which
;; make up the list
;; Typically Clojure ( and all Lisps ) assume that the first element of a list is a
;; function call, and all other elements are parameters which are passed to the
;; function
;; The most basic of functions are the basic arithmetic, + - * /, e.g
(+ 2 2) ;; Add 2 and 2, returning 4
(* 1 2 3) ;; Multiple 1, 2 and 3, returning 6
;; If you don't want Clojure to interpret your form, then you can quote it with either the
;; character ' or the function 'quote'
'(+ 2 2) ;; All elements of the form are taken as a list
;; In this instance creates a list of 3 items
(quote (+ 2 2)) ;; Identical to the above
;; If we want to evaluate a list of elements, we can use the function 'eval'
(eval '(+ 2 2)) ;; Returns the value 4
;; The examples for quote and eval above show the use of nested lists
;; Each element of a list can be an individual item or a list itself
;; Lists are evaluated to a single item as the list is interpreted
(+ (* 2 3) (* 4 2)) ;; Becomes (+ 6 8)
;; Which becomes 14
;; There is not limited to the depth of nesting in Lisp
(+ (+ (+ 2 2 ) 3) 4)
;; Whitespace and commas
;; Its worth exploring how Clojure ( and Lisp ) uses whitespace to separate
;; elements of the list, and how commas ',' can be used to show individual
;; items but are ignore by the interpreter
;; Maths Operators
(+ 2 2)
(- 1 2)
(* 2 3)
(/ 4 2)
(quot 3 2) ;; quot[ient] of dividing numerator by denominator. Returns 1
(rem 3 2) ;; remainder of dividing numerator by denominator
(mod 2 3) ;; Modulus of num and div
(inc 2) ;; Increments the number by 1
(dec 3) ;; Decrements the number by 1
(max 3 5 7) ;; Maximum number in list
(min 2 4 1 7) ;; Minimum number in list
;; Boolean operations
;; In Clojure, boolean True is represented as true and boolean False as false
;; Everything except `false' and `nil' is true in boolean context.
;; In Clojure nil means 'nothing', and is not the same as (), the empty set
;; Comparison operators
;;; = ;; Compares if the 2 elements are the same and the same value
(= 2 2) ;; true
(= 2 2.0) ;; false
;;; == ;; Compares the numeric value, true if equals
(= 2 2) ;; true
(= 2 2.0) ;; true
(not= 2 3) ;; Not equal to each other
(< 1 2 3) ;; Tests first element is less than all others
(> 2 3 4) ;; Tests first element is less than all others
(<= 4 4) ;; Tests first element is less than or equal to all others
(>= 6 5 5) ;; Tests first element is great than or equal to all others
(compare 3 4) ;; Returns a negative number, zero, or a positive number
;; when x is logically 'less than', 'equal to',
;; or 'greater than' y
;; Tests. Unary methods that test the value of a result. Return true or false
(zero? 1) ;; Is the number 0, true or false
(pos? -1) ;; Is the number postive, true or false
(neg? 1) ;; Is the number negative, true or false
(even? 2) ;; Is the number even, true or false
(odd? 3) ;; Is the number odd, true or false
(number? 'hello') ;; Is the number a number, true or false
(rational? 1. ;; Is the number a rational ( not a decimal ), true or false
(integer? 1.3) ;; Is the number an integer, true or false
(ratio? 22/7) ;; Is the number a decimal number expressed as a ratio, true or false
(decimal? 1.4) ;; Is the number a decimal number not expressed as a ratio, true or false
(float? 3) ;; Is the number a floating point number, true or false
So thats it, thats a real basic introduction to Clojure, next lesson will introduce you to functional programming. After all isn’t that why you are learning Clojure, its all about the functions !
Once we get your head into functional programming we look at core types of Lists, Vectors, Sets and Maps, and we use functional programming to manipulate them
Just found an amazing website which shows how to create a fully built environment sensor using a Raspberry Pi
Full instructions for building, configuring and programming for the AirPi can be found here
I’m slowly building individual items to monitor rooms around the house, but this breaks the back of all the hard work. Nice One !
Quite often you want to extract you configuration data into a config file, the Windows INI is still a popular format, well understood by most people who like to get their hands dirty on the command line.
When using ini files, you basically want to do 3 things
This little code snippet does the lot in one go
import configparser
import os.path
from optparse import OptionParser
config = ["-c", "--config", "config", "Configuration File, INI format"]
parser = OptionParser()
parser.add_option(config[0], config[1], dest=config[2], help=config[3])
(options, args) = parser.parse_args()
if None == options.config:
parser.print_help ()
exit(0)
if not os.path.isfile(options.config):
exit(0)
parser.print_help ()
config_file = configparser.ConfigParser(interpolation=None)
config_file.read(options.config)
Basically, import the necessary libraries, then set up the Command Line arg, in this instance -c, but you can add more by calling add_option() with new values
Next call parser.parse_args() to parse the command line args passed to you app. If nothing is there, or ‘config’ is missing, or not a valid file, then bomb out print useful help text
Finally load the ini file by using ConfigParser. I use ‘interpolation=None’ so that the parser does not substitute variables inside the ini allowing me to use $ and other non alpha numeric characters where needed