IoT – Intel Galileo Gen 2 Hello World

Intel Galileo Gen 2 is the Arduino based IoT board that enables to interact with sensors and actutators.  The advantage with Galileo is to make ease of writing Arduino’s sketches for reading values from sensors and sending information to LCD display et all as well as you can write typical program using Node.js,  Python and even Mono to make this as IoT agent/propagator/integrator.

SPI flash is the default storage for firmware and it has embedded Linux 3.8 and Arduino libraries.   The extensive part of Linux can be installed on SD card with capacity up to 32 GB.  This post I share my experience on using SPI part of Galileo.  I use OS X as host computer.

Setting up the stage

Following things are needed to run the Hello World sketches I used in this post using SPI part of Galileo:

  • Micro-USB to USB cable
  • 5V Power Card

First of all connect 5V power card followed by connect the Galileo with your computer using Micro-USB to USB cable.  The USB client side should be connected on Micro-USB as shown below.  Once you connected with your host computer, you can check whether it is connected successfully on /dev folder.  In OS X, it will be like cu.usbmodemXXXX where XXXX is the port number as highlighted below.


You should upgrade the firmware using Intel Galileo Firmware Updater application if you are just unboxing Galileo.

Download OS independant version of Intel Galileo Firware Updater instead of OS X specific.  It requires older version of Java.


The firmware update will take few minutes.  Once this is done you can use Arduino IDE to run sketches.

Setting up Arduino IDE is straightforward in OS X.  Arduino 1.6.5 is compatible on Mac OS X 10.7 Lion or newer.  You should check the availability of Intel Galileo board in the IDE by Tools > Board and Tools > Port.  Very first time you might need to install the board using Boards Manager (Tools > Board > Boards Manager).  You should search Arduino Certified type of board where you can see Intel i586 Boards by Intel that includes Galileo as shown below.

 Now you can select Intel Galileo Gen2 as board and /dev/cu.usbmodemXXXX as port.

In the Arduino IDE, choose File > Examples > 01.Basics > Blink in Arduino IDE and upload it into your board.

The LED in the board will be blinked as highlighted above after uploaded the sketch.

Obligatory Hello World

The real journey of Galileo starts if you play around with sensors.  I bought SeedStudio Grove Start Plus kit that comes with necessary sensors, base sheild, RGB backlight LCD and necessary cables.  I use  the following items for this excercise:

  • base shield
  • sound sensor
  • RGB backlighted LCD.

The base shield is used as an expansion board that is compatible with the Arduino interface on Galileo.  The below figure shows the Ardunio interface.

The base shield will be like below

  
 You should connect the base shield on this interface as shown below.


The sensor and LCD can be connected with Galileo through this shield. Sound sensor to be connected to A0 and LED to be connected to D7 of the base shield. 

// this is for RGB backlighted LCD
#include <Wire.h>
#include "rgb_lcd.h"

const int pinSound = A0; // listen on A0
rgb_lcd lcd;

// threshold value for the sound level
int thresholdValue = 600;

void setup()
{
// print init message on LCD
lcd.begin(16, 2);
lcd.print("Sound Alert!");
}

void loop()
{
// read the value of the sound sensor
int sensorValue = analogRead(pinSound);

lcd.clear();

// If the measured sound level is above the threshold, display
// the value with red background
if(sensorValue > thresholdValue)
{
lcd.setRGB(255, 120, 120);
}else
{
// otherwise green background
lcd.setRGB(120, 255, 120);
}

lcd.print(sensorValue);
// delay time
delay(500);
}

The final outcome as shown below:

Don’t use elephant for your garden work

While learning the new Tez engine and query vectorization concepts in Hadoop 2.0, I came to know that the query vectorization is claimed as 3x powerful and consume less CPU time in actual Hadoop cluster. Hortonworks tutorial uses a sample sensor data in a CSV that is imported into Hive. Then a sample has been used to explain the performance.

The intention of this post is neither explaining Tez engine and query vectorization nor Hive query.  Let us familiarize the problem I have worked before get to know the purpose of this post. :)

One sample CSV file called ‘HVAC.csv’ contains 8000 records that contain temperature information on different building during different days. Part of the file content:

Date,Time,TargetTemp,ActualTemp,System,SystemAge,BuildingID
6/1/13,0:00:01,66,58,13,20,4
6/2/13,1:00:01,69,68,3,20,17
6/3/13,2:00:01,70,73,17,20,18
6/4/13,3:00:01,67,63,2,23,15
6/5/13,4:00:01,68,74,16,9,3

In the Hive, following configurations are specified to enable Tez engine and query vectorization.

hive> set hive.execution.engine=mr;
hive> set hive.execution.engine=tez;
hive> set hive.vectorized.execution.enabled;
      hive.vectorized.execution.enabled=true

I execute the following query in my sandbox  that surprisingly took 48 seconds for a ‘group by’ and ‘count’ on 8000 records as shown below:

select date, count(buildingid) from hvac_orc group by date;

This query groups the sensor data by date and count the number of building for that date.  It produces 30 results as shown below:

Status: Finished successfully
OK
6/1/13  267
6/10/13 267
6/11/13 267
...
Time taken: 48.261 seconds, Fetched: 30 row(s)

Then I plan to write simple program without MapReduce castle, since it is just 8000 records. I created a F# script that read the CSV (note that I did not use any CSV type provider) and using Deedle exploratory library (again, LINQ can also help). I achieved the same result as shown below.

module ft

#I "..\packages\Deedle.1.0.0"
#load "Deedle.fsx"
open System
open System.IO
open System.Globalization
open System.Diagnostics
open Deedle

type hvac = { Date : DateTime; BuildingID : int}

let execute =
	let stopwatch = Stopwatch.StartNew()

	let enus = new CultureInfo("en-US")
	let fs = new StreamReader("..\ml\SensorFiles\HVAC.csv")
	let lines = fs.ReadToEnd() |> (fun s -> s.Split("\r\n".ToCharArray()))

	let ohvac = lines.[1..(Array.length lines) - 1]
				|> Array.map (fun s -> s.Split(",".ToCharArray()))
				|> Array.map (fun s -> {Date = DateTime.Parse(s.[0], enus); BuildingID = int(s.[6])})
				|> Frame.ofRecords

	let result = ohvac.GroupRowsBy("Date")
				|> Frame.getNumericCols
				|> Series.mapValues (Stats.levelCount fst)
				|> Frame.ofColumns

	stopwatch.Stop()
	(stopwatch.ElapsedMilliseconds, result)

In the FSI,

> #load "finalTouch.fsx";;
> open ft;;
> ft.execute;;
val it : int64 * Deedle.Frame =
(83L,
BuildingID
01-06-2013 12:00:00 AM -> 267
02-06-2013 12:00:00 AM -> 267
03-06-2013 12:00:00 AM -> 267
04-06-2013 12:00:00 AM -> 267
...

The is completed within 83 milliseconds. You may argue that I comparing apple with orange. No!.  My intention is to understand when MapReduce is the savior.  The parable of the above exercise is that be cautious and analyze well before moving your data processing mechanisms into MapReduce clusters.

Elephants are very effective in labor requiring hard slogging and heavy lifting. Not for your gardens!! :)

Note that the sample CSV files from HortonWorks is clearly for training purpose. This blog post just take that as an example to project the maximum data-generation capability of a small or medium size application for a period. The above script may not scale and will not perform well with more than the above numbers. Hence, this is not anti-MapReduce proposal.

Unpacking Apache Storm in developer box

It is a long time pending task to evaluate and learn Apache Storm. Storm infrastructure needs Nimbus and Zookeeper.

My intention is to install Storm in my regular Ubuntu single box instead of any cluster environment/VMs, the reason is Apache Storm is just a jar file.

Zookeeper was already installed in my machine as single-cluster mode. I already have other prerequisites Java 6 or greater and Python 2.6.6 or greater.

I simply extract apache-storm-0.9.1-incubating.tar.gz into my app directory. We need to play around with two directories as like
apache-storm-0.9.1
|______ bin
|______ conf

Update the following settings in conf/storm.yaml.


storm.zookeeper.servers:
    - "127.0.0.1"
storm.zookeeper.port: 9191
storm.local.dir: "/mnt/storm"
nimbus.host: "127.0.0.1"
supervisor.slots.ports:
   - 6700
   - 6701
   - 6702
   - 6703

Open three terminal windows, switch to super user and type the following commands respectively from the bin folder.


./storm dev-zookeeper
./storm nimbus
./storm supervisor
./storm ui

The last command opens Storm UI portal at localhost:8080 in browser.

Book Review – Heroku up and running by Neil & Richard – O’Reilly

 

hrk_rc_cat

Neil is a Ruby developer and deploying many applications in Heroku.  His book “Heroku Up and Running” is in early release stage.  Most of the contents are ready and hence review has been made.

As like Heroku ecosystem’s simplicity, this book has only 100 pages with 8 chapters.

As a cloud developer, the first chapter “What is Heroku” is bit bored.  The second chapter “How Heroku Works” is pretty straightforward, simple and nicely written.  A conceptual/flow diagram is the only missing point.  Reader has to go with full load of text.  The third chapter explains performance and scalability on “Dynos” and Postgres database.  It is worth read.

Chapter 4 covers Heroku Regions – Is it that much worth?

Chapter 5 is fully dedicated for Postgres which is helpful for the respective people.  It is a good resource.

Chapter 6 starts with deployment best practices followed by “HOWTO”.  It is well written.

Chapter 7 and 8 are really much needed for any people who want to deploy their app on Heroku.  Neil really puts good effort to on these two chapters to differentiate highly from Heroku manual. :)

The book would be even better if it provide some simple samples than syntax examples.

This book gives me mixed feeling but I suggest this book for its chapter 6, 7, and 8.

You can buy this book at http://shop.oreilly.com/product/0636920027409.do