Tuesday, 2 August 2016

Machine Learning - A simple explanation

I've recently dived into machine learning and in this blog post I will brief about the journey and experiences so far. The interest grew since I first applied NuPIC - Numenta's Platform for Intelligent Computing in anomaly detection.This was a project in Intel's Ultimate Coder Challenge and you can find more about it here.

NuPIC is inspired by neocortex -  a majority of brain mass.It is observed that the neocortex does not differentiate between the stimulus irrespective of the source.What I mean here is that, you can attach eyes to your tongue and somehow you will learn to see! I'm not kidding here, scientists and researchers have tried it.

Coming back to nupic, it is open source and developed by Numenta. My experience with nupic was quite good.But since I didn't develop everything from scratch and this is one demerit if you want to learn, they however have written a white paper which explains everything in detail.

NuPIC uses  Hierarchical Temporal Memory ( previously - Cortical Learning Algorithm) as the guiding algorithm. HTM is based on the fact that our neocortex functions in a hierarchical structure.The neurons at the lowest of the hierarchy represent and map the stimulus or sensory data and pass on the stimulations to the layer above them and as we move up in the hierarchy, we find that the layers respond to more abstract data inputs.

One more important term concerned with HTM is SDR or sparse distributed representation.These can be considered as the datastructures of the neocortex.The sensory inputs are converted into SDR by various techniques - spatial and temporal pooling.I will not go in much detail here.You can find my implementation at this link and the code here.

Well, Intel Ultimate Coder Challenge is over but I am still learning and working on machine learning.Andrew Ng course is very helpful.As of now, I can say that this course is the best if somebody wants to develop an understanding of machine learning.

I say, Machine Learning = Math + Statistics + Probability + Physics + Coding. You also need to have good imagination power to imagine the graphs. And yes graphs are very helpful in ML and in data visualization in general.A simple plot can suggest the kind of problem you have in front ( regression, classification, etc).

Let me take a very simple example of linear regression and we'll try to build an approach to solve the problem.

Suppose we have a data set X = {2, 3 ,4 5} , y = {5,5,10,12}. Given this data set, let's visualize it first.

We haven't defined our problem statement yet.It can be something like - "Given the above data set, predict the value of y at say, 7?" This looks like mathematical problem and can be solved mathematically if one knows the law that governs the X and corresponding y.This law is an equation.In ML it is referred to as a hypothesis. The plot suggests that a straight line equation can solve the problem, so let's use it.

We have y = mx + c . Here, known values are x and y.Given a simple data set that we have, we can easily calculate the unknown terms m and c.We can also solve our problem by using any two data set points and putting them in our equation but then, it will satisfy just those two points and won't generalize.Also, imagine a lot of data points.In such scenario there will be many more unknown variables and we'll require a much more complex equation.Maybe  higher degree polynomial.

ML can solve our problem gracefully.It can come up with a hypothesis which will generalize over a large data set.Above problem is a linear regression problem ( regression because, the data is continuous and linear because the relation between x and y is almost linear ).

Coming up with the hypothesis that best suits our data sets and generalizes to a large extent is the main goal of machine learning.The program learns from data and then comes up with the finest hypothesis.For making our program to learn, we need to  train it with the training data that we have provided.Mostly, the data set is divided between training set and testing set.Testing set is the one which the program hasn't seen so far and will be required to show predictions on that data.

How do we train?ML has something called as Cost Function which is usually the mean-square error function. Practically, what we do is, we assume some random value of the parameters m and c and then put it in our hypothesis. We calculate the out put this hypothesis produces and compare with the actual value (y) at that particular x.We square the difference of the two and repeat it with all the data sets.This gives us the mean-square error value.

But where is the learning? 

Learning is in minimizing the cost function.The cost function is calculated and then the valueo of parameters is found for minimum Cost function ( close to zero ).We update the value of the parameters so that our gradient is pointed towards the minima. This kind of approach is called Gradient Descent. So when you train with the data-set, the program is doing some gradient descent to find out the best fit values of m and c. 

Finally after getting the value of m and c ( the weights ), we are ready to predict for new un-seen values of x!

Above plot shows the best fit line to predict the unknown values.You can see this line passes through mid of the data points.This would have been a better plot with more number of data points.But we get the idea.

I hope this helped in developing a basic understanding of a ML problem statement.

Sunday, 26 June 2016

IOT VAIDYA in Intel Ultimate Coder Challenge - Part 1

Hello people! It's been quite long time since my last blog post.The reason being involvement in Intel's Ultimate Coder Challenge about which I mentioned in my last blog post.So let me give you a little summary of what we've been upto this whole time.

We are team IOT Vaidya and we are developing a Cognitive Healthcare Solutions for rural areas.We will be sensing vital parameters such as ECG, Heart Rate, Blood Pressure, Galvanic Skin Response and Body Temperature and sending it to a doctor who may be sitting in a far off place - far away from rural areas. Now comes the cognitive aspect of the project.Not only will the doctor be able to assess the patient by himself, but also will be assisted by our Machine Intelligence System to decide on key judgments.

While there can be many parameters to implement the Machine Intelligence upon, we are choosing ECG.The reason being it is time series data stream and hence has an inherent Temporal Memory behavior. What all that mean, will be explained on the official challenge's blog after we are done with some implementation of this machine intelligence system.

We are very confident of it working well.

So, getting back to what has been done in the last 5 weeks -

Week 1
This week saw us introducing our project using a block diagram.

Week 2
By the second week we were able to check if our cloud interaction was taking place.This was a simple demo of a laptop(device)  communicating with IBM Bluemix's IOTF Service.

Since the underlying working of this communication is based on a very important IOT protocol - MQTT, we also gave a little introduction on the same.

By the end of this week, we actually showed the working of MQTT protocol in a local environment to get a feel of Internet of Things.We explained in detail what all things are required by a developer to get his/her MQTT environment up and running.

Week 3
Finally, our Intel Commercial IOT Development kit arrived and we got familiar with it's components.We were very delighted to see all those cool tech at one place.Take a look at them yourself.

The kit includes Dell Wyse 3290 IoT Gateway, Intel Curie Board ( Arduino 101/Genuino 101 ) and Grove Sensor Kit.
And the best part was we got one of those sets for each one of us!Happy!

Finally we were able to get our hands on the dev kit and our first experiment was with Genuino 101.We interfaced our Pulse Sensor with Genuino Board and got the output on Processing software.
The output was just as expected and you can view the demo here.
For people having trouble uploading sketch on their genuinos, the blog also explains what needs to be done in those scenarios.

Week 4
After having interfaced the pulse sensor with genuino, next we interfaced Blood Pressure Sensor.In the challenge's blog post, we have explained in detail about the sensor.We had also talked about High Blood Pressure and why is it important to know your number. We learnt that the High Blood Pressure doesn't have visible symptoms so we should get our BP checked for any abnormality in the BP numbers( SYS and DIA).
Here is an image of the Blood Pressure sensor that we are using.

We also introduced two more sensors after this - GSR and Temperature.

Week 5
This week was spent on configuring our Intel IoT Gateway ( Dell Wyse 3290 ) and getting it connected with Genuino Board over Bluetooth LE.
For the demonstration purpose, we interfaced our Pulse Sensor to the genuino board and uploaded a HeartRateMonitor Sketch to it.Next, we ran a python script on the Gateway and enjoyed the view!

Genuino Serial Monitor

Dell Wyse 3290 IoT Gateway Terminal

What is next?
Now that  final look of our project is shaping up (having interfaced almost all the sensors), we are working on addition of cognitive aspect to it.There are many important anomalies out there in healthcare.What we are planning to detect is Cardiac Arrythmia. For now, we will be targeting this important anomaly using Hierarchical Temporal Memory Algorithms. After successful implementation, we will try to detect another very critical situation in India - Cataract. We believe in same old principle that - PREVENTION IS BETTER THAN CURE - always.

IoT Vaidya on Twitter- @iot_vaidya

Wednesday, 25 May 2016

Intel Ultimate Coder Challenge For IOT

Hi Guys!It is with great pride that I announce that my team has been selected in Intel Ultimate Coder Challenge For IOT.This is an international level challenge and there will be five finalists(including us!) competing for First Prize ( 20,000 USD ) and Second Prize ( 10,000 USD ).It had started officially on 23rd May 2016 and will take 8 weeks i.e, till 14th July 2016 to finish.

My team is called IOT VAIDYA.We will be trying to bridge the gap between rural patients and the doctors.This will be a standalone project targeting basic diagnosis to cure recommendation in the native languages.

Guys, get ready for super level of making, breaking and learning.

And yes, I will try to update whatever there is to share with you people.The next will contain details about the amazing kit that our team will be getting.You can also follow the challenge here.Till then stay tuned.Cheers!

Monday, 18 January 2016

IBM Bluemix : IOT Tutorial

It’s been quite some time now that I have been playing with IBM Bluemix platform and applying it in various IOT projects.You can make your application on this platform and bind it to some services.The services can not only be used by the run time applications on bluemix but also can be accessed by programs running on remote machines.This tutorial will be on how to set up IBM Bluemix for using it's IOTF service.The client that I'll be using is a python client.

This particular implementation publishes the CPU utilisation(in percentage) of my computer to the IoTF cloud. Find the source code on my GitHub.


1.You need to register on IBM Bluemix.

2.You should have Linux machine with paho-mqtt package installed for python.

3.Basics of MQTT.

4.Knowledge of Python programming.

So let’s get started!

Assuming that you have registered and are logged into IBM Bluemix,you will see a page something like this :
Few names that are important here : ORGANISATION , REGION, SPACE.

ORGANISATION – It is like your unique Identity on Bluemix.

REGION – There are two regions(US and UK).By default,my region was US.

SPACE – It is like a directory where your work is saved.All the apps and services are added to your space.I have named my space as “dev”.You can give any name.

After creating the space you can add apps or services to your account.I will add a service called Internet of Things which can be found in the CATALOG tab.This service gives you cloud platform to publish or subscribe the data.The data is published or subscribed using MQTT protocol.I will not elaborate on MQTT.As for now, you should know that it is a machine to machine light weight protocol for implementing Internet of Things.
Name your IoT service and Leave it unbound.Then hit create.
You will be asked to add a device.Now device is a “thing” which you want to get connected to the cloud.This “thing” will be called as device hereafter.Any device which intends to publish or subscribe the data needs to be registered first.We do it by adding this device.
Name your device.Device name will be referred to as device type.You can add description of your device.This is not necessary though.Hit next and you will be asked to add device ID.Now this can be any unique ID.Usually we give MAC address of the device.Any unique sequence of characters will do.Remember that “:” i.e, colons are not valid so you need to remove these from your MAC address.

Leave the authentication token as blank as it will be automatically generated.Hit next and you will see a page containing all the important credentials required to authenticate your device.You need to put this information in your code that runs on your machine.
That’s it.Now add this newly created device.

You will see your device added in the list.The status of the device initially will be something like this.

Now,I will run the python code on my machine and let’s see what happens.

You can see the data on the Bluemix as well.Click on the device name and refresh the prompted page.You will see the data being published by the machine/device.
IBM Bluemix provides various other interesting services and supports many runtime languages to build applications.Do explore them and keep learning!

If you found this blog post helpful please like and subscribe for more such articles.

Friday, 1 January 2016

Galileo-Link : Connect THINGS to Social Network

Hello people!The following project is a basic implementation of what can later give rise to a dedicated social network for sensors. Have you ever wondered how would it be if all your inanimate things can talk on social network like Facebook, twitter? Green-house updating it’s health on it’s timeline? This project is an effort to give social life to the “things”.

I have implemented the project using Intel Galileo board as the sensor gateway, you can use any other board running Linux OS with python installed. I tried making my home talk. That is, posting sensors data installed on Intel Galileo kept in my room to the Facebook account by name “Galileo Link” wall. So all we need is access to Facebook Graph API and a python script.

To access the Graph API, we need to create a Facebook App on facebook developers page. Now follow these steps:
1. Click Add a New App -> Choose platform WWW -> Choose a new name for your app -> Click Create New Facebook App ID -> Create a New App ID -> Choose Category -> Click Create App ID again.

2.Go back to Apps Dashboard -> Select the new app -> Settings -> Basic -> Enter Contact Email.

3.Go to Status & Review -> Do you want to make this app and all its live features available to the general public? -> Toggle the button to Yes -> Make App Public? -> Yes. This will enable others to see posts by your app in their timelines – otherwise, only you will see the wall posts by the app.

4.Now – you should see a green dot next to app’s name, and the text This app is public and available to all users.

5.Make a note of the App ID and App Secret (Click Show next to it; you will be asked to re-enter your Facebook password). Now that the app has been created, the next step is to get Facebook OAuth Token.

Steps to get Facebook OAuth Token :

1.Go to Graph API Explorer-> In the Application drop down -> Select the app created -> Click Get Access Token -> In Permissions popup go to Extended Permissions tab -> Select manage_pages, andpublish_actions These permissions will allow your app to publish posts acting as the page -> Click Get Access Token -> You will see a message saying “{App} would like to post publicly to Facebook for you. Who do you want to share these posts with?” -> I chose Public for maximum visibility – as I wanted to post to a public page.

2.Make a note of the short-lived token shown in Graph API Explorer.

3.Facebook has deprecated offline access, the next best thing is long-lived token which expires in 60 days. We will convert the short-lived access token noted above to a long-lived token. For that, fill in the values in the URL below and open it in a browser:

Replace the values in {}.

4.You should see access_token={...}&expires={...}. This new access_token is the long-lived token we will use in our Python script.

5.long-lived token will also expire eventually, be prepared to perform this Step 3 again before that happens!

6.We will use Facebook Python SDK to access Facebook’s Graph API. You can install it using pip: pip install facebook-sdk.

NOTE : Some might get AttributeError: ‘module’ object has no attribute ‘GraphAPI’ . This maybe because of bad installation or corrupt packages. Other facebook packages may have been installed and it might be a possibility that the python script is importing from wrong path.
You might want to go to those directories where Facebook instances are installed. To know the directory. create a python script :

import facebook
print facebook.__path__

The path to the facebook will be displayed.Go to the directory delete facebook packages and then re-install it again.Problem solved!

Now the next important thing is the page ID which can be found in About tab of the Facebook account (usually it is a big number). Finally the code on Intel Galileo will post to Facebook wall of the Account.
Follow this link to my github and find the python script for this project.
Here is the link to my Galileo-Link!
Do like,follow and share this blog post if you found it useful.

Tuesday, 29 December 2015

Install LAMP on Linux

Following is the step by step procedure to install Apache MySQL and PHP on Linux.Together these are called LAMP.

First of all you need to make sure that you are logged in as the root user.

#apt-get install update #apt-get install apache2

Install php using following command.

#apt-get install php5 libapache2-mod-php5

After installation, you need to make certain changes as follows.

#gedit /var/www/html/phpinfo.php

Next step is to install mysql-server.

#apt-get install mysql-server

During the installation there will be several prompts.

set the password for MYSQL “root” user.
#apt-get install libapache2-mod-auth-mysql php5-mysql phpmyadmin

select the server as apache2 when prompted.
again there will be prompts for password ; make it the same as before
There are few changes to make in the configuration file apache2.conf

#echo -e “\nInclude /etc/phpmyadmin/apache.conf” >>/etc/apache2/apache2.conf

And you are all set to create your php website.

Arduino in Linux : Some common errors

I wanted to run Arduino IDE in ubuntu 14.04. Though it earlier looked simple but while I wrote a simple blinky program following error occured.
avrdude: ser_open() : can’t open device “/dev/ttyUSB0” : Permission Denied 
ioctl(“TIOCMGET”): Inappropriate ioctl for device
Note the device name “/dev/ttyUSB0” is a kind of module that caters to the connectivity and serial communication of the arduino board here.

So I began googling and after few searches I found the solution for the problem.

On the command line I wrote:
~$sudo usermod -a -G dialout <username>
~$sudo chmod a+rw /dev/ttyUSB0
where <username> is the name of your username in ubuntu.