Introduction to Big data

Introduction:

Talking in general terms, today Data is Everywhere. Our each and every activity on our devices like mobiles, PCs, TVs etc  involves data. Let's consider some of our normal day to day activities in present times:-

• We use Facebook for uploading pictures, liking someone's pictures, sending messages etc
• We go to sites like Flipkart, Amazon, myntra and searches for some products, buy some products etc
• We go to a shopping mall and buy products and pay through our cards.
• We use smart watches, smart bands, sensors etc

All of the activities discussed above are our day to day daily life activities. Each of these activities produces data in one form or the other.like sending a message, searching for a product, paying from cards, getting health info from bands etc everything generates data.

Today organizations store all these data from our activities for their business purpose.

         

Traditional structured data:-

This is the data we have in old times ( present in today's world as well and will remain with us forever). Structured data is the one which can be stored and represented in tabular format like Employee data, Student data etc. For processing and managing this type of data we use DBMS.(like sql queries,joins etc) Even today also we have structured data and it will remain with us forever. It is getting used for OLTP purposes (i.e for supporting day to day transactional purposes like banking sector).

Modern Data:-

Today data not needs to be structured data only. It can be unstructured as well i.e. the data which can't be stored in tabular format like audio, video , social media logs etc

What is Big data:-

Big data can be structured , unstructured  or any type of data having following characteristics:-

1. Volume:- (Data at rest) i.e. size of data , whenever data exceeds certain limits that it become difficult to process it using traditional tools then it becomes Big Data. For e.g. data present in  data warehouses is of huge amount and it can be termed as Big data.
2. Velocity:- (Data in motion) It refers to speed of data. Data which is getting produced fast needs to be process fast so as to do the processing in real time. For e.g. real time credit card fraud detection.
3. Variety:- (Data in many forms) i.e. structured ,unstructured, semi structured data etc.
4. Veracity:- (Data in doubt) It refers to correctness of data i.e. how much data is free from noises and uncertainties etc. For e.g. Today we are using data for generating various insights ,decision making etc but still 1 in 5 business users does not believes the insights generated by the data. Doubt is always there and that is what we call it as veracity.

These are the 4 main Vs of  Big data i.e if  our data have any of the above characteristic in it then it can be considered as Big Data.

It consists of many other Vs as well (like valence, vulnerability, validity, variability etc). All of them combines together to make most important V of  Big Data i.e. Value.

It is very difficult to process Big Data with the help of traditional tools like DBMS. So Big data has many tools and technologies of its own.

Why Organizations store each and every data and What are the Advantages of storing Big data?

As we already discussed that today Organisations store each and every data produced by us. They store it to gain various knowledge discussed below from that data by using AI/ML techniques:-

1. Knowing Search Patterns:- Depending upon what we are searching, how and when we are searching it helps an organization to know about our search patterns and our interests  etc.
2. Semantic Analysis:- If we are sharing any post on Facebook having text in it then using that, Facebook can do semantic analysis from that post i.e. it will come to know about our mood, liking, dislikings etc And in today's time it can even do semantic analysis from our likes, audio, video posts , youtube comments,twitter comments  etc 
3. Detecting payment card frauds:- By capturing our payment patterns banks can detect if any fraudulent activity is getting performed on our cards

These are just few reasons of storing our data there can be many more reasons for storing our data by organisations. The ultimate goal of that is to boost the business and gain competitive advantage over other similar businesses.

Advantages of storing the data:-

There is a very popular quote saying that:

" If you torture the Data long enough, it will confess to anything"  :- Ronald H. Coase

That means more the organizations dig into that data more info they will get about their customers i.e. they can serve their customers in more better way as compare to their competitive organisations. 

Storing the data and gaianing various knowledges discussed above from that data will help an organisation to get important insights from the data which will help them in better decision making and hence will be very helpful to attract more and more customers.

In short storing this data and getting insights from that help organisations to Understand each and every Customer separately . (And it is possible only because of Big Data)

For e.g. with the help of insights organisations are able to make different types of product recommendations to each and every customer based upon their interests. i.e. 

• If you go to some site in internet you will get some ads based upon your interests. Another person if go to same site he will see some other ads based on his interests.
• We get notification from myntra that product you are searching for 5 days ago is on 80% off whereas other person might get same type of notification for other product.

So we came to know how organisations can use our data to derive knowledge from that and understand every customer separately.

As we are done with general introduction of Big data now lets see how Big Data is Revolutionizing the Banking Domain.