We differentiate Big Data from traditional data by one or more of the four V’s: Volume, Velocity,Variety and Veracity
1)Volume: Volume is the amount of data generated that must be understood to make data based decisions.
A text file is a few kilobytes, a sound file is a few megabytes while a full-length movie is a few gigabytes.
Example: Amazon handles 15 million customer click stream user data per day to recommend products.
Extremely large volume of data is major characteristic of big data
2) Velocity: Velocity measures how fast data is produced and modified and the speed with which it needs to be processed. An increased number of data sources both machine and human generated drive velocity.
Example: 72 hours of video are uploaded to YouTube every minute this is the velocity.
Extremely high velocity of data is another major characteristic of big data.
3) Variety: Variety defines data coming from new sources—both inside and outside of an enterprise
It can be structured, semi-structured or unstructured.
Structured data is typically found in tables with columns and rows of data. The intersection of the row and the column in a cell has a value and is given a “key,” which it can be referred to in queries. Because there is a direct relationship between the column and the row, these databases are commonly referred to as relational databases. A retail outlet that stores their sales data (name of person, product sold, amount) in an Excel spreadsheet or CSV file is an example of structured data.
A Product table in a database is an example of Structured Data
Semi-structured data also has an organization, but the table structure is removed so the data can be more easily read and manipulated. XML files or an RSS feed for a webpage are examples of semi-structured data.
Example: XML file
Unstructured data: Unstructured data generally has no organizing structure, and Big Data technologies use different ways to add structure to this data. Typical example of unstructured data is, a heterogeneous data source containing a combination of simple text files, images, videos etc
Output returned by ‘Google Search’
4) Variability – This refers to the inconsistency which can be shown by the data at times, thus hampering the process of being able to handle and manage the data effectively.
You can see that few values are missing in the below table
|Department||Year||Minimum sales||Maximum sales|
Data available can sometimes get messy and maybe difficult to trust. With wide variety in big data types generated, quality and accuracy are difficult to control.
Example: A Twitter post has hashtags, typos and abbreviations.
Micro Strategy is one of the few independent and publicly trading Business Intelligence software provider in the market. The firm is operational in 27 Countries around the globe.
Pega Systems Inc. is a Cambridge, Massachusetts based Software Company. It is known for developing software for Customer Relationship Management (CRM) and Business process Management (BPM).
Workday specialises in providing Human Capital Management, Financial Management and payroll in online domain.It is a major web based ERP software vendor.
Power BI is business analytics service by Microsoft. With Power BI, end users can develop reports and dashboards without depending on IT staff or Database Administrator.
SharePoint is a web application that is works with Microsoft Office. SharePoint was initially marketed as Document Management and Storage System.