The term "Big Data" is used for referring to datasets that are typically in the order of petabytes, exabytes and even larger. The data could be either structured or unstructured. In simple terms any data that has a definite structure and could be stored easily in relational databases could be considered structured. Unstructured data does not have an identifiable structure, examples are email, images, word documents, phone conversations, etc.
With the advent of cheap digital storage and advancements in computational power we are seeing an explosion in the growth of digital data. It is estimated that 90% of the data in the world today has been created in the past two years. According to IDC, in 2011 more than 1.2 zettabytes of information will be created and stored and by 2020 will grow to 44 times that of 2009.
The big question is what does it mean for us?
Should this scare us? Governments and companies acting like big brothers with their ability to collect and retain personal information about us and our behaviors through records of our searches, chats, health records and any other form of interactions that can be digitally tracked.
Shelving the privacy concerns for now let us see how it can benefit the businesses and consumers. Businesses can analyze this data to gain efficiencies within their organization and strategic advantage over their competitors. Consumers will greatly benefit from the innovations that will affect some core sectors like health care where effective use of data will help doctors make more informed decisions in treating patients. Real time traffic and weather data from Mobile phones and GPS devices will help avoid traffic congestions.
What opportunities does it bring for engineers and scientists? Big Data is still mostly a buzz-word and adopted by larger companies - internet giants like Google, Yahoo, Facebook; retailers like Walmart; financial services companies. So there are lot of businesses that in coming years will be looking for talent to manage and use big data and associated services.
In the upcoming blog I will talk about some of the popular tools associated with Big Data.
No comments:
Post a Comment