Brief summary:
If you think data is gold, then what we call big data, Is it diamond? Yes! Today we are living in a very exciting world, where the data is more precious than ever before. No matter what business you are doing, each and every company embracing big data right away, one of the fastest-growing segments of the IT industry, and without exception everyone is trying to get value from it.
Where it starts?
I would like to quote an article which was released on may 2011 as a part of research done by MGI and McKinsey’s Business Technology Office titled with “Big data: The next frontier for innovation, competition, and productivity” where the study says the exponential growth of big data and how to anticipate the future, later on October 2012 Gartner, the leading IT research and advisory company publicized a report about the subsequent job opportunities and how would the organisations need to shift their focus to transform, discover, and optimizing their business and also how to innovate industries according to the advancement of big data, in the meantime the Harward Business Review published an article called “Data Scientist: The Sexiest Job of the 21st Century” about the importance of people like data scientists, termed them as a new breed and estimated their shortage becomes a serious constraint in various sectors. But the actual impression has been started by Google, the search engine giant has been released their papers in 2003 and 2004 about their Google file system (storage mechanism) and map reduce (computation mechanism) and how to tackle this un expected un structured growth of data, from then onwards the open source distributing storage and processing framework like Hadoop has came into the picture, and then the companies like Cloudera, Hortonworks and MapR have been built their own tools on top of Hadoop and distributed their services and support towards to handle enormous amount of the data,
On a lighter note the data became big starts many years before the current buzz around, but as I mentioned above is the most recent years, after the industries had serious glimpse on big data.
Who generates this data?
Historically the data is being generated by employees or workers of a company into the computers. But now it’s happening to see generally users posting their own data through various social media channels, even machines also no exceptions like video surveillances, so the scalability is much higher than earlier, and the amount of data producing by these users or machines with high volume, velocity and variety is at a staggering pace, in addition to that the decrease in digital storage costs and increase in computing power is one more aid for the growth of this digital foot prints, Even though dominating sectors like ecommerce, social media having direct impact on this technology, but it is already prevalent in many industries like BFSI, manufacturing, life science/healthcare, energy, retail, government, media, aviation, aeronautical and telecommunications. Eventually every industry is producing the data, but the amount of the data generating is much higher rate than earlier.
Good analogy, for every second that the Large Hadron Collider at CERN can generate 40 terabytes of data. For every 30 minutes that a Boeing jet engine creates 10 terabytes of operations information. According to 2014 IDC report from 2013 to 2020, the digital universe will grow by a factor of 10 – from 4.4 trillion gigabytes to 44 trillion. It more than doubles every two years.
How the education system is helping?
As new innovation reach the market, new challenges will arise, so the question arise how do we continue to pace with new progress, how do we guided towards better future, Organizations everywhere will continue to ramp up their hiring, but we won’t have enough people with Big Data skills to go around, and those unable to adopt these approaches will be rapidly left behind. The companies itself facing the challenges to get the right information at the right time from gigantic amounts of data, that’s why surging in the demand for the people who can coax treasure out of messy unstructured data and deliver valuable insights has become overwhelming now a days.
There is a gap between the demand and supply of big data-savvy professionals and technologists, as we know the data experts will be a scarce and valuable commodity, so to tackle this challenge the education system has been doing tremendous job seamlessly. There are so many universities and IITs offering postgraduate and undergraduate programs to addressing the issues in this area in a focused manner, some of those are explicitly on data science, big data and predictive analytics.
Now most of the education systems have been tremendously shifted from classroom to online so as to learn at our own pace, and the fact is we need only internet connection. MOOCs (massive open online courses) providing higher education partnered with top world class universities like Harvard, Stanford, MIT and many more, they are open, online and free. Especially the companies like Coursera, Edx, Udacity and Udemy offering the best courses related to data science, big data, and the most of the courses are combination with basics as well as advanced statistics and computer sciences, in addition to this there are various interactive online platform providers like Data Camp, Code School and Dataquest which are offering remarkable data science courses via web with nice user interface and including datasets online for practice. Even Big data companies like EMC and IBM offering the courses through their own education channels, some of them offering grading and certificates for the students who were accomplished the courses successfully. Some providers also giving an opportunity for the users who are expert in some areas can create their own course on these platforms for open source marketing.
Source: http://hpc-asia.com/big-data-the-recent-history-its-evolution-and-challenges/
