What’s in store for Asian smallholder farmers in the Big Data hype?

By Camille Anne Mendizabal (World Agroforestry Centre) | December 29, 2017


Exploring how Big Data’s potentials can be used to enhance Asian farmers’ climate resiliency.

 

Smart tractors, agribots, survey drones, texting cows—these may seem like agriculture buzzwords, but with Big Data accelerating agricultural digitalization, these may soon come into fruition and be seen in farms in Asia.

What caused the hype?

The information age we are in now provided four technological milestones which paved the way for the digitalization of agriculture through Big Data.

First among these milestones is the improvement of peoples’ access to smartphones and data services. This provided opportunities for them to access agriculture information that could guide them in making farm-related decisions.

Secondly, the increased availability of cheaper smart agriculture sensors also helped farmers in monitoring their farms and adapting their practices to changing climatic conditions and environmental factors.

Another milestone that hastened the digitalization of agriculture is the improvement of the quality of satellite information and satellite images which led to better and more updated climate forecasts. Lastly, in our enhanced ability to analyse and interpret data provides better for support climate-smart agriculture (CSA) research and development efforts.

What is Big Data’s niche in CSA?

If the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) succeeds in utilizing Big Data, its biggest impacts can be seen in improving the following stages in the market-value chain: planning, selecting quality inputs, on-farm production, storage, and access to markets.

In the planning stage, Big Data is deemed most useful in helping farmers decide what to plant and when to plant it. It can also be used to guide farmers in selecting high-quality inputs. During production stage, applied data can potentially improve precision and adaptability of farming interventions.

Digitally warehouse receipts and digitally-enabled harvest loans may help reduce postharvest losses while they are stored. Moreover, the improved climate forecasts can also provide farmers with timely reminders and alerts on climate-related risks which are useful in monitoring farm operations and preventing yield loss.

Harnessing Big Data’s potential also enables the combination of climate forecasts with micro-insurance systems to further enhance farmers’ resilience to climate risks.

Big Data can also be sourced from social media. Through this, we can make the most out of the farmers’ groups established through social media platforms, Facebook posts and tweets by using them to build information database.

During the Joint CCAFS SEA-SA CSA Workshop in Hanoi, a special panel discussion tackled the potential of using Big Data to increase agricultural productivity, and at the same time manage climate-related risks. Photo: Duong Minh Tuan/ICRAF

Can smallholder farmers benefit from Big Data?

Despite the rosy picture that Big Data presents, it cannot be denied that we still have a long way to go before we can reap the benefits from it and before these benefits trickle down to smallholder farmers. As Andrew Jarvis, one of the Flagship Leaders of CCAFS said:

Big Data provides huge promise, but a handful of success stories for smallholder farmers.”

Dr. Leocadio Sebastian, CCAFS Southeast Asia programme leader, raised concerns about how using big data can be used further widen the digital divide. As of now, only commercial farms have access to technologies which can make sense of big data.

Unfortunately, 76% of the farmers in Asia are smallholder farmers, the majority of which do not have access to these technologies. Hence, the challenge now is for CCAFS to help make it work for this 76%.

Social differentiation in access and illiteracy in using these technologies also pose a challenge in this digitalization. Thus, CCAFS should work on downscaling information from forecasts to something more comprehensible and more relevant to farmers’ context.

How can CCAFS make Big Data work for smallholder farmers?

As of now, there is an insufficient publicly available data on agriculture which can be used to build a sustainable data ecosystem that scientists, extension workers and farmers can access. Building an information ecosystem on CSA that is more accessible to people and resolving data privacy issues could help address such problems.

Moreover, building the capacity of a new generation of agricultural scientists and field agronomists to enhance their skills not only in analyzing, and interpreting data, but more importantly in providing farmers with comprehensible, personalized, and actionable information should now be prioritized.

Creating an enabling environment for establishing public-private partnerships can also help resolve privacy issues in utilizing big data and can help maximize available technologies owned by public and private sectors to further develop information services for farmers.

If these abovementioned challenges are resolved, the rosy picture of modernized, climate-smart agriculture that now seems as a hype can finally be turned into reality.

Camille Anne Mendizabal is the junior communications specialist for the World Agroforestry Centre Philippines. She is also a communication consultant with the CCAFS SEA program.


Article Disclaimer: This article was published by the CCAFS-CGIAR and retrieved on 01/06/2018 and posted here for information and educational purposes only. The views and contents of the article remain those of the authors. We will not be held accountable for the reliability and accuracy of the materials. If you need additional information on the published contents and materials, please contact the original authors and publisher. Please cite the authors, original source, and INDESEEM Inc. accordingly.


Using Big Data to Help Combat Malnutrition in Africa

Cropped-Image-624x274
Source: Farming First, 2017

 

Debisi-115x150
Dr. Debisi Araba, CIAT

 

In this guest blog for the #SDG2countdown campaign‘s week on SDG2.2: ending malnutrition, Dr. Debisi Araba outlines a new big data initiative designed to detect malnutrition before disaster strikes. Dr. Araba is a member of the Malabo Montpellier Panel, set up to provide evidence and promote dialogue for better outcomes in Agriculture and Food Security in Africa, and serves as Africa Director for the International Center for Tropical Agriculture (CIAT.)

Chronic malnutrition affects one in four people in sub-Saharan Africa. This increases the region’s vulnerability during food crises and compromises the continent’s overall development. We know we need to improve the way we respond to food crises to protect African food production systems, farmers, and the people who depend on the food they produce.

Food crisis response

Currently, responses to food crises are reactive, not proactive. Signs of malnutrition may not be apparent until a food crisis erupts, and decision-makers lack the data to combat crises, making response coordination difficult.

There are various forces that influence nutrition and it can be difficult to identify how they converge to cause widespread problems. In addition, most governments and aid organizations use multiple metrics and separate tracking systems to measure malnutrition. With so much data to absorb, it can be easy to miss early indicators of trouble brewing before a crisis kicks in.

This makes it impossible to form proactive food policies and escape the trap of constantly reacting to disruptions rather than getting ahead of hunger. Interventions are also limited to the household or community level and rarely focus on national and regional systems.

How data will help

An innovative new approach to collating and analyzing large sets of data could enable this shift to early action. Through machine learning, computer programs track complex and constantly changing data from multiple sources in order to “learn” from them and make predictions.

The International Center for Tropical Agriculture (CIAT) is applying machine learning technology to search for early signs of potential crop failures, drought, rising food prices, and other factors that can trigger food shortages. Over time, this bespoke system – known as the Nutrition Early Warning System (NEWS) – will become “smarter” and more accurate so that data can be used to predict the likelihood of malnutrition threats before they occur, while also suggesting mitigating measures.

NEWS would enable governments, donors, farmers, health care providers, NGOs and food companies to contribute towards and implement more rapid, tailored interventions.

CIAT will coordinate the development of NEWS, to be deployed in collaboration with partners to alert decision-makers to nutrition threats well ahead of a crisis. Initially, CIAT will use NEWS to focus on boosting nutrition in sub-Saharan Africa. By picking up food shortage triggers, the system will give relief agencies, donors and governments information they need to make informed decisions about agricultural policies and programmes.

Ongoing surveillance is expected to provide multiple recommendations for future nutrition interventions. The recommendations can be tailored to the needs of individual countries through national “nutrition dashboards”.  These will further refine insights available through NEWS. The dashboards will be accessible via a secure website that will regularly monitor and post updates on key nutrition and food security indicators.

32559796114_794e773c66_k-543x362

 

Refining NEWS for Africa

CIAT, which leads the CGIAR Platform for Big Data in Agriculture, has already seen success using big data approaches to tackle agricultural challenges. In 2014, some 170 farmers in Colombia avoided potentially catastrophic losses after CIAT experts used a machine learning algorithm to analyse weather and crop data. It revealed drought on the horizon, and farmers were advised to skip a planting season, saving them more than US$3 million.

We now need to work with partners to track indicators of malnutrition in West, East and Central, and Southern Africa and to create and fully develop NEWS.

The NEWS white paper calls for collaboration between governments, development and relief agencies to find robust methods to track malnutrition indicators. In particular, it urges potential partners who want to harness big data to address fundamental challenges linked to agriculture and nutrition in the developing world, to join CIAT’s effort to create and fully develop the potential of NEWS across Africa.


 


Article Disclaimer: This article was published by the Farming First and retrieved on 08/01/2017 and posted here for information and educational purposes only. The views and contents of the article remain those of the authors. We will not be held accountable for the reliability and accuracy of the materials. If you need additional information on the published contents and materials, please contact the original authors and publisher. Please cite the authors, original source, and INDESEEM accordingly.


 

 

 

 

 

 

 

 

The disruption starts now: new big data program promises to shake up agriculture

Big-Data-Platform-mob_lo
Source: CIAT Blog 2017

By | May 15, 2017


A new multimillion-dollar initiative plans to disrupt food production across the developing world, with the aim of making it more productive, efficient and resilient – all through the power of information.

The CGIAR Platform for Big Data in Agriculture is jointly led by the International Center for Tropical Agriculture (CIAT) and the International Food Policy Research Institute (IFPRI), with tech giants IBM and Amazon among a list of high-level partners.

It brings together thousands of experts, from crop scientists to computer programmers to collect, process and analyze vast amounts of data on crops, weather, soils and more, with the aim of producing some of the most precise and reliable recommendations for farmers, governments and policymakers in developing countries.

The USD$30 million initiative will be officially launched today at the Information and Communications Technology for Development (ICT4D) conference in Hyderabad, India.

 

 

 

 

 

“It’s time for smallholder farmers to stop looking at the sky and praying for rain,” said Andy Jarvis, a Research Director at CIAT.

“With enough data and enough analysts, we’ll be able to say if the rains will be late or on-time. We’ll be able to say which crops to plant, when to plant and how much fertilizer or water to use. We’ll be able to anticipate shocks, reduce risks and maximize opportunities for profitable, sustainable agriculture.”

Early efforts by CIAT to apply so-called big data approaches to agriculture in Colombia have proven successful. In 2013 a team of analysts ran studies using decades of data from the country’s meteorological office and rice growers’ federation. The resulting recommendations on sowing times are estimated to have saved farmers in the country’s Córdoba Department around USD$3.6m in input costs in a single season. CIAT and its partners have also used big data approaches to tracking deforestation in the Amazon in near-real time.

But the benefits of the big data revolution have not yet reached the vast majority of smallholders and policymakers in developing countries. The spread of smartphones and internet connectivity to many rural areas means many farmers are now better able to generate, share and receive important data to help guide agricultural decisions and investments.

“There’s no reason for precision farming to be the preserve of the fortunate few anymore,” continued Jarvis. “While the data revolution has been a boon for farmers in richer countries, it needs to be democratized so that the world’s 500 million smallholders can benefit too – after all, they produce 70% of the world’s food.”

Others Platform partners in the include the Bill & Melinda Gates Foundation, the World Bank, the universities of Penn State and Michigan State, Kings College London, and PepsiCo, which has pioneered the use of big data to manage supply chains for consumer goods.

“With enormous expertise and processing power now at our disposal, this is the next frontier in agricultural research-for-development,” said Jawoo Koo, a Senior Research Fellow at IFPRI, and an expert in spatial data and analytics. “Better use of data can help drive better policy decisions, helping solve development problems more quickly, cheaply, and at a greater scale than before.

“If we’re going to achieve the United Nations’ Sustainable Development Goals of increasing food production, reducing poverty and tackling climate change, one of the quickest ways will be to close the digital divide between rich and poor farmers. This will help ensure the world’s farmers and policymakers are making informed choices that produce the biggest impacts. The CGIAR Platform for Big Data in Agriculture aims to do exactly that.”

The Platform will focus on three priority areas:

1. Organize – data on soils, climate, crops and more will be organized, standardized and made publicly available by the organizations that generate it. The Platform will begin by prioritizing the free and open sharing of data held by researchers at CGIAR – a global network of agricultural research organizations.

2. Convene – foster new partnerships between the agricultural science and technology sectors in order to bring together the best minds, and accelerate progress towards achieving the United Nations’ Sustainable Development Goals.

3. Inspire – put the data and partnerships into practice via a USD$4m fund to support innovative projects with big data approaches at their core, such as real-time monitoring of pest outbreaks, or site-specific recommendations for farmers on water and fertilizer use.

 


Additional information


The CGIAR Platform for Big Data in Agriculture aims to harness the capabilities of big data to accelerate and enhance the impact of international agricultural research. The six-year initiative will provide global leadership in organizing open data, convening partners to develop innovative solutions, and demonstrating the power of big data analytics through inspiring projects that focus on improving agriculture in developing countries and informing policymakers. bigdata.cgiar.org.

The International Center for Tropical Agriculture (CIAT) is a scientific research organization committed to sustainable food production and improving rural livelihoods in Africa, Asia, and Latin America. As well as developing new techniques and approaches to make agriculture more profitable, competitive and sustainable, for 50 years CIAT has been a trusted provider of impartial advice on agricultural and environmental issues to governments and policymakers all over the world. www.ciat.cgiar.org

The International Food Policy Research Institute (IFPRI) seeks sustainable solutions for ending hunger and poverty. IFPRI was established in 1975 to identify and analyze alternative national and international strategies and policies for meeting the food needs of the developing world, with particular emphasis on low-income countries and on the poorer groups in those countries. www.ifpri.org.

Both CIAT and IFPRI are part of CGIAR, a global agriculture research partnership for a food-secure future. Its research is carried out by 15 centers in close collaboration with hundreds of partner organizations. www.cgiar.org


Article Disclaimer: This article was published by the CIAT Blog 0and retrieved on 05/15/2017 and posted at INDESEEM for information and educational purposes only. The views and contents of the article remain those of the authors. We will not be held accountable for the reliability and accuracy of the materials. If you need additional information on the published contents and materials, please contact the original authors and publisher. Please cite the authors, original source, and INDESEEM accordingly.


 

Your Pot Habit Is Making Climate Change Worse

shutterstock_205435432
Photo Credit: stdesign/Shutterstock

Using Big Data to Predict Terrorist Acts Amid Privacy Concerns

big-data-580x435


By | October 13, 2016


Before Ahmad Khan Rahami planted bombs in New York and New Jersey, he bought bomb-making materials on eBay, linked to jihad-related videos from his public social-media account and was looked into by law enforcement agents, according to the Federal Bureau of Investigation.

If only the authorities had connected the dots.

That challenge — mining billions of bits of information and crunching the data to find crucial clues — is behind a push by U.S. intelligence and law enforcement agencies to harness “big data” to predict crimes, terrorist acts and social upheaval before they happen. The market for such “predictive analytics” technology is estimated to reach $9.2 billion by 2020, up from $3 billion in 2015, according to research firm MarketsandMarkets.

It’s the stuff of a science-fiction movie like “Minority Report,” in which Tom Cruise played a Washington cop who used technology to arrest people before they carried out crimes. It’s also a red flag for privacy advocates already fighting U.S. spy programs exposed by Edward Snowden and the FBI’s demands that Apple Inc. help it hack into encrypted mobile phones.

The idea is to make sense of the vast and disparate streams of data from sources including social media, GPS devices, video feeds from street cameras and license-plate readers, travel and credit-card records and the news media, as well as government and propriety systems.

‘Fundamental Fuel’

“Data is going to be the fundamental fuel for national security in this century,” William Roper, director of the Defense Department’s strategic capabilities office, said at a conference in Washington last month.

For the first time, the White House released a strategic plan on Wednesday to advance research and development of artificial intelligence technology, including to predict incidents that may be dangerous to public safety.

Weeks before Rahami allegedly carried out the attacks in September, he bought circuit boards, electric igniters and ball bearings — all of which are known bomb-making materials, according to charging documents from the FBI.

In previous years, he was flagged by U.S. Customs and Border Protection and the FBI after he made trips to Pakistan and after his father told police he was a terrorist, before recanting the remark.

Law enforcement agents could have been tipped off that Rahami was moving toward an attack had all of those data points been culled together in one place, said Mark Testoni, chief executive officer and president of SAP National Security Services Inc., a U.S.-based subsidiary of German software company SAP SE.

“This is a big data world now,” said Testoni. He said his company has developed a computer platform for doing predictive analytics that is being used in a limited way by a Defense Department agency and by a national security agency. He declined to name the government customers or specify what they are doing.

The technology to predict events is only in its infancy, Testoni said. National security and law enforcement agencies also have different rules when it comes to obtaining and using data, meaning there are walls between what can be accessed and shared, he said. U.S. law enforcement agencies, for example, need a court warrant to access most data.

Big Brother

Privacy advocates express concern about the “Big Brother” implications of such massive data-gathering, calling for more information and public debate about how predictive technology will be used.

“There’s often very little transparency into what’s being brought into the systems or how it’s being crunched and used,” said Rachel Levinson-Waldman, senior counsel to the National Security Program at the Brennan Center for Justice at New York University School of Law. “That also makes it very hard to go back and challenge information that might be incorrect.”

Computer algorithms also fail to understand the context of data, such as whether someone commenting on social media is joking or serious, Levinson-Waldman said.

Testoni’s company and others such as Intel Corp. and PredPol Inc. are among a handful of firms pioneering the use of predictive analytics and artificial intelligence for clients from local police departments to U.S. national security agencies.

More than 60 local police departments in the U.S. have started making use of a service sold by PredPol, which calls itself “The Predictive Policing Company,” to forecast where crimes might occur based on past patterns, said co-founder Jeff Brantingham.

What, Where, When

Its system, developed in collaboration with the Los Angeles Police Department, uses only three types of data: what type of crime occurred, when and where, Brantingham said.

Then, a software algorithm generates the probability of crime occurring in different locations, presented as 500-foot-by-500-foot squares on a computer display or a printed map. With that insight, police departments then can make decisions about how best to apply their resources, such as sending cops to a high-risk area, or which security cameras to monitor, Brantingham said.

PrePol’s system doesn’t make predictions about who will commit a crime, so it stops short of a system that might identify a terrorist in the making.

“Interdicting places is, by and large, an approach that is more in line with protecting civil liberties than interdicting people,” Brantingham said.

Even with such limits, privacy and civil liberties groups oppose the use of predicting policing technology as a threat to the Constitution’s promises of equal protection and due process.

‘Fortune-Teller Policing’

“This is fortune-teller policing that uses deeply flawed and biased data and relies on vendors that shroud their products in secrecy,” Wade Henderson, president and chief executive officer of the Leadership Conference on Civil and Human Rights. “Instead of using predictive technology to correct dysfunctional law enforcement, departments are using these tools to supercharge discrimination and exacerbate the worst problems in our criminal justice system.”

eBay, Amazon

Vast databases that companies have created for online commerce and communications could help law enforcement and national security agencies build predictive systems if they are allowed to tap into them. Technology companies have terms of service that set out how much personal information can be kept and sold to outside companies such as advertisers, and most resist handing over such data to the government unless a court orders them to do so.

Predictive analytics are already being used by companies like eBay Inc., Amazon.com Inc., and Netflix Inc. to crunch their users’ Internet activity to forecast what they might be interested in. Companies like Facebook Inc. and Twitter Inc. have access to over a billion social-media accounts. The storehouse of data on Americans will only grow with digital feeds from Internet-connected appliances and wearable devices.

Social media, in particular, is a valuable tool in tracking potential terrorist attacks, said Eric Feinberg, founding member of the Global Intellectual Property Enforcement Center, which is a private company. His firm has patented technology that can scan for hashtags across different social media platforms and in different languages for communications that indicate terrorist planning.

“Our software is about pattern analysis,” Feinberg said. “We focus on the communications stream.”

‘Open Source Indicators’

The U.S. government is working on initial efforts to gain insight into global social and political trends.

A program under the intelligence community’s research arm called Mercury seeks to develop methods for continuous and automated analysis of intercepted electronic communications “in order to anticipate and/or detect political crises, disease outbreaks, terrorist activity and military actions,” said Charles Carithers, spokesman for the Intelligence Advanced Research Projects Activity.

The agency also previously funded the Open Source Indicators program, which “developed methods for continuous, automated analysis of publicly available data in order to anticipate and/or detect significant societal events,” such as mass violence and riots, mass migrations, disease outbreaks and economic instability, Carithers said.

CIA Forecasts

The CIA draws a distinction between using technology to anticipate events, versus predict them. The agency is using sophisticated algorithms and advanced analytics, along with publicly available data, to forecast events. The initial coverage focuses on the Middle East and Latin America.

“We have, in some instances, been able to improve our forecast to the point of being able to anticipate the development of social unrest and societal instability to within three to five days out,” said Andrew Hallman, the agency’s deputy director for digital innovation.

In its annual report in June, the Defense Science Board said, “Imagine if national leaders had sufficient time to act in emerging regional hot spots to safeguard U.S. interests using interpretation of massive data including social media and rapidly generate strategic options.”

“Such a capability may soon be achievable,” the board said. “Massive data sets are increasingly abundant and could contain predictive clues — especially social media and open-source intelligence.”

Poindexter’s Legacy

If U.S. intelligence agencies develop an advanced system to predict terrorist acts they might call it “Total Information Awareness.” Except that name has already been used, with unhappy results.

Retired Admiral John Poindexter created the “Total Information Awareness” program for the Pentagon’s Defense Advanced Research Projects Agency in 2002 to find and monitor terrorists and other national security threats using data and technology.

The program became so controversial, especially over concerns that privacy rights would be violated, that Congress canceled funding for Poindexter’s office in 2003.

Having been there and done that, Poindexter now says predicting terrorism is possible but would require a lot of data, such as banking information, analysis of social media, travel records and classified material.

The system also has to include strong privacy protections that the public can review, said Poindexter, who said he was working on such a “privacy protection application” when his program was canceled.

“You have to develop public trust in the way this is going to work,” said Poindexter, who continued developing the technology after leaving government through Saffron Technology Inc., a cognitive computing company that Intel bought in 2015 for an undisclosed price. Intel declined to comment.

“The government’s priorities should be to solve the privacy issue and start ingesting massive amounts of data into memory bases,” Poindexter said. “You have to get the public on board with the idea that we can collect and search information on terrorist planning that doesn’t have an adverse impact on innocent people.”


Article Disclaimer: This article was published by Insurance Journal and was retrieved on 10/15/2016 and posted here at INDESEEM for information and educational purposes only. The views, ideas, materials and content of the article remains those of the author. Please cite the original article accordingly.


 

Researchers on Big Data: More than a lot of numbers!

Photo by 123RF.com

By Steve Sonka and Yu-Tien Cheng, University of Illinois November 03, 2015 | 7:01 am EST


Big Data — the current buzzword of choice. Today it’s very easy to be overwhelmed by the hype promoting Big Data. Farm media, newspapers and general media, and conference speakers all extol the future transforming effects of Big Data, stressing that “Big Data will be essential to our future, whatever it is.” The goal of this article, and the series of five that follow, is to begin to unravel that “whatever it is” factor for agriculture.

We’ll definitely explore “whatever it is” from a managerial, not a computer science, perspective. Potential implications for agriculture will be the primary emphasis of the following set of articles:

1.Big Data: More Than a Lot of Numbers! This article emphasizes the role of analytics enabling the integration of various data types to generate insights. It stresses that the “Big” part of Big Data is necessary but it’s the “Data” part of Big Data that’s likely to affect management decisions.

2.Precision Ag: Not the Same as Big Data But… Today, it’s easy to be confused by the two concepts, Precision Ag and Big Data. In addition to briefly reviewing the impact of Precision Ag, this article stresses that Big Data is much more than Precision Ag. However, Precision Ag operations often will generate key elements of the data needed for Big Data applications.

3.Big Data in Farming: Why Matters! Big Data applications generally create predictions based on analysis of what has occurred. Uncertainty in farming, based in biology and weather, means that the science of agriculture (the Why) will need to be integrated within many of the sector’s Big Data applications.

4.Big Data: Alive and Growing in the Food Sector! Big Data already is being extensively employed at the genetics and consumer ends of the food and ag supply chain. This article will stress the potential for capabilities and knowledge generated at these levels to affect new opportunities within production agriculture.

5.A Big Data Revolution: What Would Drive It? Management within farming historically has been constrained by the fundamental reality that the cost of real-time measurement of farming operations exceeded the benefits from doing so. Sensing capabilities (from satellites, to drones, to small-scale weather monitors, to soil moisture and drainage metering) now being implemented will materially lessen that constraint. Doing so will create data streams (or is it floods?) by which Big Data applications can profoundly alter management on the farm.

6.A Big Data Revolution: Who Would Drive It? Over the last 30 years, novel applications of information technology have caused strategic change in many sectors of the economy. This article draws on those experiences to inform our thinking about the potential role of Big Data as a force for change in agriculture.

Big Data: More Than a Lot of Numbers!

Innovation has been critical to increased agricultural productivity and to support of an ever increasing global population. To be effective, however, each innovation had to be understood, adopted, and adapted by farmers and other managers.

Although Big Data is relatively new, it is the focus of intense media speculation today. However, it is important to remember that Big Data won’t have much impact unless it too is understood, adopted and adapted by farmers and other managers. This article provides several perspectives to support that process.

Big Data Defined

“90% of the data in the world today has been created in the last two years alone” (IBM, 2012).

In recent years, statements similar to IBM’s observation and associated predictions of a Big Data revolution have become increasingly more common. Some days it seems like we can’t escape them!

Actually, Big Data and its hype are relatively new. As shown in Figure 1, use of the term, Big Data, was barely noticeable prior to 2011. However, the term’s usage literally exploded in 2012 and 2013, expanding by a factor of 5 in just two years.

With all new concepts, it’s nice to have a definition. Big Data has had more than its fair share. Two that we find helpful are:

•The phrase “big data” refers to large, diverse, complex, longitudinal, and/or distributed data sets generated from instruments, sensors, Internet transactions, email, video, click streams, and/or all other digital sources available today and in the future (National Science Foundation, 2012).

•Big Data is high-volume, -velocity, and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making (Gartner IT Glossary, 2012).

These definitions are impressive. However, they really don’t tell us how Big Data will empower decision makers to create new economic and social value.

From Technology to Value

In the next few paragraphs, we’ll move beyond those definitions to explore how application of Big Data fosters economic growth. In this article, we’ll present non-ag examples because today there is more experience outside of agriculture. The following articles in this series will focus on agriculture.

Big Data generally is referred to as a singular thing. It’s not! In reality, Big Data is a capability. It is the capability to extract information and craft insights where previously it was not possible to do so.

Advances across several technologies are fueling the growing Big Data capability. These include, but are not limited to computation, data storage, communications, and sensing.

These individual technologies are “cool” and exciting. However, sometimes a focus on cool technologies can distract us from what is managerially important.

A commonly used lens when examining Big Data is to focus on its dimensions. Three dimensions (Figure 2) often are employed to describe Big Data: Volume, Velocity, and Variety. These three dimensions focus on the nature of data. However, just having data isn’t sufficient.

Analytics is the hidden, “secret sauce” of Big Data. Analytics refers to the increasingly sophisticated means by which analysts can create useful insights from available data.

Now let’s consider each dimension individually:

Interestingly, the Volume dimension of Big Data is not specifically defined. No single standard value specifies how big a dataset needs to be for it to be considered “Big”.

It’s not like Starbucks; where the Tall cup is 12 ounces and the Grande is 16 ounces. Rather, Big Data refers to datasets whose size exceeds the ability of the typical software used to capture, store, manage, and analyze.

This perspective is intentionally subjective and what is “Big” varies between industries and applications. An example of one firm’s use of Big Data is provided by GE — which now collects 50 million pieces of data from 10 million sensors everyday (Hardy, 2014).

GE installs sensors on turbines to collect information on the “health” of the blades. Typically, one gas turbine can generate 500 gigabytes of data daily. If use of that data can improve energy efficiency by 1%, GE can help customers save a total of $300 billion (Marr, 2014)! The numbers and their economic impact do get “Big” very quickly.

The Velocity dimension refers to the capability to acquire, understand, and respond to events as they occur. Sometimes it’s not enough just to know what’s happened; rather we want to know what’s happening. We’ve all become familiar with real-time traffic information available at our fingertips.

Google Map provides live traffic information by analyzing the speed of phones using the Google Map app on the road (Barth, 2009). Based on the changing traffic status and extensive analysis of factors that affect congestion, Google Map can suggest alternative routes in real-time to ensure a faster and smoother drive.

Variety, as a Big Data dimension, may be the most novel and intriguing. For many of us, our image of data is a spreadsheet filled with numbers meaningfully arranged in rows and columns.

With Big Data, the reality of “what is data” has wildly expanded. The lower row of Figure 3 shows some newer kinds of sensors in the world, from cell phones, to smart watches, and to smart lights.

Cell phones and watches can now monitor users’ health. Even light bulbs can be used to observe movements, which help some retailers to detect consumer behaviors in stores to personalize promotions (Reed, 2015). We even include human eyes in Figure 3, as it would be possible to track your eyes as you read this article.

The power of integrating across diverse types and sources of data is commercially substantial. For example, UPS vehicles are installed with sensors to track the engine performance, car speed, braking, direction, and more (van Rijmenam, 2014).

By analyzing these and other data, UPS is able to not only monitor the car engine and driving behavior but also suggest better routes, leading to substantial savings of fuel (Schlangenstein, 2013).

So, Volume, Variety, and Velocity can give us access to lots of data, generated from diverse sources with minimal lag times. At first glance that sounds attractive. Fairly quickly, however, managers start to wonder, what do I do with all this stuff?

Just acquiring more data isn’t very exciting and won’t improve agriculture. Instead, we need tools that can enable managers to improve decision-making; this is the domain of Analytics.

One tool providing such capabilities was recently unveiled by the giant retailer, Amazon (Bensinger, 2014). This patented tool will enable Amazon managers to undertake what it calls “anticipatory shipping”, a method to start delivering packages even before customers click “buy”.

Amazon intends to box and ship products it expects customers in a specific area will want but haven’t yet ordered. In deciding what to ship, Amazon’s analytical process considers previous orders, product searches, wish lists, shopping-cart contents, returns, and even how long an Internet user’s cursor hovers over an item.

Analytics and its related, more recent term, data science, are key factors by which Big Data capabilities actually can contribute to improved performance, not just in retailing, but also in agriculture. Such tools are currently being developed for the sector, although these efforts typically are at early stages.

So What?

In this discussion, we explored the dimensions of Big Data — 3Vs and an A. The Volume dimension links directly to the “Big” component of Big Data. Variety, Velocity and Analytics relate to the “Data” aspect. While Volume is important, strategic change and managerial challenges will be driven by Variety, Velocity, and especially Analytics.

Unfortunately, media and advertising tend to emphasize Volume; it’s easy to impress with really, really large numbers. But farmers and agricultural managers shouldn’t be distracted by statistics on Volume.

Big Data’s potential doesn’t rest on having lots of numbers or even having the world’s largest spreadsheet. Instead, the ability to integrate across numerous and novel data sources is key.

The point of doing this is to create new managerial insights that enable better decisions. While Volume and Variety are necessary, Analytics is what allows for fusion across data sources and new knowledge to be created.

Emphasizing the critical role of Variety of data sources and Analytics capabilities is particularly important for production agriculture. Individual farms and other agricultural firms aren’t likely to possess the entire range of data sources needed to optimize value creation.

Further, sophisticated and specialized Analytics competencies will be required. To be effective, however, the computer science competencies also need to be combined with knowledge of the business and science aspects of agricultural production.

At times this sounds complicated and maybe threatening. Visiting with a farmer from Ohio about this topic recently, he made a comment that is helpful in unraveling this complexity. He noted that effective use of Big Data for him as a Midwestern farmer is mainly about relationships.

The relevant question is, “Which input and information suppliers and customers can provide the Big Data capabilities for him to optimize his decisions?” And he noted, “For farmers, managing those relationships isn’t new!”


Article Disclaimer: This article was published by Agprofessional.com and was retrieved and posted at INDESEEM for information and educational purposes only. The views, opinions, thoughts, and information expressed in this article are those of the authors. Please cite the original and INDESEEM accordingly.


 

 

Clemson researchers and IT scientists team up to tackle Big Data

Media Release


Jim Melvin, Public Service Activities October 29, 2015


 

Alex Feltus is an associate professor in genetics and biochemistry at Clemson.
Alex Feltus is an associate professor in genetics and biochemistry at Clemson. Image Credit: Jim Melvin / Clemson University

CLEMSON — While researchers at Clemson University have recently announced an array of breakthroughs in agricultural and life sciences, the size of the data sets they are now using to facilitate these achievements is like a mountain compared to a molehill in regard to what was available just a few years ago.

But as the amount of “Big Data” being generated and shared throughout the scientific community continues to grow exponentially, new issues have arisen. Where should all this data be stored and shared in a cost-effective manner? How can it be most efficiently transferred across advanced data networks? How will researchers be interacting with the data and global computing infrastructure?

A team of trail-blazing scientists and information technologists at Clemson is working hard to answer these questions by studying ways to simplify collaboration and improve efficiency.

“I use genomic data sets to find gene interactions in various crop species,” said Alex Feltus, an associate professor in genetics and biochemistry at Clemson. “My goal is to advance crop development cycles to make crops grow fast enough to meet demand in the face of new economic realities imposed by climate change. In the process of doing this, I’ve also become a Big Data scientist who has to transfer data across networks and process it very quickly using supercomputers like the Palmetto Cluster at Clemson. And I recently found myself — especially in just the past couple of years — bumping up against some pretty serious bottlenecks that have slowed down my ability to do my best possible work.”

Big Data, defined as data sets too large and complex for traditional computers to handle, is being mined in new and innovative ways to computationally analyze patterns, trends and associations within the field of genomics and a wide range of other disciplines. But significant delays in Big Data transfer can cause scientists to give up on a project before they even start.

“There are many available technologies in place today that can solve the Big Data transfer problem,” said Kuang-Ching “KC” Wang, associate professor in electrical and computer engineering and also networking chief technology officer at Clemson. “It’s an exciting time for genomics researchers to vastly transform their workflows by leveraging advanced networking and computing technologies. But to get all these technologies working together in the right way requires complex engineering. And that’s why we are encouraging genomics researchers to collaborate with their local IT resources, which include IT engineers and computer scientists. This kind of cross-discipline collaboration is reflecting the national research trends.”

In their recently published paper titled “The Widening Gulf between Genomics Data Generation and Consumption: A Practical Guide to Big Data Transfer Technology,” Feltus, Wang and six other co-authors at Clemson, the University of Utah and the National Center for Biotechnology Information discussed the careful planning and engineering required to move and manage Big Data at the speeds needed for high-throughput science. If properly executed, sophisticated data transfer networks, such as Internet2’s Advanced Layer2 Service, as well as the inclusion of advanced applications and software, can improve transfer efficiency by orders of magnitude.

“Universities and other research organizations can spend a lot of money building supercomputers and really fast networks,” Feltus said. “But with research computing systems, there’s a gulf between the ‘technology people’ and the ‘research people.’ We’re trying to bring these two groups of experts together and learn to speak a common dialect. The goal of our paper is to expose some of this information technology to the research scientists so that they can better see the big picture.”

It won’t be long before the information being generated by high-throughput DNA sequencing will soon be measured in exabytes, which is equal to one quintillion bytes or one billion gigabytes. A byte is the unit computers use to represent a letter, number or symbol.

Kuang-Ching “KC” Wang is an associate professor in electrical and computer engineering and also networking chief technology officer at Clemson.
Kuang-Ching “KC” Wang is an associate professor in electrical and computer engineering and also networking chief technology officer at Clemson. Image Credit: Jim Melvin / Clemson University

In simpler terms, that’s a mountain of information so immense it makes Everest look like a molehill.

“The technology landscape is really changing now,” Wang said. “New technologies are coming up so fast, even IT experts are struggling to keep up. So to make these new and ever-evolving resources available quickly to a wider range of different communities, IT staffs are more and more working directly with domain science researchers as opposed to remaining in the background waiting to be called upon when needed. Meanwhile, scientists are finding that the IT staffs that are the most open-minded and willing to brainstorm are becoming an invaluable part of the research process.”

The National Science Foundation and other high-profile organizations have made Big Data a high priority and they are encouraging scientists to explore the issues surrounding it in depth. In August 2014, Feltus, Wang and five cohorts received a $1.485 million NSF grant to advance research on next-generation data analysis and sharing. Also in August 2014, Feltus and Walt Ligon at Clemson received a $300,000 NSF grant with Louisiana State and Indiana universities to study collaborative research for computational science. And in September 2012, Wang and James Bottum of Clemson received a $991,000 NSF grant to roll out a high-speed, next-generation campus network to advance cyberinfrastructure.

“NSF is increasingly showing support for these kinds of research collaborations for many of the different problem domains,” Wang said. “The sponsoring organizations are saying that we should really combine technology people and domain research people and that’s what we’re doing here at Clemson.”

Feltus, for one, is sold on the concept. He says that working with participants in Wang’s CC-NIE grant has already uncovered a slew of new research opportunities.

“During my career, I’ve been studying a handful of organisms,” Feltus said. “But because I now have much better access to the data, I’m finding ways to study a lot more of them. I see fantastic opportunities opening up before my eyes. When you are able to give scientists tools that they’ve never had before, it will inevitably lead to discoveries that will change the world in ways that were once unimaginable.”


This material is based upon work supported by the National Science Foundation (NSF) under Grant Nos. 1443040, 1447771 and 1245936. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF.


Article Disclaimer: This article was published by Clemson University and was retrieved on 10/29/2015 and posted here at INDESEEM for information and educational purposes only. The views, thoughts, research findings, and information contain in the article remains those of the authors. Please cite the original and this source accordingly.