In the most formal and technical sense, Big Data refers to exceptionally large databases created by the digital revolution. As computers and the Internet became a core tool of virtually every organizational activity, they have generated volumes of raw data. Over time, it became clear that if the vast amount of data produced could be tapped and organized, it would have tremendous commercial and organizational applications. Initially, only very large organizations with substantial resources and sophisticated computer capabilities could afford and/or had the expertise to do it. But now enough has been compiled that access to Big Data’s advantages are rapidly becoming more mainstream.
Of course, large databases are not entirely new. The government has been creating databases in some form for many years; the Census Bureau compiled and made available – at taxpayer expense — large amounts of data for a long time. As computer technology became more sophisticated, however, it became clear that it is within the grasp of the private sector to gain entry into this field. Key early efforts have been in marketing where the commercial value of information about markets and ways to reach and engage customers in an increasingly targeted way is very valuable. Amazon is a classic example of how Big Data can be used to build new business models.
In short, the development of information technology paved the way for the information age, and Big Data is a natural outcome of that broad cultural trend. It has been a boon to big business as it has tapped into Big Data to enhance everything from product development to marketing.
Not An End In Itself
So, what does this mean for an association in today’s fast–paced world? How does a medium–sized or small non–profit organization that does not have the resources or expertise to mine Big Data or access expensive databases benefit?
Keep in mind that Big Data is not an end in itself. It is only valuable as a tool to enhance decision making and operational effectiveness. And it is a tool that can be applied – at least to some degree – to many of the traditional processes of associations. Here’s why:
- Access to Big Data has become increasingly available and affordable as more and more of it is compiled and competition among data providers drives down price.
- The pure notion of Big Data has morphed into a broader concept of accessing, compiling and analyzing data that is not necessarily as “big” as the original concept suggests, making it more feasible for smaller organizations to access.
- The pioneering work of the early Big Data compilers generated a broad level of expertise and new analytic techniques that can be applied to smaller, more accessible data sets (including internally generated data) which are often as useful as gargantuan data sets.
While there are many ways to apply Big Data insights, the following, some of which can be completed in-house with small data sources, are top on my list. Often, external expertise is required to translate data into information for an organization’s particular needs.
Assessing and Enhancing the Business Model
The business model is one of the most important and most misunderstood components of organizations. Descriptions of the business model are often not much more than an inventory of revenue sources without much analytics underpinning that explains how each revenue source relates to the market(s) that generate it or how the various revenue sources combine to create a total profile of the organization’s revenue base as it relates to mission and core functions.
The increased availability of rich marketing and internal performance information, together with more sophisticated data analysis techniques sparked from the Big Data movement, greatly enhances the basis for an organization to construct a much deeper understanding of its business model. It enables an organization to describe what it thinks its business model is, but also to explore what it is in reality (not always the same thing) as well as to identify ways to enhance it. This starts with more intense analysis of internal data from sales and member/customer feedback (i.e. small data) but also includes the intelligence from more sophisticated external data on broader market characteristics and performance (much of it from external data sources) as a way to identify opportunities to clarify and improve the business model.
As associations look for more ways to maintain minimal membership dues levels, the concept of revenue diversification has become increasingly important.
This is, of course, part of the broader business model but it warrants separate consideration. The creative process of identifying new sources of revenue has generally begun from looking at today’s revenue streams and seeking spin–offs, or looking to non–revenue producing activities and assessing their revenue potential. The marketing information the Big Data movement generates may provide insights into potential new markets and examples of potential revenue sources that associations have not considered. For example, new insights regarding member profiles and how they compare with other similar populations (i.e. from enriched data analysis) could help identify commercial services that could meet members’ needs while simultaneously generating royalty income. Or, better information available about government or foundation grants could identify mission–compatible projects unknown to the organization.
Membership data is tapped by most organizations for obvious reasons: identifying which segments are trending up (and down) and assessing member satisfaction across a range of areas. However, there may be additional or non traditional analysis that can be conducted using existing membership data combined with data on industry and broader social trends. How does the organization’s current membership trends correlate with population preference trends in the broader association environment and beyond? What does that indicate about the direction of future membership? One particularly fertile area for inquiry relates to how associations communicate with members. Increased availability of data on how members and prospects use social media as their preferred source of information and mode of communication can be useful in designing more efficient/effective ways of reaching and engaging members.
Generating typical financial statements (balance sheet, income and cash flow statements) is often the extent of an organization’s financial analysis. However, how much more can the financial data indicate? To what extent can revenue and expense data be displayed and analyzed differently by functional categories or other units of analysis to uncover deeper insights on activity patterns that have yet to be identified? To what extent is data available to provide a basis for comparing an organization to similar organizations that have more robust financial profiles?
Staff often gather and analyze current sales and customer satisfaction data to identify trends. While this internally generated data can be very useful, the Big Data movement has been especially active in the marketing arena and has generated enhanced analysis techniques as well as access to expanded marketing databases. Tapping into these resources can substantially enhance in–house data and market analysis capabilities. However, significant work already in this area provides insights into what is available and what is involved to access it. In–house exploration is a good place to start, but outside expertise on what is available may help accelerate the potential.
Performance Assessment and Dashboards
We’ve all heard “what gets measured gets done.” Most associations engage in some form of performance assessment to gauge progress against plans and many use an easy-to-read dashboard to display the information. While many ways exist to measure performance, it’s difficult to quantify meaningful performance targets and then collect accurate data related to the targets. Often, process or input measures become substitutes for real output measures because of difficulties related to measurement. This is a continuing challenge, but the Big Data movement provides some new tools to improve this process. Data and analysis of performance over a large number of organizations provides new insights into what measures correlate highly wit real impact over time.
Effective and efficient communication continues to be a challenge for associations. How can the organization best communicate with members and other stakeholder groups? How can your messages dominate your target audience’s other information sources? Analyses of large databases have added insights into the effectiveness of certain types of communications and messaging. Additionally, richer data is available to help identify target audiences more precisely, focusing messages. Social media feedback is an important source of such new data.
An area often overlooked as a data–rich opportunity is public affairs. Traditionally, much of the analysis of public affairs initiatives focused on polls and basic historical vote counting. The advent of the Big Data era has added new levels of insight into patterns of behavior in political and legislative arenas. Organizations involved in advocacy and lobbying should look closely at data and analysis techniques to focus their messaging, advocacy and lobbying.
Other In–House Data
Financial, membership, and marketing data are obvious sources of in–house data, but there are others. For larger organizations with substantial staff, the human resources database can provide interesting insights on productivity and can be enhanced through the use of external HR databases. Other areas might include analysis of the use of data processing resources and the efficiency of governance processes, especially in organizations with large decision-making groups such as a house of delegates or board of governors.
Tapping Into Big Data
While starting with “small data” can be instructive, the benefits of using data more intensively and non–traditionally may require tapping into external data sources, many of which have been generated from the Big Data movement. But starting to look outside for ways to capitalize on Big Data can be daunting: there’s an abundance of data and expensive tools to find it. Very large organizations with massive IT capabilities are the real pioneers and major players in Big Data. They speak about terabytes and petabytes of data, clearly beyond the reach of most non–profit organizations. However, with a systematic approach, the pioneering work of the big players can benefit smaller organizations.
Start with the End in Mind
Knowing where to start tapping into Big Data requires a targeted goal or goal set. An organization should ask: what are the questions that would be most instructive in planning strategic direction, improving specific program areas, or enhancing the organization’s overall presence and identity?
Start looking for insights by exploring how current internal data can be analyzed differently or enhanced through additional data generation within the organization’s current scope. Begin to develop hypotheses or tentative responses to the questions that have been targeted. Be open to subsequent questions. Often, the second or third level questions can generate useful insights.
Identify External Data Sources
Explore availability of external data sources that can supplement in–house data. No central source exists to curate total data types and available data sets, so Google search or using another search engine for data by topic is probably the best approach, following the search sequence outlined below:
Public Data Sources
Searching the availability of data from government (federal, state, county and local) sources is likely the best approach to finding access to free or low cost data. Also, consider international quasi–governmental sources, such as the U.N.
Free Non–Governmental Data Sources
Searching free Big Data yields a surprising number of entries. Many include public (mainly government) sources, but there are many non–governmental sources, too. Some are university based- (both public and private) and there are many from commercial sources. There is no standard approach to sifting through sources other than by the usual keyword approaches. This is why “starting with the end in mind” is critical. While there may be some really useful data available that can lead to valuable insights, most organizations cannot devote the human resources required to sift through all possibilities. Experience can be a great tool to narrow the search to sources that are proven to be useful, but gaining that experience can be a labor-intensive and expensive process initially.
One approach to narrowing a data search is to search the literature on a topic of interest and then look to the bibliographies of helpful articles for information on data sets the authors used. This leverages the initial hard work of others to good advantage. In many cases, university graduate students have completed that work.
Another approach to identifying data sources from among many available is to use external expertise, such as consultants.
Begin Analysis Early
Extracting value from data is an iterative process. Avoid the all too common mistake of waiting until a large volume of data has been gathered before beginning the analysis. Analyzing early using preliminary data often provides good insights into which data are useful, as well as if the questions initially posed in the “start with the end in mind” step were the right questions. Data analysis can be relatively straightforward or quite complex, involving sophisticated statistical modeling, etc. Knowing up front what level of analysis will be required to generate insights is important. This is another area where external expertise may be needed. External input can extract insights that might otherwise go undiscovered and can be helpful in identifying even simple patterns in data that insiders are too close to the subject to spot.
The technology revolution has generated analytics capabilities not even dreamed of 40 years ago. Big Data, the ability to compile enormous data sets that offer new insights into how markets, society, and governments work, is one phenomenon that remains in an early evolutionary stage but which already offers great potential benefit. The field has progressed so that smaller players who are not positioned to be generators of Big Data can nevertheless be productive consumers if they know how. Big Data has generated a new field of expertise beyond data generation and analysis.New approaches to data analysis have emerged that focus on knowing what questions to ask, where to go in the Big (and small) Data environment to access data, and how to analyze it to extract value for organizations seeking new insights and competitive advantage. This field will continue to develop.