Pulling together some thoughts from this morning's Big Data summit, hosted by Aster Data at the St. Regis in Washington, D.C...
In previous posts on Big Data and deep analytics, I've touched on my interest in this space. Simply put, with the explosion of data ingestion in most companies (both in terms of volume and complexity), sales and marketing executives need a better and faster way to identify complex customer trends, purchasing patterns and market segments.  This morning's presentations -- jump started by Curt Monash, the author of the very informative DBMS2 blog -- focused on what companies are trying to do to enable deep analytics.  While the brand recognition of BI heavyweights MicroStrategy and SAS's trumps that of Aster's, the Silicon Valley-based company's CTO + co-founder, Tasso Argyos, really rekindled my appetite for "agile data management"-focused ideas and discussions.

Case-in-point: Tasso shared four common use cases for Aster's software, optimized for advanced analytics running on commodity hardware:

1. Forecasting;
2. Modeling;
3. Customer segmentation; and
4. Clickstream analysis

Over the last few weeks, I've had my hands full with a number of spectrum-related opportunities at Computech.  Be it interest from the independent regulator and competition authority for the UK communications industries (Ofcom) to a new supply agreement to provide spectrum auction services to Industry Canada, an RFP from Ireland's Commission for Communications Regulation to a handful of our auction team working in Mexico City with COFETEL, it seems like my days have been filled to capacity with talk of managing, allocating and regulating this scarce, valuable resource that cannot be seen or felt.

Spectrum Holdings in the 700MHz band range (c/o the FCC's Spectrum Dashboard)

Heeding the advice of my good friend Tien Wong, I've focused more of my personal time on better understanding telecommunication-related issues that are impacted by such governmental interests.  Tien, the CEO of the VC firm Opus8, suggested that my last five years running business development at our firm, coupled with Computech's 16 years of IT-related support at the FCC, affords me a rather interesting perspective on how technology + the telecommunications "industry"intersect (*by which I mean regulators & industry alike).  As my recent posts show, my technical interests trend towards deep data analytics; conceptually, this matches up nicely with our firm's expertise with spectrum management.  Indeed, the balancing act that so many countries' regulatory bodies need to strike continues to play out on the global stage (and yes, we're happy to play a supporting role).  Just as with agile data management, the key for most telecommunication authorities centers around real-time knowledge of spectrum demand, use and oversight.

So, over the next few weeks, I'm going to shift my writing focus from Big Data, data-driven dashboards and data analytics to the technical implications of managing a spectrum lifecycle.  Being that I'm heading up to Ottawa on Tuesday for their annual 20/20 Spectrum conference -- and to Brussels in June for the annual European Spectrum Management conference -- what better time (or place) to start sharing my observations than now on DCSpring21.  While I can't promise specific topics, my travel schedule does provide some general direction.  Due up this week: my observations from Canada based on perspectives shared by senior spectrum, radio and telecommunications officials from around the world.

Plenty of businesses invest money and resources trying to integrate, organize and manage data. Yet too often, it seems they become overwhelmed by it.  Gartner Research says it best: such "efforts result in fragmented views, siloed information and missed opportunities. This incoherent vision can ultimately damage competitiveness and increase IT and business costs. In today’s economic climate is that a risk worth taking?"  So in the spirit of yesterday's post on Big Data, a "simple" rendering of a data management technology stack (at least as we think about it @Computech):

While some might find this diagram abstract, the goals of agile data management are becoming clearer to me:

• Make data in an organization's databases available in an understandable + explorable format;
• Provide easy access & navigation from one subject area to another (e.g. combine data from one application with another application's data); and
• Ensure data flows within an organization become transparent -- regardless of which operational system generates/maintains the data.

Periodically, I look back on what I've written here -- and also on Computech's Concomitantly.  Last night, as I read Full 360's blog on the Agile Data Warehouse Story, I realized I'd jumped headfirst into the agile data management space without really defining an important concept: Big Data. Used to describe the massive quantities of raw and processed data that are being generated today in applications such as retail operations, let me add some color to the Big Data concept by pasting a sentence from the afore mentioned blog:

"Think of all the McBurgers sold every minute of every the day in a dozen different combinations. That purchasing information is valuable - but only if it can be made manageable, and accessible."

True, it’s become relatively easy (and cheap) to collect data. However, organizations of all sizes continue to wrestle with a fundamental challenge: making sense of what they’ve collected given the speed with which new types of data are generated. So as business leaders begin to execute new strategies based on better analytics and information, organizations with individualized, distributed computing and storage environments may soon require a more flexible, agile, and service- oriented solution to improve their proficiency in handling Big Data.  Make sense?  Drop me a line + let me know your thoughts.

After posting my last entry, I had a spirited (and educational) back-and-forth with a colleague here at Computech.  Our resident data expert, Hari Donthi warned me that when many think about business intelligence, most think about buying BusinessObjects or implementing a bunch of reports.  So, let me try and clarify my interest in Big Data + deep/exploratory/quantitative analysis.

Within our IT firm, I've heard us talk about ways to minimize the time between when data is generated and the time when data is available for analytics.  Questions like "how do we do this as data volumes get big, and the sources of data generation become disparate and uncoordinated" are the rule, not the exception.  A few others further piqued my curiosity:

• Why is it so difficult to just get all the data you can possibly get your hands on and make it available for Analysts immediately?

• Why is reducing "analytic latency" important, especially to government agencies?

• What exactly is "Analytics?"  Do the terms "Statistical Inference" and "Data Mining" completely and accurately describe this field?

Three seemingly innocent words have taken my imagination captive: agile, data and management.  Thanks to the patience of a few colleagues at Computech, I've started down a road of Big Data, business intelligence and analytics.  (*Large, complex data sets, regression analyses, quant forecasts, optimization models, simulations... I must be crazy.)

At the proverbial risk of crashing through an open door, technology makes it possible to collect huge amounts of data.  What's cool about where we are today relates to new solutions and innovative trends in managing + analyzing this influx.  Sure, it's become relatively easy to collect data. However, organizations of all sizes continue to wrestle with a fundamental challenge: making sense of what they've collected given the speed with which new types of data are generated (and the sheer volume of that data).

While most of my family members happily wear the "quant jock" label, I've historically been on the outside looking in.  Not so any more.  While many peers complain of swimming in a vast sea of data, I see this expansion of information as a source of competitive advantage.  The promise of agile data management -- a topic I plan on writing more about on DCSpring21 -- goes to this point.  In fact, when I hear people talking about companies analyzing data and making decisions, it takes a bit of patience not to yell out: companies don't really do this -- people do.  After all, can a company access, change and manipulate data in new ways?

O.K., I'm off my soapbox... for now.