A New Year's wish

Big data needs a product like Microsoft Access

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

The trend toward self-service in business analytics has been good for the big data industry. But in order for the user-oriented paradigm to take deep root, the industry needs to change the way it is approaching it. The information worker and data analytics worlds need a big data product akin to Microsoft Access.

Access itself was never especially safe or appropriate for production database applications. But business users were able to use it to stretch their imaginations. By having a tool that could build working database application prototypes, users were able to take ownership of what databases could do and how they could be used. Access allowed business users to experiment with databases and implement something in a relatively short period of time. Access provided everything necessary to create a database system that could almost work in a mission-critical capacity.

While lauding a tool that facilitates incomplete success may seem absurd, such a tool is essential to getting successful systems built. Access allowed users to actualize the systems that they wanted, and those systems that passed subsequent peer review created user demand and demonstrated efficacy. This situation was no cul-de-sac, as many such systems were eventually re-implemented by specialists using more professional tools. Without Access, arguably, those systems would not have been implemented at all.

A user tool helps production tools

Access heralded the beginning of self-service data management and, perhaps ironically, it gave rise to widespread adoption of client/server databases in the enterprise. In order for Hadoop and other big data analytics technologies to see the same sort of adoption, we need a tool like Access that can serve as a catalyst, allowing business users to model concretely the kinds of big data systems that they need.

Such a product, call it “Big Access”, would connect to cloud data sources, spreadsheets, enterprise data sources, log files, and perhaps certain machine data beyond those log files. Big Access would also provide functionality for data quality, data blending, and data shaping. It would provide basic data visualization capabilities, though it would leave the fancy stuff to tools that already cover the visualization space.

Big Access would also provide predictive analytics functionality. The amount of explicit effort required to build a predictive model on existing data in Big Access would actually be quite small. Big Access would build such models transparently, in the background, such that it could offer the ability for the business user to run predictive queries on whim.

Beyond bits and pieces

We have tools that fulfill some of these capabilities already. But current products are task-driven; they have a specific purpose and are used explicitly for that purpose. Conversely, Big Access would provide functionality that business users don’t necessarily realize they need. Big Access would determine from context which analytics capabilities were required and would be most useful. It would then make those capabilities available to the user, without burdening the user with a manifest of what the necessary underlying technologies were.

Big Access could run on top of Hadoop. Big Access could run on top of Apache Spark. It could also run on top of Spark Streaming and Spark’s MLLib and even on top of Spark SQL or Hive or Pig. You get the idea. Big Access wouldn’t provide innovative big data technology. It would provide innovation in the usability of existing big data technology.

Developers need it too

Big Access would be programmable. In Java. In Python. In C#. In JavaScript. No programming would be required but custom code would be accommodated. Big Access would be query-able using SQL and could be integrated into mainstream programming environments as if it were a relational database. In fact, a Big Access database, developed by a business user, and deployed to a company server, could immediately be integrated into a line-of-business application by any enterprise developer.

Of course, the same developers could integrate their applications with Hadoop today almost as easily, but many developers don’t realize this. A simple desktop tool that deployed the database to a company server would in fact be more approachable to many developers than would a Hadoop cluster. After the Big Access database was migrated to full-fledged Hadoop, the application could be migrated to Hadoop as well. In this way, Big Access would provide an on-ramp to big data technology for business users and enterprise developers alike.

Enable individuals, win the enterprise

When users can work with products in relative privacy, a greater intimacy between those users and the products can emerge. For example, this is why so much data work gets done in Excel even when, technologically, it is not always the best tool for the job. This is also why people use search engines. And, in fact, this is why so many users have worked with Microsoft Access itself.

Big Access would provide a bridge to users. Some, including entrepreneurs and technologists, may view that as mere fit and finish. But the absence of a tool like Big Access is holding back broader success for big data technology and, ultimately, for those same entrepreneurs and technologists.

If we want data and analytics to be as essential to information workers as documents, spreadsheets, presentations, email, and search are today, then we need big data tools to be as ubiquitous, approachable, and commonplace as search engines and office suite applications. We are not there yet. We need to be there. Perhaps 2015 will be the year.

19 Responses to “Big data needs a product like Microsoft Access”

  1. Louis Nauges

    This is a great post, covering a real need in most organizations.

    I think there is already a product in the market which respond to your wishes: its name is BIME ( http://www.bimeanalytics.com).
    This a very powerful and easy to use full SaaS BI solution, launched in 2009 by a French Start-up which has now offices in the US.

    BIME works on top of most “On premise” solutions but is an amazing tool to “access” Big Data platform like BigQuery by Google or AWS RedShift.

    My experiences, with many large organisations using BIME, show that this tool is easier to lear than Access and can be managed by Business Analysts in a few days.

  2. Terrific article. Great start to a new work year. People (sometimes clients, more often associate developers) invariably ask us at some point “Why are you still using Access for anything?” Now I can forward a link to this discussion. In a way, it illustrates the way people become obsessed with using “the latestest technology” in every situation – even when it comes with unnecessary, time-consuming complexity. Looking forward to seeing how this unfolds. Big Access – now there’s something to dream about!

  3. The article makes an excellent point. Many business intelligence, statistical languages and predictive analytics tools require an extensive learning curve or strong knowledge of statistical concepts. Designing software for a self service approach is a different problem that developing tools for a knowledgeable data scientists. Having used and marketed BI tools, my experience suggests that few current tools have targeted the regular manager, which is an important issue in a world with a shortage of data scientists. But such tools do exist. Angoss KnowledgeSeeker is a a relatively user friendly automated analytical tool. BeyondCORE is a particular friendly end user oriented tool that not only exploits the parallelism possible in cloud processing, but also guides users through the review of the automated analysis.

    • goranrice

      Fully agree with your comment.

      However, from what I understood, he was not talking about adoption in his article but rather about how there is a technological gap for Big-Data DW to have a solution like Msft Access to export and consume the data by whatever service that would consume it. BI or app’s, etc…

      Btw, I invite you to look into http://www.simpleql.com and give us your feedback. We are dealing with the adoption that you mentioned in your post from a different aspect. I’m the head of product and would love to hear your thoughts.

  4. donaldswilde

    I’m also in the business of writing analytics s/w for embedded devices talking to the Cloud, and I’ve been quite happy with RapidMiner community edition (v5 == free) which includes a MySQL adapter and has an enterprise growth path for many more. JMP is also good, but JMP is a knowledgeable data scientist’s tool where RM is actually a neophyte’s tool. It integrates with R, can output CSV, and comes with a raft of free plugins for various kinds of data mining queries. Sure, it’s not a Hadoop front end, but it is a very useful tool indeed.

  5. @Stewart states things nicely. I’ve worked with true client/server databases for 20 years. I’m not saying Access systems are or were good in production. I’ve spent lots of time rebuilding those apps with true enterprise tools. But Access was a portal to the opportunities, and let end-users understand what they needed to in order to appreciate the value of them.

  6. David D

    Wow. Nailed it. I’m in the business of building software, both operational and analytical, for unsophisticated users (corporate tax departments). They need all detailed purchase/sale transactions worldwide, plus all financials – lots of data. But they also need a simple experience that hides complexity. It has to meets them where they are, which is excel for all, access for the very sophisticated. This type of power prototype environment would allow users to mock up what they want and pass it off to development for finish. Home run if it existed.

  7. Stewart Robbins

    Guys, I think the point of the article is that MS Access enhanced demand for BI and analytics rather than developing good databases – and it was cheaply supplied by a ‘safe’ vendor known to (and generally trusted by) non-analysts. Whether people like it or not, MS Access has been integrated into LoB applications. They are a devil to ferret out and replace but we generally get the chance because the business case is clear. It is much harder to do things ‘properly’ when someone isn’t familiar with and dependent on the value they get from an existing tool… To get people using next gen data, the article argues we need next gen Access (including text mining, anyone?). I’m pretty sympathetic to that argument…

  8. @Jordan, @Mike, @wojciechgryc and @22nd_century: I have worked with Power Pivot since it was in beta and known as Project Gemini. I think it’s a great tool (as is Tableau, @Ralph). But it *is* a tool, and not something that builds line-of-business applications that integrate analytics. Plus, it’s more of a power-user tool than a business user tool. Same goes for the rest of what is now the Power BI suite, including Power View and Power Query. They are very good products as far as they go, but they don’t go as far as what I’m calling for here.

  9. gengstrand

    Within the spirit of this article, I believe that big data already has a product like Access. See http://glennengstrand.info/analytics/oss for examples (both proprietary and open source) of feeding OLAP spreadsheets with big data. For easy programmatic access to big data see http://glennengstrand.info/analytics/fp

    Both of these trends are actually quite popular these days (though perhaps not as mainstream as SMB using MS-Access). There are also plenty of DaaS vendors bringing big data capability to SMB with limited technical resources.

    Though no quite an example of big data, take a look at http://it.toolbox.com/blogs/future-of-work/learning-data-mining-with-knime-65104 as an example of making data mining accessible for the rest of us.

  10. wojciechgryc

    I feel the same way about MS Excel. Extremely widespread analytics tool and people even write code in the form of equations or macros. Business users that would otherwise say they are “coding luddites” end up writing fairly sophisticated applications.

  11. Access is the bane of real database developers this is a horrible idea. Whenever this toy database is compared to real databases its from someone who doesn’t understand either…

  12. mike locke

    What is needed is not an a Microsoft Access equivalent, but a Microsoft Excel equivalent where one does not need a computer science degree and the average worker can use with a little training