Do I Need a Database?

What is a database?

A database is a scalable archive that serves as a way to organize large data sets, often combined from various sources. A wide range of database solutions are available – from expensive, on-premise Oracle-based systems, to lightweight, nearly-free open-source solutions available through Amazon Web Services or Microsoft Azure.

What problems does a database solve?

Let's look at three primary situations that you might encounter that would indicate it may be time to move your data infrastructure to a database.

Your data are too big

The first and easiest situation to identify is where your data are simply too big to analyze through a conventional spreadsheet tool. You may have encountered a scenario where you go to download data into Excel and frustratingly "break" it because the tens/hundreds of thousands of rows you want to look at max out your machine's ability to process the data. This can make it nearly impossible to do meaningful analysis or generate reports.

Databases are optimized to store and process very large data sets, and because users can access subsets of raw data via queries from a database tool, databases remove the need to download entire data sets in order to run analyses.

Your data are too complex

Databases solve a key issue related to data sourced from multiple places. In your organization, you likely report on data from different systems – a CMS, a CRM, accounting tools, social media accounts, point-of-sale systems (the list goes on!). A well-designed database solution provides standardization that enables you to merge and report on data from various systems fluidly.

You have more extensive reporting needs

In addition to the size and complexity of your data sets, your reporting requirements can also be a primary factor when deciding to migrate to a database environment.

Frequently reporting the same data

Popular reporting tools (e.g. Qlik, Tableau, etc.) that can connect directly to database tables make reporting much easier because underlying data can be automatically refreshed and updated in the database. By automating the data refresh and report calculations, you minimize the risk of duplicates and errors, freeing up time for your team members to focus on other priorities.

Distributing reports or data to larger audiences

Databases are centralized data stores that can be accessed securely from any computer within your organization (with the proper permissions). This enables teams to more autonomously access and report on the performance of their respective areas.

In addition, reporting tools often offer server versions of their products that complement their software by allowing users to publish reports to a central location. This means that instead of creating a static PDF file or PowerPoint deck and emailing reports, you can simply share a URL with an interactive version of your report that users can easily access. Note that there can be additional costs associated with enabling these features, so check with your software provider before implementing this solution to understand the full costs associated.

To help you determine if your reporting needs require a database, you can look at this case study with one of Project Evident's clients, First Place For Youth, that covers their journey away from old reporting systems into an automated environment in AWS.