Your checklist for selecting the perfect Business Intelligence platform for your Business

After years of working with these platforms and deploying them in different companies, we thought it would be helpful for anyone looking to decide on a specific BI software to have a checklist or a set of aspects to keep an eye on when making an evaluation. Some of these items may not be essential for your business, but you'll find others to be a turning point when deciding on one tool over the other.

We have started evaluating each of the platforms we have worked with using these criteria. We found it very useful for comparing different options and making justified recommendations to our clients. Specific platform evaluations are coming soon!!

Before starting with the evaluation, we recommend you to talk with your team and other stakeholders to answer the following questions.

• What are your expectations for the BI platform? What problems do you need this to solve?
• Do you have budget constraints? Do you have time constraints regarding how soon the tool needs to be up and running?
• Who will be the audience?
• Are there people with specific technical knowledge available to help? BI analytics, Data Engineers, Programmers, etc.? Or do you expect it to be handled mainly by non-technical users?
• How much data do you have? Is it considered big data (some tables with more than one terabyte of data) or smaller than that?

Now, let’s move to our Criteria checklist. We will list each evaluation area and briefly describe what it means to us.

We strongly believe that this list will change over time, and we will try to keep it updated as much as possible. As BI platforms evolve, some of these items may become available in all of them, making no point in evaluating them anymore. In the same way, as new features develop on some tools, they will become a new comparison/evaluation criteria to keep in mind.

They are all pretty similar, which one should I choose?

1. Data Connectors

Your business data resides in the best-case scenario in a cloud data warehouse, in some production databases, or data lakes. There are several well-known database solutions in the market you may be using right now, such as Amazon Redshift, Snowflake, BigQuery, Mysql, PostgreSQL, Oracle, Mongo DB, Maria DB, etc. Also, you may be, God forbid, using files like excel, google sheets, or plain text as csv/txt files to track your data.

We need to consider what types of data sources the tool gives support to. Can you connect all your databases to the tool? Can you connect your plain text or sheets files? Does it require IT support to create the connections?

This criterion is one of the most critical points to take into account. You need to list all your databases and data origins and see if they are supported by the tool.

2. Data Modeling Capabilities

Data modeling is generally one of the first steps you take after connecting your data to the tool. It usually consists of: selecting tables and columns you need, renaming tables and columns to give them a better understanding, choosing the right types of data and proper format for your columns, create calculated fields, defining dimensions and metrics, joining different sources together to create a more useful and understandable model based on the source tables and files.

A good BI tool will cover most of those basic requirements, and some others will give you even more capabilities that will allow you to prepare your data before you can start analyzing it.

This is an important feature to consider if you don’t count with a data-dedicated team or your data sources are not modeled.

3. SQL vs Drag and Drop

Modern BI tools (especially those oriented to small-mid business) can interact with editor users without writing a single line of SQL or any other specific technical language. You can query your data and build your charts and dashboards through graphical user interfaces such as drag and drop functionality. If you don’t have any technical background or a dedicated data team, you need to focus on these kinds of tools.

Of course, you’ll be missing the great benefits of your data warehouse or database engines. A usual approach is to have some initial technical setup using SQL, for instance, where a road can be paved with clean and understandable datasets and models. Then you can easily start building your analyses using the visuals UIs.

Learning the basics of SQL is highly recommended as this is the primary language your databases will understand, and you can achieve things difficult, if not impossible, otherwise. SQL knowledge will also give some understanding of how database objects are related to each other, and this way you will be more productive creating using the UI features.

4. Models Reusability

BI tools usually support models’ reusability throughout means of dataset definitions or something similar to that. This is the ability to create a model (possible joining and grouping data), establishing a universal point of definition of metrics and dimensions that can be consumed by any who have been granted access.

Some BI tools don’t provide model reusability, forcing you to set your definitions in all the required places. Thus, you will need to replicate the models several times, and changes propagations can be a real pain and time-consuming.

Model reusability will also accelerate all data analysis and dashboard creation. If you can not model your data at the tool level, you may need to make it at your database level.

5. Merging Data Sources

Suppose you don’t have a dedicated data warehouse holding all your data coming from different origins and databases. In that case, the possibility of merging different data sources to create views or calculations can be a lifesaver. Of course, data connectors for your sources need to be supported by the tool.

This is useful when you need to create a dataset, a model, or simply a chart mixing columns and values coming from a different database or file.

BI tools usually provide connectors but it’s not possible to join data from different databases, even if it’s the same type of database engine. It’s a plus if the tool provides this functionality.

6. Monitor and Audit Tool Usage

Some BI platforms provide prebuilt dashboards and metrics that allow you to monitor the tool’s performance, watch user interactions and data sources performance, errors, usage quotes limits, query response timings, among other features. You can also have access to logs that help you debug issues or queries that need some attention.

7. Dedicated Storage, Cache, and Extra Processing Power

As you can imagine, underlying data sources can be hectic access points, having to execute hundreds of queries in a determined moment and sharing them with different stakeholders.

Some tools provide memory space or temporary tables on their end where queries results are placed for posterior reuse (caching) or processing with internal mechanisms. Other tools rely a hundred percent on the capabilities of your DWH or database, hitting the engine with lots of queries created on the background of the tool with every modification in your visuals, filters, calculated fields, etc.

This is an important feature to bear in mind if you have performance issues in your databases, cannot afford a more powerful DB engine, or have a DWH pricing plan that charges on compute time.

8. Filters and Parameters

All BI tools bring you (or they should) different ways of filtering your data directly from the UI in accordance with the visualizations used. These filters are usually created by using values on a particular table column or by taking values from a dataset or a query.

Parameters can also be available to create custom or selectable values that can be part of calculations, queries, filters, etc.

Some questions you need to be asking:

• How do filters and/or parameters work?
• Do they need to be defined in a data model?
• Can they be passed dynamically to the query executed to save processing time?
• Can be filters and parameters reused?
• Is there any security risk by sharing charts or dashboards with filters or parameters, especially outside the company?

It is worth mentioning “date filters” since they are, arguably, the most used types. That’s why it’s crucial to spend some time knowing how many different date filters are available and how to use them. Typical options would be date picker, free text date, calendar or range calendar (select an initial and an end date), period selection (last month, quarters, years), automatically generated or based on your data.

9. Available Visualizations

A visualization (or chart) is a representation of your data through visual objects, such as tables, lines, bars, figures, etc. These representations give relevance to some aspects of the data you need to highlight so readers can easily interpret. Some visualizations can be very powerful, combining dimensions and metrics that show more meaning than the raw data.

You need to look at the tool’s available charts, how they are used, and see how flexible they are. Standard visualizations are Bar Chart, Line Chart, Scatterplot, Pie Chart, Waterfall Chart, Funnel Chart, Heat Map, Histogram, Box Plot, Maps, Tables, Indicators, Area Chart, and TreeMap.

10. Reuse of Visualizations

Often you will array visualizations that convey the desired meaning of your data into dashboards so your stakeholders and other readers can understand what you are trying to communicate. When you create multiple dashboards, you will notice how handy it is to reuse visualizations that have already been deployed in other dashboards. This will speed up the creation process and help manage them and give a unified view showing the same information in several places.

Some tools give you the possibility of re-using your charts, others help you by using global datasets, and others force you to replicate everything in each dashboard (sometimes needing to re-write SQL queries).

11. Custom Visualizations

It’s probable that you eventually run into a situation when the available visualizations in the tool are not flexible enough for your needs, or there’s not a chart that covers what you’re looking for.

Some tools let you add a third company visualizations. Also, there are plenty of open-source visualization modules you can use and even modify through javascript and CSS code. Other options include languages with visual capabilities such as Python or R. It is advisable to verify if any of these possibilities are possible.

To some extent, this customization of charts can be complex and may need a specialized intervention of a programmer or a web designer.

12. Images

Images can be relevant for representing your business: products, catalogs, design purposes, etc. So you need to evaluate if you’re able to include images in your dashboards (embed them into your tables or in the layout), download them, create some dynamic behavior in your images according to your data results like include links or tags in them.

13. Embedded Charts and Dashboards

Most tools provide a way to embed charts and dashboards into your applications and websites, integrating the functionality of a BI tool into them. There are many benefits of having this feature: the democratization of data and insights, sharing your data with clients, increasing the value and user experience of your application/website.

Being able to customize the look and feel of your embedding is a plus since you avoid interrupting the visual experience of users.

Extra costs can be charged separately from your BI tool plan, so it’s relevant to know if what additional fees may be involved. Also, a web designer can be required.

14. Pricing

One of the most critical points to evaluate is the pricing. Some vendors are entirely open about their pricing, which is excellent, but others prefer to deal with it privately.

You’ll find tools that are 100% free as a whole or with limited features. You can find users or cooperative plans. Others will charge a flat monthly/annual license with free unlimited viewers and an additional fee for every “power” user (those who develop and administer the tool).

There could be hidden prices such as support, customizations, training, excess quotes, implementations, etc.

So, before starting to adopt a particular vendor, it is highly recommended to know precisely your budget limits, the specific purpose of the tool, what amount of creator and viewer users are necessary, and the features provided you need.

15. Scheduled or Automatic Reports

A good tool should provide the means to deliver information right to your users easily. Scheduling reports or dashboards allow you to set up specific parameters to deliver information strategically: selecting your targets, choosing frequency and time (daily, weekly, monthly, etc), and delivery means (email, messaging system, website, storage services).

Some aspects to consider here:

• Is there a limit to the number of schedules you can create?
• Is the Email/FTP server functionality provided, or do you need to implement one of your own?
• Can you send scheduled reports to external recipients (not a user of the tool)
• Can you set filters and parameters before sending the reports?
• What file formats can you include in your schedule report: CSV, Excel, PDF, etc.
• Is there any preventing loss mechanisms, resending email if an issue arises in the generation, error reporting, etc.?
• Schedule tracking (does the recipient opened the email, any click performed, etc.).

16. Alerts

Hand in hand with scheduled reports, we have the alerts. An alert is an automated message, dashboard, or chart sent via email, Slack, SMS, or any other tool when some predefined conditions have become true. These alerts usually take the form of automatic reports when something we’re expecting not to happen happens.

17. Exporting Data

Sometimes, it is helpful to see the data used in a chart or a dashboard or even download it to a spreadsheet for further or custom analysis. If this feature is available, we need to know if there’s any limit in the number of rows you can download. Some tools allow you to export your data to more powerful storage, such as S3 or Google Drive. You could do it on the fly or by scheduling the export at a specific time. Take a further look to ensure the BI platform provides a way to ensure security; only allowed users should be able to download or export data.

18. Performance of Big Tables / Visualization of Big Tables

Most, if not all, BI tools on the market will behave reasonably well when you don’t have giant tables. If your tables have a thousand rows and don’t grow that fast, you are good to go. But, if your tables hold millions of rows (big data), the performance or limits of rows imposed by the tool can be the weak link. Some tools are not happy to draw a chart with millions of rows and may freeze or fail.

Here comes in handy if cache or special processing mechanisms on the tool end are provided. If you cannot avoid issues in your BI tool with millions of rows, you will need to create smaller models in your DWH or database of choice.

Some tools state they are meant to deal with millions of rows, but the proof of the pudding is in the eating, and some of them will disappoint.

19. Security

You can repeat it till you drop it; securing your data is crucial. It is vital to ensure that the right people have access to the right information. The BI tool should provide instruments to accomplish this.

Some questions you need to be interested in:

• How does the tool handle user groups, roles, and access to data and dashboards?
• Is there any granular permission level to access reports and/or data points within the reports (rows, tables, dashboards, folders)?
• Can security defined at the model level be enforced for all downstream analytic content?
• Is there any mechanism to ensure security when you share your dashboards?
• Authentication/login methods?
• Are provided access and utilization logs?

20. App Integrations

There are plenty of different tools or applications we interact with every day in this interconnected world we work in.

Lots of modern BI tools have several integrations with a few useful apps. The most common are:

• Github (Bitbucket, GitLab, SourceForge, or any other version control tool): this integration provides a double benefit: You can use Github to keep track of your changes and backup SQL queries, code, dashboards, charts, and any other objects. And also, you can pull data from Github to create dashboards of commits, collaborations, changes, etc of your development teams.
• Slack (or any other communication platform): this is a standard integration offered these days. You can use Slack to push manually or using a scheduler, messages, charts, dashboards, and alerts.
• Google Suite: also common these days. You can push or pull data from Google Sheets, Docs, Drive, Gmail, etc.

Other valuable integrations could be SalesForce, Google Analytics, JIRA, Zendesk, Mixpanel, QuickBooks.

21. Historical Tracking of Changes

Being able to browse the changes introduced in queries, datasets, charts, dashboards at any point in time is helpful. This is important to control who made what change and when it was made, or debugging an error or revert a change.

Tools can globally track modifications; others only track the history of changes you made as a user. Sometimes this is accomplished using built-in features or using integration with other tools such as Github.

22. Interactivity in Dashboards

It is highly attractive, letting your dashboard behave dynamically and allowing the user to interact with the charts. Not only the user experience is improved, but more importantly, allowing you to discover new insights just by clicking on a visualization.

In a dashboard with multiple charts, a highlight action can have a broad impact on any of those objects. You can select any dimension you’re interested in and have a cascade effect in the whole dashboard. Usually, you need to define these actions and effects.

When you have web pages, images, or any other objects, embedded in your dashboard, you can target them with dynamically generated URL actions, for instance.

A common and useful action is allowing you to go to related dashboards or charts.

Drilling down capabilities on charts is also a common action you can take by clicking on a dimension or a measure to view more specific details. Also, you can opt for a rolled-up action, so you can quickly and easily understandable information on a higher level.

Drilling down/rolling up is frequently automatically enabled in date dimensions, so you can browse from a minute to a quarter dimension easily.

23. Pre-generated Models and Visuals

Some tools dig into the data for you, creating automatic models and visualizations. This could be delivered through a sophisticated artificial intelligence module or just as a simple representation of your business, giving you a starting point if you don’t know where to start.

24. Data Catalogue

Your efforts in creating data models and stunning dashboards can be frustrating if users don’t understand the meaning of data or cannot find what they’re looking for.

If you don’t have specific tools for data governance management, it would be a handful that the BI solution incorporates a data catalog where you can give a single source of the definition. It consists of metadata in which definitions of data store objects such as base tables, columns, relationships, calculations, views, aliases, value ranges, indexes, users, user groups, and sometimes space to define your dashboards and visualizations.

25. Collaboration and Versioning

This is an important topic when several people are allowed to create and edit objects in the BI tool.

Logical questions we can be asking are:

• Can multiple users be working in the same dashboard simultaneously and not interfering among them?
• Are you able to see changes other colleagues introduced?
• Can you share changes in a dashboard without publishing changes?
• Can you share them with people who don’t have a user in your tool or even are outside your company?

Another helpful way to collaborate is allowing people to add comments on your dashboards or charts. This can be simple as a text or more sophisticated, letting you have threaded discussions, uploading images or videos, and then tagging the whole conversation as closed, done, or pending.

Some tools allow you to have draft instances of your objects that can be shared and tested separately from the published version. After being approved, they can be merged to or be the replacement of the public version. This is what is called the continuous development of production dashboards.

26. Error Handling

A good tool should be elegant when showing and communicating errors. This is especially important when you’re developing something, and some error arises: are these messages clear enough to help you solve the issue? Some tools briefly will tell you: “an error has occurred”.

When errors or some changes happen on the underlying data sources, dashboards usually get broken, or an error message and previous successful data execution are shown.

Does the tool provide any mechanism of letting you know something bad has happened (email, messages)?

27. Advanced Analytics using Programming Languages

You can install or use pre-installed programming languages in some tools. The most common ones are R and Python, powerful programming languages with extensive capabilities for analyzing and manipulating data.

Sometimes you can even install libraries and packages for these languages to extend their functionality with features that enable complex analysis, machine learning, or more sophisticated visualizations. You can easily reuse code already deployed in other applications.

This availability can demand specialized members in your team, such as programmers.

28. Cloud vs Desktop

Some tools have a clear difference between their desktop version and their online or cloud version. In others, this difference is blurry, and practically they let you do anything on both versions. Usually, a good policy is using desktop versions for draft versions connecting with test databases so you can avoid interfering with the public online version.

It’s important to verify that the desktop version lets you publish or push your changes to the production version, being the destiny of the online tool itself or some version control system.

29. Functions and Formulas

Besides the powerful functionality coming with the underlying database, BI tools usually offer an extensive set of functions and formulas you can apply to your data. This can be as easy as clicking on a function to add or as complex as dealing with a proprietary programming language.

It is important to look at how these functions are described in the documentation (if available) and any examples available. Also, how complicated they are and if there is a way to measure their performance.

30. Predictive and Prescriptive Analytics

Some tools offer Predictive Analytics; by using your historical data up to today, they try to figure out what’s probable to happen. For instance, you can have a line chart with your historical sales, and these features can automatically add trends for the following year.

A more complicated and rarely provided is Prescriptive analytics, which utilizes machine learning, powerful mathematical and statistical algorithms to outline possible future scenarios, past and current performance explaining the why’s and how’s of the situation and suggest a course of actions.

31. Customize Dashboard Views

Being able to change fonts, colors, layouts it’s a plus. Some tools come with prebuilt themes you can select from; others are much more flexible to the point you can modify HTML code and stylesheets. Mobile device views vs. desktop views availability are desirable to evaluate, too. Sometimes tools automatically create a responsive view according to the device used by the user; others will let you create different views for the same dashboard. You don’t take for granted this point if your target audience is planning to use a diverse set of cellphones, laptops, and desktops to view your reports.

32. Online Support and Documentation

You can be sure you’ll bump into issues, errors, questions, and doubts about any tool of choice. That’s why it is crucial to count on support. Most of the tools provide support; you may need to verify their availability and response time and any extra charges.

Online documentation and videos are a great ally to self-learning and get in to

uch with the full functionality.
Also, it is advisable to check out if there are any communities built around the tool. Those communities share tips, solutions, and experiences via blogs or communication platforms such as Slack.

Many vendors also provide seminars and courses for their clients and, some of them even schedule one-to-one meetings to listen directly from users about how they are managing.

33. Self Documented Dashboards and Charts

Dashboards and charts can be challenging to understand, even more, if you don’t count with a data catalog (see previously explained). Is there any way to easily add context to our objects, like adding description text, hover messages (when mouse upon a chart or element for a couple of seconds), links to documentation.

34. Difficulty Migrating to Another Tool

A tool migration can be needed for many reasons. Migration is an aspect often overlooked.

Suppose the tool is very specialized, for instance. In that case, you need to create your models with proprietary language, or all your visuals and data transformations are built upon the tool, it could be complicated to do a clean migration. The best-case scenario would be to have all your models and queries written in something universal such as SQL.

Some tools have export functionalities that allow you to export programming code and script directly to repositories or files.

One last thought here: maybe you have adopted a tool after giving it a good look, but after a time of using it, you release it’s not what you were expecting. Don’t let yourself be trapped in a tool you’re not comfortable with; it’s better to reinforce your efforts moving to a new tool than pay the cost of having a tool that’s not a good fit for your business.

Final Thoughts

Choosing the right BI software for your business isn’t always easy. As you may notice, there are plenty of aspects to consider when making the decision. As the first step, make sure to speak with the current and future data users to understand their needs, expectations, culture, technical/business knowledge, goals, and budget. This will help you to draw that starting line.

Something else to keep in mind is that by maintaining your data models and main business logic on the Data Warehouse side, you will have more flexibility to move to a different BI platform if the one you picked first didn’t satisfy your company’s needs. If you have a lot of business logic and complex calculations and formulas embedded on dashboards it will be harder for you to migrate later.

Last is pricing; some tools fit different budgets; you can start almost for free and scale your investment as your business grows.

Thanks for reading and stay tuned for specific BI platform evaluations coming soon.

Dynamic Data.