Kibana Limitations: How to overcome them.

Elasticsearch brings a lot of value

One of the business intelligence challenges companies face when using Elasticsearch is that Elasticsearch manages data in JSON documents and has no support for SQL.

This means traditional BI tools like Power BI and Tableau don’t work with Elasticsearch without a lot of help. That help comes from development teams and engineering efforts to move Elasticsearch data into a relational database–which in many cases removes the main benefits of using Elastic in the first place.

Elasticsearch has provided a great solution to this in the form of Kibana, which is part of the ELK stack. But it can run into limitations as companies scale up their data infrastructure.

Kibana is great up to a point

Kibana is a good BI solution for simple usecases where Elastic is the only data source and a single Elasticsearch index is the only source of data for visualizations.

But more and more often these days companies use a diverse stack of data sources that include Elasticsearch as well as a number of other database technologies–both SQL and NoSQL based.

Kibana’s limitation of only working with Elasticsearch might not be an issue in the beginning, but as you grow and your analytics solutions scale up, it may start to limit what you can do.

Many companies start out with Kibana because of the low cost (free is hard to beat) and low barrier to entry but find themselves in a tricky position a year or two down the road. Often they will find themselves needing to figure out how to scale beyond Kibana without an obvious path of how to do so. This is where Knowi may be a good fit.

What’s the solution?

Unlike with Kibana dashboards, with Knowi you can visualize data across multiple indexes. You can dynamically blend data from other sources, like relational data stores or REST APIs. And you can do so out-of-the-box, without having to build a data warehousing pipeline.

Knowi natively supports SQL-style queries even when working with NoSQL data sources like Elasticsearch, MongoDB, Couchbase, DataStax, and Cassandra. So the problem of getting Elasticsearch to work with traditional BI tools is eliminated.

Knowi in tandem with Kibana

One concern people often have when thinking about adding a new BI tool is that they will have to rip out all of their old Kibana dashboards that they created–even the ones that are still doing everything they need to do. This isn’t necessary with Knowi.
Knowi and Kibana work well together in tandem, so those Kibana dashboards that are still good the way they are can stick around indefinitely.

More about Knowi

A crucial aspect of monitoring sensitive systems is to detect certain issues proactively so that action can be taken in advance before it leads to severe consequences. For example, say you’re using Knowi to manage server resources. You may wish to operate at under 90% of max server memory utilization at all times. Knowi allows you to set a rule which automatically sends an alert when you exceed this figure. This would help the operations team bring down the memory utilization and hence avoid any unwanted outage on the system. You can set up the alerts to be sent to you either via email, webhook, or have them inside Slack. With webhooks, when the condition is triggered, Knowi sends a POST request with a JSON document containing the payload of the data. You can then use it to drive action within your application or any external application you connect to. With Slack integration, when a condition is triggered, Knowi sends a message to a predefined channel or to a specific person. Users can share these alerts in Slack or ask follow up questions.

The search-based analytics feature (also called natural language business intelligence) allows non-technical users the ability to query the data just like they would ask questions in a google search.

Remember earlier when we described getting a triggered alert in Slack? Users can then act on that alert by asking additional questions--all without leaving Slack.

For example, say you are heading up the sales team and you create an alert to tell you if the number of deals closed at the end of the week is 20% lower than the previous week. Then, some time later, you get that alert. The Knowi bot in Slack tells you that closed sales are down by 24% compared to the prior week. You decide to look at the overall trend and type into Slack the following: “/Knowi show me the total closed sales weekly for the past 3 months”. When you see the column chart you realize that the reduction is because the prior week was a trade show where you had a huge surge in sales.

Kibana does not offer robust user management features. This means whoever has the link to the Kibana dashboard can see the data. This might not always be what you want--especially if you’re working with sensitive data. If you need more secure dashboards, you can build those dashboards in Knowi and restrict user access. For example, with Knowi, a dashboard can be shared with a specific user or group and you can specify if the user has View or Edit access. View access restricts the user to a view mode where they can consume the dashboard, analyze the data, apply temporary filters, and download the data but cannot make any changes to the dashboard.
Knowi offers machine learning for your data as well. Knowi currently offers three machine learning models: Classification, Regression, and Anomaly Detection. For example, Anomaly Detection is often used to identify unusual patterns that do not conform to expected behavior (called outliers). This has many applications in business use cases, including intrusion detection, system health monitoring, fraud detection in credit card transactions, and fault detection in operating environments. Knowi’s data preparation wizard makes it easy to build your machine learning model and even create a Trigger Notification based on the results that will alert you if a pre-set condition has been met. The ML feature also allows you to build machine learning-based triggers in your workflow.

Want to see it in person?

Sign up here for a 15-minute demo where we will learn more about your use case and recommend a few features that will help augment or replace Kibana.

Common questions

You can certainly give it a shot. But the reality is that those platforms simply weren’t built with NoSQL data sources like Elasticsearch in mind. They were designed at a time when analytics entirely consisted of SQL and spreadsheets. Modern dynamic data architectures require modern dynamic solutions.
No, Knowi actually works quite well in tandem with Kibana. Although many users do eventually migrate over, it is not necessary to do so.

Yes, a good number of our users are AWS users.

Essentially everything. We list 30 or so that are common use cases here, but Knowi was built from the ground up to be the most flexible business intelligence tool out there. Our powerful REST API integration allows connection to most data sources with relatively little effort. And the entire platform is built on data virtualization technology that allows us to create native connections to even NoSQL data sources like MongoDB, Cassandra, Couchbase, and of course Elasticsearch.

Not exactly. Although both Knowi and Kibana can be used to build dashboards with Elasticsearch data, the use cases are very different. Kibana works well for smaller companies (especially those on a budget) who are only using Elastic data. Knowi tends to fit better with bigger, more complex, often enterprise use cases where the company uses Elasticsearch, but they also use numerous other data sources and it’s not as simple as popping a single Elastic index into a visualization. We often recommend Kibana as a low-cost starting point to startups who are interested in Knowi but are too early in the process or don’t have the budget yet.

Picking a Kibana alternative depends greatly on the use case. Kibana is primarily used as a log monitoring tool, but it has also been used for a wide range of IoT analytics and business intelligence applications.

The four Kibana alternatives that fit most use cases are:

Ready to learn more about Knowi?