In my opinion there are many tools and products in this space and they all seem somewhat confused in their target audience (is it marketed to management, analysts producing reports or developers supporting the analysts?) The boundaries between these projects is often fuzzy and they are often complicated (does it include a scripting language?) When you are starting from close to scratch with an application and it's backing database and being asked to produce timely reports, I think these tools aren't the best place to start.
My process has been to talk to stakeholders and sketch out reports they find useful, preferably getting a set of data together that many people find useful. Running these reports against a read-only replica of production data is typically not a big lift. If dashboards are required, write this data out to another database to back the dashboard, probably on some set schedule. It hasn't been long but now we have the bones of an ETL process that is already returning value.
At that point I think these tools start to look more compelling. Now we have a handle on the source data, what it looks like and any places where we need to do something tricky to connect the dots to get our data out. We know what the reports and dashboards look like.
In short, we know what we need these tools to do and where they can help us.
One thing I often see in data people (particularly consultants) is that they are very focused on building reports and dashboards in my experience (manufacturing plant) Analytics should be the focus - if you get the analytical part right - make the data simple/easy to interrogate then the reporting can evolve naturally as a consequence.
Unfortunately I think the tools for a lot of this (exploratory data analysis) are somewhat lacking. I think we are starting to see new tools emerge - especially around Timeseries data that are promising but I don't think things are at the level where a non expert user can quickly glean insights from data in a frictionless way.
I disagree. Reports and dashboards provide decision makers with information that they require to make decisions. This is always step one.
Randomly exploring data for "insights" is why so many companies are turning against "Data Science"; it rarely bares fruit. This work should be focused on dialing in business processes.
Agreed. A lot of my background is in banking / fintech. So much scrambling on "adhoc analytics" can be avoided if you have really well constructed, standard reports and dashboards.
The department I started my career at didn't have this and every exec request was a bespoke, adhoc request, tailored to answer that one question. I spent many late nights writing SQL and building excel files / powerpoints to answer a single question.
When I started at a fintech as the first data person, we built a few primary dashboards where you could drill down for detail. Those answered over 80% of the questions people asked.
I don't think we disagree (Report and dashboards are useful decision making tools), but in practice hiring external people to come in and build them causes more problems then it solves. In my experience processes on the ground can and will change and all of these pre-written reports and dashboards quickly become ossified and outdated.
If you take steps to make sure the data is properly stored and queryable stakeholders can maintain and develop their own reporting. Make the data understandable and accessible and reporting will follow. Human friendly schemas and such help a lot.
My process has been to talk to stakeholders and sketch out reports they find useful, preferably getting a set of data together that many people find useful. Running these reports against a read-only replica of production data is typically not a big lift. If dashboards are required, write this data out to another database to back the dashboard, probably on some set schedule. It hasn't been long but now we have the bones of an ETL process that is already returning value.
At that point I think these tools start to look more compelling. Now we have a handle on the source data, what it looks like and any places where we need to do something tricky to connect the dots to get our data out. We know what the reports and dashboards look like.
In short, we know what we need these tools to do and where they can help us.