I spent 5 years leading a data team which produced reports for hundreds of users.
In our team’s experience, the most important factor in getting engagement from users is including the right context directly within the report - definitions, caveats, annotations, narrative. This pre-empts a lot of questions about the report, but more importantly builds trust in what the data is showing (vs having a user self-serve, nervous that they’re making a decision with bad data - ultimately they’ll reach out to an analyst to get them to do the analysis for them).
The second most important factor was loading speed - we noticed that after around 8 seconds of waiting, business users would disengage with a report, or lose trust in the system presenting the information (“I think it’s broken”). Most often this resulted in people not logging in to look at reports - they were busy with tons of other things, so once they expected reports to take a while to load, they stopped coming back.
The third big finding was giving people data where they already are, in a format they understand. A complicated filter interface would drive our users nuts and turned into many hours of training and technical support. For this reason, we always wanted a simple UI with great mobile support for reports - our users were on the go and could already do most other things on their phones.
We couldn’t achieve these things in BI tools, so for important decisions, we had to move the work to tools that could offer text support, instant report loading, and a familiar and accessible format: PowerPoint, PDF, and email. Of course this is a difficult workflow to automate and maintain, but for us it was crucial to get engagement on the work we were producing, and it worked.
This experience inspired my colleague and I to start an open source BI tool which could achieve these things with a more maintainable, version controlled workflow. The tool is called Evidence (https://evidence.dev) if anyone is interested.
Ironically one of the major uses of analytics has been to highlight the impact of slow response time on user retention for a wide class of applications.
I also feel that speed builds trust, although I don't know specifically why. Perhaps people envision more errors or error-prone processes when a system is slow. It certainly shows more understanding of the data to present it quickly.
In our team’s experience, the most important factor in getting engagement from users is including the right context directly within the report - definitions, caveats, annotations, narrative. This pre-empts a lot of questions about the report, but more importantly builds trust in what the data is showing (vs having a user self-serve, nervous that they’re making a decision with bad data - ultimately they’ll reach out to an analyst to get them to do the analysis for them).
The second most important factor was loading speed - we noticed that after around 8 seconds of waiting, business users would disengage with a report, or lose trust in the system presenting the information (“I think it’s broken”). Most often this resulted in people not logging in to look at reports - they were busy with tons of other things, so once they expected reports to take a while to load, they stopped coming back.
The third big finding was giving people data where they already are, in a format they understand. A complicated filter interface would drive our users nuts and turned into many hours of training and technical support. For this reason, we always wanted a simple UI with great mobile support for reports - our users were on the go and could already do most other things on their phones.
We couldn’t achieve these things in BI tools, so for important decisions, we had to move the work to tools that could offer text support, instant report loading, and a familiar and accessible format: PowerPoint, PDF, and email. Of course this is a difficult workflow to automate and maintain, but for us it was crucial to get engagement on the work we were producing, and it worked.
This experience inspired my colleague and I to start an open source BI tool which could achieve these things with a more maintainable, version controlled workflow. The tool is called Evidence (https://evidence.dev) if anyone is interested.