TASK

Present your approach to designing an executive-level report that encourages renewal of a (lightly fictionalized) security product. The goal is to understand how you work, what your process is, and how you present your thinking to a group.

We would like you to cover things like:

Research:
- Who would you talk to? Why?
- Given the background that we’ve provided, what would you want to know?
- How would you structure your research?

Design:
- What would be your proposed solution?
- How did you arrive at that solution?
- How would you know when it’s the right solution?
- What questions would you ask?

BACKGROUND

The Cloud Security Product (CSP) is sold via a subscription model. Our customers renew CSP on a yearly basis. CISOs (C-level executives in charge of security for a customer organization) make all purchase and renewal decisions. CISOs do not use the product themselves; security administrators within their organization use the product. The buyer--the CISO--wants to know on a quarterly basis if CSP is providing value to their organization. This understanding factors into the annual review of the contract for CSP, and greatly affects whether or not the CISO will opt to renew. In this case, “providing value” means that CSP is catching security problems and helping the company remain compliant. Currently, these quarterly reports are created manually in Excel by Security Admins, using data exported from CSP. We would like to replace these reports with something automated, that can be scheduled by the Security Admin for the CIO/CISO.

Existing Personas:

CISOs:

CISOs are motivated by choosing the right solutions that provide value for their company. They need to defend their decision to move their company’s data to the cloud. They require evidence that every solution they're responsible for choosing has been the right decision, and CSP is no different–if they don't see evidence of value, CSP may be dropped as a solution. They also need to make sure that their company remains in compliance and avoids bad PR from breaches. They are most likely to interact with the CSP application via reading app-generated reports (assembled and passed to them by Managers or Admins). They prefer to receive reports via email. Their primary concerns are compliance (being lawful), protecting customer data, staying within budgets, and avoiding bad PR for the company.

Security Admins:

Admins spend a significant amount of time in CSP. It's important to understand that most Admins don't only spend their days resolving security incidents. Admins are primarily focused on making sure that their company's systems are running smoothly, and security considerations may be just one issue among many (unless the Admin in question is a full-time security analyst). In all cases, though, Admins want streamlined, predigested security information that will help them do their jobs. Admins will not respect a system that seems inconvenient or clunky; they don't have time to engage with a solution that isn't geared to their limited attention.

I.

Let's take a step back and sum all of this up. I am a Product Designer working for CSP Company tasked with creating an automated executive level report for the CSP software that will be read by CISOs (the decision makers when it comes to contract renewal). The CSP software, though, is not actually used by the CISOs but rather by busy Security Admins.

Here is a map sketch I made in order to help me organize the key points from the brief:

While creating this sketch my mind started buzzing with questions. A lot of the questions I had do not have to do directly with the creation of an executive level report, but I believe that it is important for me to understand the all aspects of the product and users (users being both the CISOs and Admins) so that I can design a report that will be as successful as possible.
Here are some of the questions that jumped out at me:

How exactly does CSP work? I would want to spend time reviewing the software and fully understanding all of it's features and use cases.
What is subscription model for CSP? Are there different levels? Being employed by CSP Company, I would want to know what CSP Company's business goals are - do they want their users to retain their current subscription, do they want to encourage upgrade?
How does the costs of CSP stack up against the CISO's budget? How does both the cost and the effectiveness of the software stack up against competitors?
How much does the CSP report impact returning subscriptions?
What other features does CSP have? Besides for creating reports, how to the Admins interact with these other features?

What exactly are the other responsibilities of the Admins? What do their day-to-days look like?
What are some of the other responsibilities of the CISO? How long do they spend reviewing the report? Who are they defending their decisions to? Why do they prefer email reports?
Where does the information for the reports come from? What are the steps the Admins have to take in order to create the reports? How long does it take them now?
According to the information provided, the CISO decides CSP's value based on a quarterly report of security problems assembled by the Admins - what are the specific data points included in this report that are used to determine CSP's value? Why are these points important? Does CSP provide other value to the company and to the CISO's needs other than what is currently included in the report?

In order to complete the task of designing an automatically generated report, presumably a new feature for CSP, I need to not only think about what the design of the actual report will look like, but more importantly the UX of how the report will fit into the existing workflow of CSP and its users. This is why, as I mentioned before, I believe it is important for me to learn as much as possible about the product as is now and the users.

My instinct at this point is to run straight into a research plan, but before I do that, it is crucial that I sit with my team (Product Manager, Dev, etc) in order to fully understand what our product goal is and capabilities are for this new report feature. Why has the decision been made to spend time on this report? Why does CSP Company think that by creating an automated report it will increase subscription loyalty? Isn't it the same data being represented anyways? Is subscription loyalty even the goal? If not, what is? Is our goal actually to increase productivity of the Admins? What is the timeline for this, both in terms of my time as well as dev's availability?

Obviously, due to the nature of this task, I do not have a team I can sit with in order to get answers to all my questions. But let's say I did. Now I can move on to research!

Ideally, I'd want to create a research plan that would allow me to wholly understand my users' needs. I would want to conduct interviews with both personas as well as observe them in order to find any discrepancies between what is said and what is actually done. Then, I would analyze the usage data and statistics of CSP. Following this research, I would create more robust personas and empathy maps to more clearly define the paint points and opportunities. I would conduct a competitive analysis of other cloud security products and how they present their data, deliver reports and encourage subscription renewals. And, I would also do research about best UX practices for subscription renewals and upgrades if this was not something I was already familiar with. This would then allow me to begin build app maps and user flows of how the report will be generated and sent from the Admins to the CISOs.

II.

It is important here to say that the successful design of this report is two-fold. The first part is designing a solid UX flow from creation of reports by Admins through CISO subscription renewal. This is where my research plan came in, as it required understanding the needs and goals of the user. The second part though, which has yet to be discussed, is the importance of the data visualization within the report. You can have the most user friendly flow of report creation and delivery, and the most enticing visual design, but without being able to properly and easily understand the data, what really matters most, the rest is irrelevant.

My own personal experience with data visualization is limited, but in order to complete the task to the best of my ability, I did some online research on the best data visualization practices. Here are some guiding principles I learned:

Design for a specific audience.
In my interviews and observations of my users, I will have learned about their capabilities and levels of data understanding (which may be different for the CISOs and Admins). It is important that the design of the data in the report match my research findings.
Label, label, label.
Don't take the risk of making assumptions that users know what is what, make everything super clear. Also make sure all naming conventions are consistent.
Use visual salience and prevent information overload.
Based on my research, I know which data points are most important for the CISO when making decision. Use design to make sure those key points don't get lost.

Show key data.
Studies show that even when presented with amazing graphics, only 10-15% of users will actually interact with data. Thus, it is crucial that all important data be shown upfront, rather than expecting users to find it themselves.
Keep it simple.
Most times, simple charts such as bar or line charts are the best solution, as they are the easiest to read and understand.
Tell a story.
This is true in all design. Design elements (and data points) intentionally woven together can tell a narrative and turn raw data into useful information.

III.

Again, due to the nature of this task, I am limited in the information and resources I actually have. Like I mentioned earlier, ideally, I would have created an app map and user flows in order to see where the new feature of generating a report fits into the current CSP product map and work flow. These resources would help me to start creating wireframes. Since my goal here is to be able to present some version of a finished design, even though I do not have details as to the type of information included in the report, I wanted to create some sketches and wireframes of potential solutions, outlining where key data points, secondary data points, and CTAs for resubscribing may be located.
‍
Here are some of my sketches:‍

After creating wireframes, depending on timeline and capabilities, I would want to create a prototype of the entire flow, from creation of report by the Admins through delivery to CISOs. I would then conduct usability tests in order to ensure our goals are being met. The usability tests would be both for the flow, as mentioned above, as well as for the report itself. Based on the CSP Company goals discussed at the beginning of the project, as well as the product and user research, I would come up with a series of criteria and metrics in order to define if the current design is successful or not. Some of this criteria might include:

Ease of learning
How fast can a user who has never seen a report before find and understand key data?
Efficiency of use/ Time on task
How long do users spend sending or viewing the reports? How fast can an experienced user find and understand key data? Specifically as compared to the old report.

Error frequency and severity
Are there any errors in the flow from automatic report generation through CISO delivery? Do the CISOs make errors when understand the data in the report?
Subjective Measures
It is also important to ask the users themselves what they think!

Based on the results from these usability tests, I would iterate. If time allowed, I would continue the cycle of testing and iteration until the solution was deemed successful by the criteria and metrics as mentioned above.

It is important to acknowledge that there are still many major puzzle pieces that were left out of this case study. Good design required teamwork and constant iteration, and throughout this whole project there should be many internal reviews and discussions (with PMs, Team Leads, Dev, etc.) that were not mentioned here. Rather, this case study represents an idealized snippet of the process and steps I would take in order to complete the task of designing an executive-level report. After the usability testing was finished and design set, the new feature would be implemented. After the feature is live, I would monitor usage data to make sure it is meeting our goals and in order to make more iterations when necessary.