Part of the “Motivate Teams to Track Time” initiative at Harvest, one of our product managers conceptualized a weekly emailed report as a way to help teams get more concrete benefit to tracking time. In her research she found that many of our end users (the folks who are required to track time by their employers) don’t have a clear understanding of the reasons they need to track time. This leads to entering your time late or not at all, and the stress of trying to remember what you did over the course of the week.
Our aim is to reduce our customers “time-to-entry”, meaning to align their time tracking with the time of the actual task they’re completing at the time. There are many ways of accomplishing this with automation and anticipatory design, however we decided to work with what Harvest already has rather than creating any new technology or data points.
The goal of the personal report email is to give Harvest customers a clear snapshot of where their time last week went, and offer insight into where they’d like to spend their time in the future. The data visualizations give people the opportunity to see where they need to fill in the gaps and a visual guide to which projects and tasks they’re spending the most time on.
For example, someone might look back and see they spent most of their time on a task they don’t like, and decide to try to limit their time on that next week. Seeing all of your time in one succinct report of the past week could also provide our customers with positive reinforcement of a job well done.
Opt-in vs opt-out
In my initial user flow above, I planned on opting-in all new users to Harvest automatically so that they would receive the report once they begin tracking time. For existing users, I was hesitant to sign them up for something they didn’t ask for and wanted to give them the opportunity to opt-in themselves.
However, one of our goals was to use this feature to make a big impact on the motivation of our customers to track time and the founder felt strongly that we should automatically opt-in all users. This would make the report email opt-out by default.
Going in this direction of automatically sending the report to all users required a few considerations: 1) We would include an intro message and beta tag to let our customers know that we trying something new (and let us know what you think in the survey provided), 2) There are hundreds of thousands of users in Harvest, how could we limit the report distribution to only the people who would be most likely to find this useful?
We learned that approximately half the users who have ever tracked time, have tracked time at least within the past month. So at least 50% of users would receive an empty report because they hadn't tracked time within the past month. In order to prevent turning on this feature on for someone who hasn't tracked time in ages (i.e. only 1 time entry from a year ago) we limited enabling the report to only customers who have tracked time within the past month.
The mockups above show iterations over the course of designing, building, releasing and testing the personal report email. Early iterations included more motivational messages and fun facts (such as how many hours left for your team to circle the Earth). However we decided to simplify the report to include only existing data in Harvest and give it a more utilitarian look and feel in line with the brand. We started out with the most essential information including total hours, broken down by the percentage of time customers spent on each project and task each week.
I included a survey link to understand more about why the report is or isn't useful for people, and what would make it more useful. After releasing the report to 5 percent of customers we started getting some feedback that the hourly totals were inaccurate. After digging into the customers accounts a bit we began to see that the perceived inaccuracy wasn’t a bug. There are two reasons folks thought the hours were inaccurate - one is that Harvest rounds the hours based on company settings, and the other is the time that the report is generated. As a result, we added a bit of text in the footer to indicate the company’s rounding settings as well as the timestamp indicating when the report was generated.
Since we were only offering a one-time generated report in this first phase, we had to take into account customers various time zones, and to ensure they would receive a report of the full previous week we generated and sent the report at 6am every Monday.
Another important tweak we made as a result of the feedback was to include client names alongside the project breakdown. Customers had written in to say that the project breakdown wasn’t useful for them because they had projects with identical names for each client. By adding the client name we knew the report would begin to be useful for a larger percentage of customers.
Success metrics and next steps
The survey allowed us to create quantitative measurements from the qualitative data. We learned whether the report was motivational for people, and what data would be most useful for them in the summary. 75% of customers wrote in saying that the report was motivational for them, which was a pleasant surprise as that was the overall goal for the project. As a result of the feedback on what customers would need to see in the report in order to make it more useful, in our next iteration of the report we’ll be implementing a new graph for billable vs non-billable hours.
Our quantitative metrics were open rate and unsubscribe rate, and open rate has stayed steady around 14%, which isn’t bad for a cold email. The unsubscribe rate has stayed below 0.5%, which is a positive sign as well.
Much of the feedback we received included requests to be able to choose a custom timeframe, generate the report independently and to be able to drill in and see more detailed information on their time. The most useful next step for the report will be to present it in a dynamic view in the app as that would solve all of these issues.