17. December 2016

How data can assist us in forming good habits

Habits are an important key to success. But how do we establish a habit? How do we know whether our perception aligns with reality? How do we decide about the next change to make? Did we succeed? I want to know. A reflection on habits, insights from data using running as an example.

Think of an amibitious goal of yours. One that requires maybe a year or more to achieve. Identified one? I believe that besides a strong motivation, good habits are key to achieving that goal. Unfortunately, developing good habits that help us progressing towards that goal is tough. At least it is in my experience.

For example, I formed the goal of getting into ultra-running about two years ago. Clearly, this was a tough challenge for me, not something I would achieve in a fortnight. Instead, I would have to put the required efforts in. But how do I make sure the training is effective, and also reasonably efficient?

One answer: seek advice and feedback from more experienced runners. And finally use data to check-in with myself. The key is to understand my current habits, see which bad habits I need to drop, and which good habits I need to develop. And this is where data can play a crucial part. It helps me see the things I don’t want to see. The bad habits, the weaknesses, the sacrifice I should do but I don’t, and so on. On the other hand, the data also helps me to look back and celebrate where I succeeded. And the latter is important when it comes to overcoming obstacles:

Develop a mind bank of positive images and thoughts – family, friends, previous successes, favourite places, a big plates of chips.

Chrissie Wellington in A Life Without Limits [1]

Why are habits so important for achieving a goal? By habit, I mean a behaviour that is learnt and recurs, and benefits you in some aspects, and creates cost for you in others. My goal with this article is, besides reflection for myself, to inspire readers to ask similar questions about their own habits.

Is it worth to collect data about habits at all? In the past, recording data and keeping detailed logbooks has been expensive. Wel, we would need to have a good habit of keeping that logbook in the first place! That is recursive, must we go deeper? No, we do not have to. In the present, we can avoid spending too much efforts on collecting the data we want. Because the collection has become very easy and effortless itself. Apps help me keep a log, calendars help me understand where my time goes, spreadsheets help me to analyse the data with little effort. For me that is perfect, as I do not like to get obsessed with keeping and maintaining those logs. I am only interested in reading them. Technology is the enabler.

So why collecting data? I’m not saying that “common sense” and self-knowledge should be replaced by data. Instead, I think that data will assist me in spotting where I am in dire need of forming a habit, and where I have successfully formed the habit, and where I am still off target.

Data in itself is brutal. It shows no mercy to the one looking at it. I look at my data and it tells me that I did not run for the last weeks.

Previous post: Quantify this: Running

Time to despair? Certainly not. This reminds me of Carl Sagan’s statement on what happens if the answers given by science do not match what we might want to hear:

For me, it is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring.

Carl Sagan in The Demon-Haunted World: Science as a Candle in the Dark [2]

The time of new year’s resolutions is around. Some studies suggest these routinely fail. I would not be surprised. I personally don’t believe in new year’s resolutions. The new year’s resolutions offer procrastination in the year before, allowing to postpone new resolutions for change. Incremental change is often easier to implement than integrating changes to our lives in a big bang fashion. What is the model with new year’s resolutions? Too often: collect a wishlist of changes, turn them into resolutions at the beginning of the new year, and then be grounded by reality in the year to come. For example, suddenly I will start eating healthy, exercising regularly, sleep sufficiently? All at once? Unrealistic. That is not pragmatic.

Instead, particularly when it comes to changes that materialize marginal gains. By marginal gains, I mean gains that you can achieve by making a change along a particular dimension of life. For example, a friend who wants to be able to sleep better, has several years of data collected on his sleeping quality. And sleep is a complex thing. If I drink coffee or coke in the late afternoon or later, my sleep quality suffers. I don’t need data to establish this causal relationship. But many other relations are much harder to establish. This is where collecting data can help. Even if the data reveals no pattern, it provides us with information.

Today, we live in the information age. Never has it been easier to collect data on your own habits. Never has it been easier to analyse that data. The more difficult part however is to make conclusions based on this data. And how to turn these conclusions into action.

Repeating myself, forming good habits is one of the crucial keys to success. We are creatures of habit. If we believe in that statement, there is a logical consequence: once we master our habits, we become masters of ourselves.

Why not just willpower to implement a habit? Why not just say “I want to do this” or “I want to stop do that”? Because willpower is thought to be a depletable resource [5]. And it is in my experience. If so, relying on willpower alone is a road to failure. And it is a painful road, too. How to overcome the problem of limited willpower then? Use our willpower on the challenging bits, and automate the rest. Willpower is required to find a way to automate, to establish the behaviour we want, in a way that requires little willpower when it comes to repeating the behaviour.

What data might that be? I have not been very specific until now. In 2015 and 2016, I have collected data on sports-related activities. This was not because I knew what kind of questions I would be asking now, but rather a feeling that I might be asking questions later. The dataset comprises

  • almost every single run (distance and time)
  • the majority of the day hikes (distance)
  • most of my cycling (distance)
  • almost every single long-distance ice-skating trip (distance)
  • strength workouts (count)
  • yoga sessions (count)

Total mileage:


Running only:


At this point, I have to qualify my statement that data collection has become easier. It has, but there are still hurdles. The data for many of the apps is siloed. For example, exporting Strava activities to a spreadsheet is currently impossible. Which meant that I had to copy the weekly totals week by week. Most of the yoga and workout sessions, I was scraping from my calendar. Still this was all completed in a couple of hours on one evening. Compared to the total amount of time spent on the activities themselves, this effort is negligible.

Some of my analysis was driven by sheer curiosity.

  • Did I cycle more or run more in 2016? I cycled more.
  • How much did I cycle on Boris bikes in London in 2016? Over 250 km.

But then more about answering questions on my training habits:

  • How far do I run on average?
    • 12 km per week in 2015.
    • 19 km per week in 2016.
  • Do I run regularly?
    • No. Deviating by 16 km from the average distance per week (in 2016).
  • How many strength workouts did I do?
    • 20 in 2015.
    • 14 in 2016.
  • How much of the running distance did I cover in races?
    • 8% in 2015.
    • 27% in 2016.
  • How often do I run a lot?
    • In more than 90% of the weeks I ran less than 50 km per week in 2016.
    • In more than 70% of the weeks I ran less than 25 km per week in 2016.

I can also understand why there were breaks in my training. Either they were on purpose (e.g. tampering, recovery, vacation), they were unplanned (e.g. sickness), due to other priorities (e.g. work), or just because of laziness.

From the data, it becomes clear that I do not have a weekly routine. Habits don’t necessarily need to be tied to fixed date & time, even though this usually helps because a fixed schedule makes it easier to automate the repeated behaviour. So this is maybe something I will need to evaluate: how can I stay flexible while not letting my training slip?

I was surprised to find that, in 2016, I spent more than a quarter of my total running distances in races. I was of course aware that I had registered for a handful of races but I was unaware that they would make such a big piece in the cake.

So, how and when does this data become useful. I believe there is a certain cycle for achieving a goal.

  1. decide where to go (the vision)
  2. decide how to get there (the plan)
  3. go (execution)
  4. see whether it is working (measure)

Here, steps 2-4 are likely iterative. Steps 1, 2 & 4 profit from data, but step 3 likely less so. Why? We don’t want to keep second-guessing ourselves during execution.

For example, I prefer to race without gadgets. I use my intution. I want to be grounded in the present, focused 100% on the step I am taking and not worrying about the rest. I want to feel my pulse, my breath, my body. And I believe that is a metaphor that also applies to other areas than running. For example, you might want to split your writing & editing task into two phases (no second-guessing during writing, all the criticism coming in the editing phase) [3].

But then, we frequently find that our plans are not working. At least I do. If so, we need to find out early, and that is why we measure frequently and iterate between 2-4. If we find there is something wrong, then the plan must change. Simple as that. Note that this does not mean that it was a bad idea to have a plan. Having the plan and being able to decide whether it is working is a learning opportunity. It also allows you to predict consequences of not following the plan [4].

In summary, I think that nowadays, it has become easy, without too much efforts, to log the data that will help us to see whether our plans are working, where there is a lack of a plan. It will help us on the road to implementing habits. Habits that in turn will help us to achieve the goals we want to achieve. A very basic statistics and a spreadsheet will already get us very far, unless we are using data to squeeze out the few percent that are required to give you the edge over the world’s best professional cyclists [6]. In my case, and in many of your cases, we don’t need the level of detail, we just can use the data to pragmatically guide us towards our goals.

[1] A Life Without Limits by Chrissie Wellington
[2] The Demon-Haunted World: Science as a Candle in the Dark by Carl Sagan
[3] Learning How to Learn, online course on Coursera
[4] Working Smarter, Not Harder, online course on Coursera
[5] Is Willpower a Limited Resource?, American Psychological Association
[6] Lactate Threshold Training by Peter Janssen