On the formation of judgment

I want to write a little bit about judgment because it's underrated almost to the point of being stigmatized. I hear a lot about data, and being data-driven, but data without judgment is almost useless. Judgment is what you do with that data, how you think about what it means, and how you determine what to do based on the meaning you arrive at. When you dig to the bottom of any transformative product, you're going to find people with well-developed judgment - perspectives, sensibilities, and convictions about what they are building and why. Judgment accumulates with experience, though accumulated experience doesn't inherently lead to better judgment. What is necessary to develop better judgment? Clarity of ideas, rigour of thought, honest measurement, and critical reflection.

Running through them one by one:

• Clarity of ideas - this isn't even about having the best ideas. It's about being able to clearly articulate the ideas that you do have, both as distinct entities and as components of larger systems. What is it, how should it work, why should it work that way, how does it respond to the problem that you're addressing (hell, do you even know the problem you're addressing in the first place?).

• Rigour of thought - this pairs with clarity of ideas in two essential ways:

  • it refines the clarity of ideas, seeking out areas of weakness or lack of clarity and bringing them greater focus

  • applied to clear ideas, rigour of thought draws out the right methods of assessment and evaluation of an idea to figure out if it's a good idea and ultimately the right idea.

• Honest measurement - to get this part right, the first two elements need to be in place. You can't have honest measurement if the idea isn't clear and the method of assessment isn't appropriate. When you've got those pieces right enough, though, you have to keep yourself honest. As much as you might love an idea and be convinced that it's The Idea, you also have to be open to the possibility that it's Not The Idea and be eager to see what the outcome of your assessment is.

• Critical reflection - this caps off the other three. When you have honestly assessed a clear idea, you should have some valuable data that you need to interpret in order to determine what to do next. This is not only true if your measurement has come up short of your aspiration; it's equally true when you've passed the assessment with flying colors. Critical reflection can also be called "Kicking Your Own Ass." Even in failure, there's probably something you did right; even in success there's definitely something you did wrong. Critical reflection requires you to accurately separate the wheat from the chaff. What's good and needs to be preserved or amplified? What's bad and needs to be fixed or removed altogether? What's the proportion of the one to the other? How close did you get to what needs to be true to prove that this is The Idea (which, hopefully, is a standard you established with your clarity of idea and rigour of thought)?

You go through this process because it produces rich feedback on your ideas, and each time you go through it you reflect on those ideas and refine them in a systematic way. Over time, practice develops intuition, and intuition accelerates the process; it doesn't replace the process nor trump the process. Intuition doesn't give you the ability to dismiss new ideas out of hand. What it does is provide you with a better lens through which to generate, examine, and interrogate new ideas. It gives your reflection sharper focus.

When you don't follow a process like this, it's not that you don't develop judgment. No, no - it's much worse than that. You develop bad judgment. You misattribute why something works or doesn't, or you don't understand the why at all and are instead just stuck with the what. You get stuck in a cycle of confirmation biases, and you're ultimately hindered in your ability to adjust your judgment in the presence of new data or new circumstances. (The unspoken scary part: even if you follow a rigourous process, you run this risk - you just increase your odds of being aware of and responsive to it).

There's a natural question of where to focus in building solid judgment. In my opinion, there are four high leverage areas, each of which is valuable on its own but which have significant multiplier effects in combination with each other:

• understanding your user - this is about being able to move beyond superficial understanding of who they are and the challenges they face. When you understand deeper motivations, behaviors, preferences - and especially when you can understand that beyond the specific problem you are focused on, you can build strong hypotheses around the kinds of product and design decisions that are likely to land.
• understanding the broad problem space in which you're designing - this is an endlessly expansive space. This is about understanding the industry, understanding the value chain you're interacting with, understanding other solutions to the kinds of problem you're working on, understanding adjacent problems and their solutions. Judgment in this area is about understanding the complex systems in which your user and your solution will need to interact.
• understanding human behavior - agnostic of any particular domain, a solid understanding of behavioral psychology allows you to identify significant points of unintentional friction that hinder the kinds of behaviors you're trying to produce.
• understanding different forms technology and their specific use cases - different forms of technology are attuned to different types of problems. The better you can identify the kinds of problems something can be unleashed on, and the converse - the kinds of problems it is badly attuned to - the more capably you can use a technology as an accelerant.

You could spend years in every single one of these areas (for the latter two, you almost certainly will spend years because the judgment you build up is valuable across domains). If that leaves you wondering where to start, I'd suggest starting from whatever feels like your biggest gap right now and build from there. The interaction of the four creates a virtuous cycle that accelerates the formation of good judgment - so shore up the weakest area first, and then look at which area will give you the most added leverage in whatever you're working on right now.