In “Difficult Problems and Hard Work”, David McIver writes:

An idiosyncratic distinction I find useful (though don’t reliably stick to) is that there is hard work and difficult problems, and these are not all that closely related. The distinction is roughly that something is hard work if you have to put a lot of time and effort into it and a difficult [problem] if you have to put a lot of skill or thinking into it. You can generally always succeed at something that is “merely” hard work if you can put in the time and effort, while your ability to solve a difficult problem is at least somewhat unpredictable.

(emphasis mine)

I think about this distinction regularly in the context of software engineering, though I think it probably applies to most “knowledge work”. At an intuitive level, I think we’ve all encountered this: there are problems that are solvable by throwing a lot of human-hours at it (“Hard Work”), and problems that are not a function of raw work hours, but rather require dealing with ambiguity (“Difficult Problems”).

The more unpredictable the task is as a function of allocated effort to task completion, the more likely it is to be a Difficult Problem.

In software engineering, Hard Work can look like:

  • Writing glue code.
  • Write a CRUD API.
  • “Increase unit test coverage”.
  • Clone feature X from system A into system B.

Whereas Difficult Problems can look like:

  • Designing an architecture for a new, ambiguously scoped system.
  • “Improve system performance”.
  • Coordinate a rollout of a backwards incompatible change across N dependent systems.

Of course, these are generalities. There are cases in which increasing test coverage is a Difficult Problem (e.g. if it requires a substantial refactor) and designing a system architecture is “merely” Hard Work (e.g. if your company has a defined list of permissible architectures).

There is an interesting moral valance to this distinction – Difficult Problems versus merely Hard Work. Knowledge workers (especially the ambitious ones) like to think of themselves as the type of people who solve Difficult Problems. Solving Difficult Problems feels like real intellectual work (and implicitly prove’s one’s capabilities). Hard Work, insofar as it could have been done by anyone confers less status. The “shiny problems” that engineers flock to tend to look, at least from afar, more like Difficult Problems than Hard Work.

While I think there is a degree to which Difficult Problems are more satisfying to work on than Hard Work, one is not inherently “better” than the other. Getting stuck on a seemingly intractable Difficult Problem can be equally frustrating than drudging through an unending stream of Hard Work. At least with Hard Work, you usually have tangible progress to display. Leaning too heavily into one or the other can lead to burnout: Hard Work tends towards overworking; Difficult Problems tend towards frustration.

One of my favorite working habits is to have two projects on my plate at a time: one project that is “just” implementation work, and another that involves some ambiguous design. If things go as planned, the implementation work finishes around the same time the design work does, so I can start implementing the design, and pickup another ambiguous task. Ideally, this is a cycle that perpetuates itself. Since implementation tends towards Hard Work and design tends towards Difficult Problems, this habit provides a pleasant balance of the two.

As a final note, one of the points that I find particularly useful from David’s article is how Difficult Problems can be transformed into Hard Work:

Effective strategies often consist of converting difficult problems into something that can be solved through hard work. They do this by reducing the cognitive cost of decision making. There are a variety of ways of doing this, of which we’ve seen three here: Focusing on decisions where pretty much anything you can do works, prioritising decisions that quickly lead to good outcomes, and delegating to external sources of decision making. There are almost certainly many more strategies that solve this problem in other ways.

In other words, Difficult Problems involve operating in a higher dimensional space than Hard Work. Strategies that can collapse the higher dimensional Difficult Problem into a lower dimensional form of Hard Work reduce the cognitive costs to solving the problem. For example, if one could take an ambiguous problem and write a rote list of instructions that would solve the problem, that is effectively collapsing a high dimensional problem space (i.e. a problem space that includes all possible solutions) into a low dimensional space (i.e. “just follow this one path described by the instructions”).

David doesn’t go quite this far, but I’d also argue that skill acquisition can transform Difficult Problems into Hard Work. In the quoted paragraph, David argues that strategies such as decision prioritization can help collapse higher dimensional problems into lower dimensional ones. I think skill acquisition does this too. By developing skill in a domain, your mental map of the problem space becomes more nuanced. This nuance lets you skip the initial discovery phase of a Difficult Problem (e.g. constructing a map of the problem space from scratch) and more easily lets you collapse the problem into a lower dimensional one that can be approached with Hard Work.

Discussion on Lobste.rs, Hacker News