Exploit or Explore?

What the story of the ubiquitous trade-off reveals about the possibility of a different life

Lev Janashvili

Sep 25, 2024

Transcript

This conversation about the exploitation-exploration trade-off is the culmination of four years of dialogue between two avid explorers. In this wide-ranging discussion, my guest and I examined not only the computational aspects of the trade-off but also its implications for dialogue and theories of psychological types. See time stamps and annotations below.

Feel free to post comments and questions, but always practice what Google preached in its IPO prospectus: Don’t be evil.

If you prefer to watch this video on YouTube, here’s the link.

To receive future episodes by email, subscribe for free.

Time Stamps and Annotations

0:00:00 - Intro

0:03:26 - What’s the story of the exploitation-exploration trade-off?

Classic optimization problem.
The quest for optimal choices that minimize regret.
Weighing the cost of exploration.
Key: optimism in the face of uncertainty.

0:06:25 - How did you develop an interest in this trade-off?

Unsolved problems in the applications of Machine Learning (ML) in quantitative finance.
Optimizing trading strategies: the balance of greed and fear.
The discoverability of “objective truth” about optimal choices.

0:09:40 - What’s the problem with the way people approach this trade-off and trade-offs in general?

Suboptimal decisions in daily life.
Adjusting our default settings to optimize the exploit-explore balance.
Applications across contexts/scales/domains.
Bayesian thinking: data gathering as a function.

0:13:33 - The problem of measurability: How do you know you’re making the optimal choices?

Measuring gains.
Zigzagging through incomplete information.

0:15:28 - Specific context: Marketed products vs. products in development?

A-B testing: optimizing choices of products and features.

0:18:44 - Grappling with the limits of measurability

What do you do when you don’t have data for a data-dependent decision?
“The distance from theory to reality is much shorter in theory than in reality.”
Statistical significance depends on data, but relevant data isn’t always available.
The model should follow the data.
Turning data into stories (without manipulation).
Zigzagging with diminishing returns.
The impossibility of 100% certainty in the Bayesian view.

0:26:00 - Over-investments in exploitation and exploration

Over-valuing existing information and undervaluing new information, and vice versa.
Humans as born explorers reading their environment for signals suggesting a shift toward exploitation.

0:30:24 - Beyond computation: dialogic and psychological dimensions

Avoiding sloppy psychological typologies: the PR industry as a case study of explore-exploit optimization.
Subconscious computational mechanisms.
Early adopters vs. late adopters.

0:34:48 - Creating safe environments for exploration

Reducing risk reduces reward.

0:37:08 - Examples of aggressive investment in exploration

Venture capital as a highly exploratory business model.
SpaceX as a highly exploratory mission-driven venture.

0:40:59 - The multi-planetary species

Is the idea an example of over-investment in exploration?
Changing our relationship with the radically distant future.
From science fiction to shaping the institutional agenda.
Spreading consciousness through the universe.
Word cloud: multi-planetary species, transhumanism, posthumanism, humans-AI fusion. At what point, are we exploring too much?
Taking positions prematurely.
Vigilance about parasitic influences and the simulation hypothesis: Distinguishing the optimal from the optimally destructive.
Evolution is exploration through gene mutation.
Fusion of biology and technology.

1:01:46 - The diminishing returns of exploitation and the bull market for exploration

The value of existing knowledge has diminished.
Ultimately, exploration is what will save us as a species, but we have to make some greedy decisions now.
When we confront great uncertainty, we expect great rewards.
Innate optimism: seeding the universe with encapsulated consciousness.

1:04:49 - The Certainty Trap

What happens when the explore-exploit spectrum gets militantly polarized.
Examining the exploit-explore trade-off from three perspectives: computational, dialogic and psychological.
What happens in confrontations of Bayesian minds and dogmatic minds.
Dogmatism: a high-confidence path to extinction.
Fear of being reinfected with doubt. The Exodus story: an interpretation of Amalek.
Wallace Line as a metaphor for polarization.
What’s being threatened: the continued dominance of false categories.
What’s being thwarted: the recognition of real differences - psychological types.
Organisms hijacked by viruses.
Becoming different species.