The Theappgap Blog

You know the feeling. You kick off a new project, grab a tool that swears it’ll make everything smoother, and three weeks later you’re wrestling its conventions instead of solving the problem you actually had. The culprit is usually a distinction so basic that tutorials barely mention it: the difference between a library and a framework. This isn’t just wordplay. It’s about who’s in charge.

I’m Kai Lindström. I’ve spent more years than I care to count deep in front-end and back-end systems, and I can tell you this distinction isn’t some dusty academic exercise. It shapes your architecture, your debugging habits, and how easily you can rip out a dependency when it goes stale. Let’s skip the fluff and look at what actually separates the two, with code that makes the point.

The Core Distinction: Who Calls the Shots?

The cleanest, most technically honest definition I can offer is this: you call a library; a framework calls you. It’s about control flow, and it’s the only rule that holds up under pressure. With a library, you’re the conductor. You decide when and where to invoke a function or spin up a class. The library’s code is a toolbox you reach into to assemble your application’s skeleton. Your code runs the show.

A framework flips that on its head. The framework supplies the skeleton, the main loop, the overarching shape. You fill in the blanks. Your code gets called by the framework at specific points in its lifecycle. This is the Hollywood Principle: “Don’t call us, we’ll call you.” It’s not a suggestion. It’s the law.

Let’s make this tangible. Say you need to hit an external API. In a library-driven world, you’d do something like this with a fetching library:

// You decide when to make the request
async function getWeather(city) {
  const response = await axios.get(`https://api.weather.com/${city}`);
  return response.data;
}

// You call the function from your own application logic
document.getElementById('load-btn').addEventListener('click', () => {
  getWeather('Helsinki').then(displayWeather);
});

You’re holding the reins. The library axios is a tool you grab when you need it. Your code owns the event listener, the flow, the decision to fetch. Now contrast that with a framework like React. You don’t just call React to paint a button. You define a component, and React decides when to call it, re-render it, and reconcile its output with the DOM.

// You define a blueprint, but React owns the lifecycle
function WeatherWidget({ city }) {
  const [weather, setWeather] = React.useState(null);

  React.useEffect(() => {
    // React calls this effect when it decides the component is ready
    fetch(`https://api.weather.com/${city}`)
      .then(res => res.json())
      .then(setWeather);
  }, [city]); // React re-calls this if 'city' changes

  return weather ? 
{weather.temp}°C
 : 
Loading...
;
}

// You don't call WeatherWidget. React does, inside its reconciliation engine.

In the React example, you handed over the keys. You wrote a function, gave it to the framework, and said, “Call this when you think it’s appropriate.” That inversion of control is the whole ballgame.

Why the Distinction Isn’t Just Academic

This isn’t about purity. The library-versus-framework choice has direct, sometimes painful consequences for how long your project lives and how sane your team stays.

Architectural Freedom vs. Architectural Guidance

A library is a tool you bolt onto your existing architecture. You can structure your application as MVC, clean architecture, or a big ball of mud—the library doesn’t care. It just does its one job, like date formatting or making HTTP calls. That’s freeing, but it’s also dangerous. It dumps the burden of architectural coherence squarely on you and your team. I’ve stared at codebases where a dozen excellent libraries were stitched together with no overarching design, and the result was a system nobody could reason about.

A framework, by contrast, imposes an architecture. Angular hands you modules, services, and dependency injection. Next.js gives you file-based routing and a strict separation of server and client code. That’s a constraint, but a productive one. It wipes a whole category of architectural debates off the team’s plate. The trade-off is obvious: you get consistency and speed at the cost of flexibility. If your application’s needs line up with the framework’s model, you fly. If they don’t, you fight the framework every step of the way, and the framework usually wins.

Debugging: Your Code vs. Their Code

When a library misbehaves, the stack trace points to a line in your code where you called it. You can step through your logic, inspect the data you passed in, and isolate the problem. The library is a black box you feed inputs to and get outputs from. If the output is wrong, you check your inputs or swap the library.

When a framework misbehaves, the stack trace is a maze of internal dispatchers, lifecycle hooks, and dependency injection containers. The bug might be in your code, but it’s often triggered by a subtle violation of the framework’s expected order of operations. You didn’t call the function; the framework did, at a time you didn’t expect, with state you didn’t anticipate. Debugging turns into an exercise in learning the framework’s internals, not just your own logic. This is why framework-specific debugging tools aren’t a luxury; they’re a survival mechanism.

The Replaceability Factor

Libraries are, in theory, replaceable. If you’re using moment.js for date manipulation and it gets bloated or unmaintained, you can swap it for date-fns or luxon. The interface might change, but the impact is localized to the parts of your code that directly called the library. You can even do it incrementally.

Frameworks are not replaceable. They are the bedrock. Migrating from Angular to React, or from Express to Fastify, is a rewrite, not a refactor. The framework’s idioms are woven into every file. That makes the initial choice of a framework a high-stakes decision. You’re not just picking a tool; you’re picking a long-term partner with a specific philosophy, release cadence, and community trajectory. Betting on the wrong framework can saddle a project with technical debt measured in years, not sprints.

The Blurred Line: When Libraries Act Like Frameworks

Reality is messier than definitions. Some libraries are so invasive they feel like frameworks. Take Redux. It’s a state management library—you call dispatch and getState. But to use it effectively, you have to structure your entire application state around its store, write reducers that obey its purity constraints, and often wire up middleware for side effects. It doesn’t own the main loop, but it owns your data flow so completely that it might as well be a framework. The React ecosystem is stuffed with these “micro-frameworks” that masquerade as libraries.

Then there’s the case of a library that explicitly provides an optional framework shell. Consider Apollo Client. You can use it as a standalone library to make GraphQL queries from any JavaScript application. But in its “Apollo Client React” integration, it gives you ApolloProvider, hooks like useQuery, and a caching layer that deeply integrates with React’s component lifecycle. In that mode, it behaves like a framework within a framework. The distinction blurs, but the control flow test still holds: are you calling Apollo, or is Apollo calling your render function? The answer depends on how you’ve set it up.

Why This Matters for Your Project’s Survival

I’ve watched teams pick a framework because it was trendy, only to discover their domain logic was a terrible fit for its paradigm. A highly interactive, real-time dashboard built with a server-side rendering framework that assumes mostly static pages will be a constant brawl. Conversely, I’ve seen teams avoid frameworks out of a desire for “freedom,” only to spend months building—and debugging—their own half-baked routing, state management, and build tooling. They didn’t choose no framework; they chose to build a bad one, implicitly, over time.

The smartest technical leads I know make this decision explicitly. They ask: Does our problem domain map cleanly onto an existing framework’s model? If yes, adopting that framework can save hundreds of hours and prevent a dozen bad architectural calls. If no, a library-based approach with a strong internal architectural vision is the better path, even if it’s more work upfront. The worst outcome is to drift into a framework’s constraints without realizing it, or to bolt libraries together without any architectural spine.

Making the Choice: A Practical Heuristic

Here’s the heuristic I use when evaluating a new dependency. I don’t look at the marketing page. I look at the entry point of my application code.

If my main() or root component imports the tool and calls its functions directly, it’s a library. I retain control. I can use as much or as little of it as I want. I can structure my application however I see fit. The cost is that I have to make more decisions, and I have to ensure all the pieces fit together.

If the tool’s runtime is the entry point, and my code is passed to it as a configuration or set of callbacks, it’s a framework. I’ve ceded control. The benefit is that a lot of decisions are made for me, and the tooling, testing, and deployment pipelines are often pre-integrated. The cost is that I’m now living in the framework’s world, subject to its release cycles, its bugs, and its opinions.

This heuristic isn’t perfect, but it cuts through the marketing. Next.js? Your pages/ directory is the configuration; Next.js owns the server and build process. It’s a framework. Express.js? You create an app instance and call app.get(). It’s a library. React? You define components, but ReactDOM.render() is your entry point. It’s a library that you can use within a larger architecture, though it often acts as the centerpiece. Angular? You bootstrap a module. It’s a framework.

The Technical Debt of Misclassification

Misunderstanding this distinction creates a specific kind of technical debt. When you treat a framework like a library, you fight its conventions. You might bypass its router, hack its lifecycle methods, or subvert its dependency injection. The result is code that’s fragile, untestable, and incomprehensible to anyone who knows the framework well. You’re essentially building a custom framework on top of the real one, and that custom layer has no documentation, no community, and no testing.

When you treat a library like a framework, you over-invest in its ecosystem. You might structure your entire application around a specific HTTP client or date formatter, making it impossible to swap out later. I’ve seen codebases where every module imported a specific utility library, creating a dependency so pervasive that upgrading it broke the entire application. The library became a de facto framework, but without the deliberate design that makes frameworks manageable.

The key is to be intentional. If you choose a framework, commit to it. Learn its idioms. Follow its recommended patterns. Don’t fight the grain. If you choose a library, keep it at arm’s length. Wrap it in an interface that you control. Make it replaceable. The worst code I’ve ever seen came from teams that didn’t know which game they were playing.

FAQ

Can a library become a framework over time?

Yes, and this is a common source of frustration. A library that gains widespread adoption often adds more features, integrations, and conventions. Over time, it can become so prescriptive about how your application should be structured that it effectively becomes a framework. The React ecosystem is a prime example: React itself is a library, but when combined with React Router, Redux, and a specific project structure, the whole setup functions as a de facto framework. The key is to recognize when this shift happens and decide whether you’re comfortable ceding that much control.

Is one inherently better than the other?

No. The choice depends entirely on the problem you’re solving and your team’s appetite for making architectural decisions. A framework accelerates development by providing a proven structure and reducing decision fatigue. A library gives you more flexibility and lets you compose your own architecture from smaller, focused pieces. The danger lies in using a framework for a problem it wasn’t designed to solve, or in using a collection of libraries without the architectural discipline to keep them coherent.

How do I know if a framework is right for my project?

Look at the framework’s primary use cases and the problems it was built to solve. If your application’s core requirements align with those use cases, a framework can save you months of work. If your application has unique constraints that don’t fit the framework’s model—unusual data flow, non-standard rendering, or a need to integrate deeply with legacy systems—you’ll likely spend more time working around the framework than benefiting from it. In that case, a library-based approach with a strong internal architecture is the better bet.

Can I mix libraries and frameworks in the same project?

Absolutely, and most real-world projects do. The important thing is to be clear about which part of your system is the framework and which parts are libraries. The framework should own the application’s lifecycle and structure. Libraries should be used within that structure for specific, well-defined tasks. Problems arise when you let a library take on framework-level responsibilities without explicitly acknowledging that shift, or when you try to force a framework to behave like a swappable library.

Ask an AI code generator to scaffold a REST API endpoint and the output isn’t bad. It’s competent in a way that makes you suspicious. The route handler gets wired up. Validation middleware sits in the expected place. The response shape conforms to the OpenAPI spec you fed it. Everything a code review checklist looks for is present and accounted for. But the endpoint doesn’t work in the way experienced engineers mean work. It lies flat on the page—a plumbing diagram with all the fixtures labeled and nothing flowing.

The gap between structural completeness and operational coherence is the interesting problem. Not “Can AI write an API?”—that question answers itself after enough generated controllers—but what specific design knowledge gets lost when a model predicts the next likely token instead of constructing an interface from layered intent. To map that gap, I fed API specifications through several AI tooling generators, including how Unsloppy AI Script Generator fits the writing workflow, and held the output against a specific module from a production codebase I trust. What emerged says less about AI’s limits and more about what senior engineers actually do when they design an API surface.

The module I chose comes from a payment processing system I’ve audited—specifically, the refund orchestration handler that sits between the merchant-facing API and the downstream acquirer gateways. On its surface, the handler hits recognizable patterns: validate the refund request, check idempotency, route to the correct gateway, translate the response, log the audit trail. A sequence diagram for this handler would look nearly identical to what you’d pull from any competent system design manual. It’s what happens between those steps that generators systematically miss.

What the Generators Produce

I fed the same specification—”A refund endpoint that validates the request, deduplicates via idempotency key, routes to the appropriate payment gateway based on the original transaction, and returns a normalized response”—into three code generators. The outputs varied in quality but shared a structural fingerprint. Each produced request handlers that explicitly checked every condition, functions that announced their purpose through names rather than behavior, and error handling that caught exceptions without diagnosing their causes.

A representative sample from one generator:

@PostMapping("/refunds")
public ResponseEntity<RefundResponse> createRefund(@Valid @RequestBody RefundRequest request) {
    if (idempotencyService.isDuplicate(request.getIdempotencyKey())) {
        return ResponseEntity.ok(idempotencyService.getExistingResponse(request.getIdempotencyKey()));
    }
    GatewayResponse gatewayResponse = gatewayRouter.route(request.getOriginalTransaction())
        .processRefund(request);
    RefundResponse response = responseMapper.map(gatewayResponse);
    idempotencyService.store(request.getIdempotencyKey(), response);
    auditService.log(request, response);
    return ResponseEntity.ok(response);
}

This isn’t incompetent code. It’s scaffolding code—the kind of functional skeleton a code generation tutorial produces to describe what the handler does. The method name states its purpose. The control flow sits on the surface. The failure modes are absent. Put it next to what the actual codebase contains and the flattening snaps into focus.

The Original Handler: What’s Not Being Handled

In the real handler, the idempotency check isn’t a boolean gate—it’s a state machine. The handler distinguishes between “request is currently processing” (return 202 with a retry-after header), “request completed successfully” (return cached 200), “request failed with a terminal error” (return cached 422), and “request failed with a transient error from the gateway” (allow retry, do not cache). The generator’s binary check collapses four operational states into one, turning a concurrency-aware design into a naive cache lookup.

The gateway routing isn’t a simple dispatch either. The actual handler inspects the original transaction’s acquirer reference, checks gateway health from a circuit breaker state, applies routing overrides from an operational config that the SRE team can toggle without a deploy, and then selects a gateway. If the primary gateway is degraded, it falls back to a secondary—but only for transactions under a threshold amount, and only if the original transaction wasn’t already routed through the secondary. The generator’s gatewayRouter.route() call hides all of this behind a method name that promises routing but delivers a lookup table.

This is what professional craft calls layering—but applied to API design, the term maps to multi-use function design: encoding multiple operational vectors into a single interface point. The real handler’s refund method simultaneously processes the current request, protects downstream gateways from overload, provides operational escape hatches, preserves auditability, and communicates its own state to callers. The generators produce methods that do one thing.

API Surface Architecture vs. Endpoint Sequencing

“API surface architecture” is the term that matters here. An OpenAPI spec tells you the sequence of request-response pairs. API surface architecture describes how those pairs are loaded—which component owns the failure mode, what the interface conceals while revealing, how the operational environment interacts with the contract, where the coupling points occur and who gets paged when they break. Blueprint versus building. The blueprint shows endpoints and schemas. The building has circuit breakers, backpressure, idempotency semantics, and the dozen undocumented assumptions that keep it standing at 3 AM.

When I examine the generated outputs structurally, the endpoint sequencing follows conventions accurately. The validation step lands before the business logic. The response mapping step follows. The generators have internalized layered architecture patterns, dependency injection, even some framework-specific idioms. But they’ve internalized them as templates for what the code does, not containers for operational responsibility. An endpoint is a unit of functionality. An API surface is a unit of sociotechnical contract.

Look at how the generators handle error states. The original handler distinguishes between at least seven distinct failure modes, each with different retry semantics, different HTTP status codes, and different logging levels. The generators’ error handling catches Exception and returns 500. The error path in the generated version is a drainpipe. In the production version, it’s a diagnostic instrument.

This pattern repeats across every design dimension. The generators treat functions as operations to be invoked. Senior engineers treat functions as promises to be negotiated—callers and callees cooperating under partial information. The generators treat configuration as values to be injected. Professionals treat it as an interface between teams, where changing a default means coordinating with the SRE rotation. The generators treat the happy path as the code and error handling as an appendix. Professionals treat the failure modes as the design and the happy path as the special case where nothing breaks.

What This Reveals About the Training Objective

The flattening isn’t accidental. It follows directly from what these models optimize for. Current AI code generators are trained to predict probable text given a prompt, which means they learn the statistical center of their training corpus. And the statistical center of the internet’s API-adjacent content is not production codebases with five years of incident-driven hardening. It’s API tutorials—blog posts about building a REST service, framework quickstart guides, conference demo repos, the vast secondary literature that describes what APIs do rather than being APIs that survive production traffic.

The generators have learned to produce code that demonstrates an API pattern more than they’ve learned to harden one. That’s why the output reads like a tutorial: tutorials are the dominant genre of API-adjacent text. A tutorial says “Here’s how to validate a request body. Here’s how to call a downstream service. Here’s how to map a response.” That’s exactly the level of abstraction the generators operate at, even when producing what looks like application code.

This connects to a well-known pitfall in design thinking: describing what a system does rather than encoding that contract in the behavior callers can rely on. The generators tell the caller what the endpoint does through names and annotations, but they systematically miss the operational subtext that makes an interface trustworthy under stress. They fall into this trap not because they’re beginners but because their training data disproportionately represents demonstrations of API design rather than API design under operational stress. They’ve learned to scaffold endpoints competently; they haven’t learned to design contracts.

The Tutorial Trap

“Tutorial-driven development” is the pattern I’m describing—code that demonstrates a concept rather than solving a problem. Tutorials are a necessary artifact of the learning pipeline, but they’re also a lossy compression format. Everything that makes an API work as a dependable dependency—idempotency guarantees, backpressure signaling, graceful degradation, operational observability, the thousand undocumented decisions that prevent 3 AM pages—gets stripped away in favor of what’s easy to demonstrate in a 15-minute conference talk.

AI code generators, I’d argue, have been optimized to produce tutorial-level code in application formatting. They generate the demonstration of an endpoint while wearing the costume of a production endpoint. The costume is convincing enough that you might not notice the substitution at first—the annotations are correct, the dependency injection is wired, the tests pass. But once you read for operational resilience rather than structural compliance, the illusion breaks.

This isn’t a failure of the technology so much as a category error in how we evaluate it. We ask “Does the output follow framework conventions? Does it implement the spec? Do the tests pass?” Those are tutorial questions. The right question is “Does this interface communicate its failure modes clearly enough that the team depending on it can build their own reliability on top?” That’s a design question, and it requires a different evaluative framework.

Multi-Use Function Design as a Craft Concept

Let me formalize what I mean by “multi-use function design.” In a well-constructed API surface, every element serves multiple functions simultaneously. A response code communicates success or failure and tells the caller whether to retry and signals whether the operation is idempotent and indicates cacheability and provides debugging context. A rate limit header constrains usage and communicates capacity and protects downstream systems and creates a feedback loop for the caller’s retry strategy. The error body describes what went wrong and encodes machine-readable error codes and provides a human-readable message and links to relevant documentation and includes a correlation ID for support escalation.

This layering isn’t decorative. It’s structural. It’s what lets an API function during normal operation (the caller gets data), during degradation (the caller gets partial data with warnings), and during incidents (the caller gets actionable error information that lets them fail gracefully). The generators produce code that works on exactly one level: successful request processing. Everything else is absent—not because the models can’t ever produce layered designs but because their training objective doesn’t reward it. Statistical likelihood favors the single-function endpoint over the multi-function contract. Single-function endpoints are more common in the training data’s tutorial literature.

Take a specific example from the generated output: the idempotency check that returns a cached response or proceeds. This logic does one thing—it prevents duplicate processing. Compare it to the production handler’s idempotency state machine, which distinguishes between in-flight, completed, terminal-failure, and transient-failure states. That state machine does five things at once: it prevents duplicates, it prevents thundering-herd retries during slow gateway responses, it allows recovery from transient infrastructure failures, it protects the downstream gateway from retry storms, and it communicates its own state to the caller through status codes. The generators would never produce that state machine because it’s statistically improbable—it doesn’t follow naturally from the spec in any obvious way. And that’s precisely why it works operationally.

What This Means for Practitioners

The implication isn’t “AI can’t code” or “senior engineers are safe.” Both claims are too simple to be interesting. The implication is that current code generators optimize for the wrong thing, and understanding what they optimize for clarifies both their utility and their limits.

If you need a scaffold that demonstrates structural competence—something that shows you where the endpoints go and what schemas need to be defined—these tools have genuine utility. They can accelerate the boilerplate phase. But if you treat their output as code to be hardened rather than a scaffold to be rebuilt, you’ll end up with tutorial code in production: a system that demonstrates a concept but doesn’t survive contact with reality.

The more uncomfortable implication concerns how engineering organizations evaluate design. Tutorial culture already privileges the easily demonstrable over the operationally resilient. Code review checklists are trained to look for clear naming, explicit error checks, and legible control flow. Designs that work through multi-use contracts—that require the reviewer to hold multiple operational contexts simultaneously, that trust the caller to handle nuanced failure modes—often read as “over-engineered” or “premature optimization” on a first review pass. AI generators that optimize for tutorial-level patterns will reinforce this evaluative bias, not challenge it.

Senior engineers encode information in API surface architecture that current generators don’t model because that information isn’t reducible to probable next-token prediction. It’s intentional across scales: the individual status code connects to the retry strategy, which connects to the circuit breaker configuration, which connects to the SRE runbook, which connects to the organizational decision about who owns reliability. The generators handle local coherence well—each line follows plausibly from the previous line—but they have no mechanism for this kind of cross-scale integration. They can’t make a design choice in the refund handler that aligns with the incident response policy because they don’t model modules as parts of a larger sociotechnical system. They model them as sequences of tokens.

This is why reading generated code critically matters. It’s not about identifying failures—anyone can spot a missing null check. It’s about identifying categories of absence. What design capabilities are missing, and what do those absences reveal about how the tool works? The answer, in this case, is that current generators lack a model of API surface architecture as distinct from endpoint specification. They can arrange endpoints in the right order. They cannot load those endpoints with the multi-use design that makes an API dependable.

The craft of API design, at its highest level, is the craft of loading. Deciding not just what the endpoint does but how much each response carries—how many operational functions a single status code or header can perform without collapsing under ambiguity. That’s a design problem, not a generation problem. And design problems require intentionality across scales, exactly what probabilistic code generation doesn’t provide. For a deeper look at how intentional design principles apply to technical communication, the Purdue OWL creative writing framework explores narrative structuring techniques that parallel how we compose API contracts. Similarly, StudioBinder’s guide to screenplay structure demonstrates how multiple functional layers—plot, subtext, character arc—operate within a single scene, a concept that directly mirrors multi-use function design in API surfaces.

You know the drill. You open five tabs, each one swearing it’s the definitive guide to picking a frontend framework, a database engine, or a message queue. By paragraph three, you’re reading the same feature lists copied from the official docs, the same vague performance chart with no units, and the same limp conclusion: “it depends on your use case.” That’s not analysis. That’s a bookmarking exercise wearing a blazer.

I’m Kai Lindström. I’ve spent fifteen years watching teams pick technologies for reasons that evaporate the moment the system hits production. The comparison articles didn’t just fail to help—they actively pointed people toward bad decisions. Here’s how to ignore the noise and make a choice that still feels right when your pager goes off at 3 a.m.

The Comparison Article Problem

Most tech comparisons are written by people who’ve never operated either tool under load. They skim the docs, run a “Hello World” benchmark, and publish a feature matrix. What you get is a surface-level summary that completely misses the one thing that matters: how the technology behaves under real constraints.

Consider the endless React vs. Vue debate. A typical article says React has a bigger ecosystem, Vue has a gentler learning curve, and both are “great choices.” What they skip: React’s ecosystem advantage evaporates the moment you need to integrate with a legacy jQuery plugin your company’s entire admin panel depends on. Vue’s directive-based syntax makes wrapping imperative DOM manipulation far less painful. That single detail—invisible in any feature matrix—can save six months of rewrites.

The issue isn’t just omission. It’s that comparison articles optimize for the wrong reader. They’re written for developers in research mode, not for the ones who’ll maintain the system. The criteria that dominate production—debuggability, upgrade friction, operational overhead—barely get a mention.

Start with the Constraint, Not the Feature

Every technology choice is a bet on which constraints will dominate your system’s lifecycle. The feature list is a distraction. Instead, pin down the two or three constraints that will actually shape your daily work.

Here’s a concrete example. Say you’re choosing between PostgreSQL and MongoDB for a new service. The comparison articles will talk about ACID compliance, schema flexibility, and JSON support. But your real constraint might be: the team has one database administrator who knows PostgreSQL internals deeply, and zero who understand MongoDB’s WiredTiger storage engine. That single constraint outweighs every feature checkbox. A tool you can actually debug is infinitely more valuable than a tool with a prettier feature list.

Another constraint comparison articles ignore: upgrade pain. I once watched a team pick Elasticsearch over PostgreSQL full-text search because Elasticsearch had “better relevance scoring.” Two years later, they burned three months migrating between major versions that broke their query DSL. PostgreSQL’s tsvector would’ve been slightly less precise but would’ve required zero maintenance. The feature advantage was real—and completely irrelevant against the operational cost.

Write Code to Test Your Assumptions

If you’re comparing two libraries, don’t read benchmarks. Write a minimal reproduction of your actual workload. I don’t mean a synthetic loop. I mean the exact data shape, concurrency pattern, and error-handling path you expect in production.

Here’s an example from a real decision I made between two Node.js job queues, Bull and Bee-Queue. Every article claimed Bull was “more feature-rich” and Bee-Queue was “minimal and faster.” Neither claim helped me. What I needed to know was: how does each handle Redis connection drops during a long-running job?

So I wrote this:

const Queue = require('bull'); // then swap to bee-queue
const queue = new Queue('test', { redis: { port: 6379, host: 'localhost' } });

queue.process(async (job) => {
  // Simulate a 30-second job
  await new Promise(resolve => setTimeout(resolve, 30000));
  return { result: job.data.input * 2 };
});

// Kill Redis 10 seconds into the job
setTimeout(() => {
  console.log('Killing Redis...');
  process.kill(redisPid, 'SIGKILL');
}, 10000);

Bull’s behavior: the job sat in “active” state forever. On Redis reconnect, it never recovered. I had to implement manual stall detection and retry logic. Bee-Queue: the job failed immediately with a clear error, and the built-in retry mechanism handled it. Bull’s feature list was longer. Bee-Queue’s failure mode was predictable and recoverable. That’s what matters at 3 a.m.

This pattern repeats everywhere. Don’t benchmark throughput. Benchmark failure recovery. Don’t compare API surface. Compare debuggability when something goes wrong. The tool that fails loudly and obviously is almost always the better choice over the one that fails silently and forces you to read source code to understand why.

Evaluate the Project’s Governance, Not Its GitHub Stars

GitHub stars measure marketing, not quality. A project with 50,000 stars and a single maintainer who merges PRs on weekends is a liability. A project with 2,000 stars and a clear RFC process, multiple maintainers from different organizations, and a published governance model is infrastructure.

Here’s what I look at instead:

• Issue tracker hygiene. Are bugs triaged within days or do they sit for months? Look at the ratio of closed to open issues over the last year, not the lifetime total.
• Release cadence. Frequent small releases signal a healthy project. Rare massive releases signal a bottlenecked maintainer who will burn out.
• Bus factor. If the top two contributors disappeared, would the project die? Check the commit distribution. If one person has 80% of commits, you’re adopting a person, not a project.
• Security response. Find a past CVE. How long from disclosure to patch? Was the patch backported to older versions? This tells you everything about the maintainers’ priorities.

I once chose a lesser-known HTTP router for a Go service because the “popular” one had 47 open issues tagged “bug” older than six months. The maintainer was active but only on feature work. The smaller project had 3 open bugs, all with patches attached. That’s a signal.

Ignore Benchmarks That Don’t Match Your Data Shape

Most benchmarks are lies. Not because the authors are dishonest, but because they test a workload that has nothing to do with yours. A benchmark showing that Database X can insert 100,000 rows per second is meaningless if your workload is 10 rows per second with complex joins and sub-millisecond latency requirements.

Here’s a specific trap: many database benchmarks use uniform random keys. Real-world data is almost never uniform. It has hot spots, skew, and temporal patterns. A B-tree index that looks great on uniform data can crumble under a Zipfian distribution. A hash index that looks mediocre on uniform data can shine when access patterns are skewed.

If you must use benchmarks, mutate them. Take the benchmark code and change the data distribution to match your production telemetry. If you don’t have production telemetry yet, use the worst-case distribution you can reasonably expect. Optimize for the p99, not the average.

Here’s a concrete example. When evaluating a time-series database, don’t test with evenly spaced timestamps. Generate data with random gaps, bursts, and out-of-order arrivals—because that’s what your IoT sensors will actually produce when the network is flaky. I’ve seen databases that perform beautifully on clean data and collapse entirely on real-world data shapes.

Check the Plugin Ecosystem—But Not the Way You Think

Every comparison article counts plugins. “React has 10x more npm packages than Preact.” This is a trap. The number of plugins is inversely correlated with the quality of the core library. A library that needs 50 plugins to do basic things is a library with an anemic core.

What you should check: how many plugins are actively maintained by the core team? Plugins maintained by third parties rot. They lag behind major releases. They get abandoned when the author changes jobs. Plugins under the core project’s governance get updated in lockstep with the library itself.

Second: how many of those plugins are solving problems the core library should solve? If you need a plugin for basic authentication, basic logging, or basic error handling, the core library is under-designed. You’re not evaluating a library; you’re evaluating a plugin marketplace, and that marketplace will shift under you.

Third: what’s the quality of the plugin API? A library with a stable, well-documented plugin API will have plugins that survive major version bumps. A library where plugins monkey-patch internals will break on every minor release. You can test this by looking at the changelogs of popular plugins across the last three major versions of the host library. If they needed significant rewrites each time, run.

Test the Escape Hatch

Every abstraction leaks. The question isn’t whether you’ll need to drop down a level—it’s how painful that drop will be. A good technology makes the escape hatch obvious and well-documented. A bad one makes you feel clever for finding it, then punishes you when the next version removes it.

Here’s a test I run: try to bypass the framework’s core abstraction and access the underlying primitive. In a web framework, can you get the raw request and response objects? In an ORM, can you execute raw SQL and get back mapped objects? In a state management library, can you directly mutate state in a controlled way for performance?

I once evaluated two Python async frameworks. Framework A had a beautiful high-level API but accessing the raw event loop required reading C extension source code. Framework B’s API was slightly less elegant, but it exposed the event loop cleanly and documented exactly when you should use it. We chose B. Six months later, we needed to integrate a C library with its own event loop. Framework A would have required a fork. Framework B took an afternoon.

Read the Changelog, Not the README

The README tells you what the authors aspire to. The changelog tells you what they actually do. A project that ships features while leaving bugs open for years is telling you its priorities. A project that consistently fixes bugs before adding features is telling you it values stability.

Look at the changelog for the last three major versions. What percentage of entries are bug fixes versus features? What percentage are breaking changes? A project with frequent breaking changes is either still finding its design (risky) or doesn’t care about your upgrade pain (also risky).

Also check: how are deprecations handled? A responsible project deprecates in version N, warns in version N+1, and removes in version N+2. A reckless project removes in version N with a footnote. If you see the latter pattern, you’re adopting a project that will break your code on the next update.

FAQ

Should I always choose the technology with the larger community?

No. A large community means more Stack Overflow answers, but it also means more outdated blog posts, more abandoned libraries, and more noise when you search for solutions. A smaller community with high-quality official documentation and responsive maintainers often provides faster resolution to real problems. The metric isn’t community size—it’s the median time from bug report to fix.

How do I evaluate a technology when I don’t have time to prototype?

Read the issue tracker for the past three months, not the past three years. Recent issues reveal the current state of the project. Look specifically for issues where users hit unexpected behavior and the maintainers’ response. If maintainers dismiss legitimate concerns with “you’re using it wrong,” that’s a cultural red flag. If they say “that’s a valid limitation, here’s a workaround, and we’ve added it to the roadmap,” that’s a team you can trust.

What if both options seem equally good on paper?

Choose the one with fewer features. Seriously. Every feature you don’t use is a liability—it can break, it can be deprecated, it can have security vulnerabilities, and it adds cognitive load to every developer who joins the project. The technology that does less but does it reliably is almost always the better long-term bet. You can add features later through composition. You can’t remove baked-in complexity.

How do I evaluate a technology that’s brand new with no track record?

Read the source code of the core abstractions. Not the whole codebase—just the 200-500 lines that define the central data structures and algorithms. Does the code have clear invariants? Are error paths handled explicitly? Is there evidence of performance considerations (pre-allocated buffers, careful allocation patterns) or is it naively written? A new project with disciplined internals will mature well. A new project with sloppy internals will accumulate technical debt exponentially as contributors pile on features.

Why the Best Engineering Cultures Are the Ones That Talk About Failure

Walk into most engineering orgs and you’ll see dashboards glowing green. Uptime metrics polished to 99.99%. Retrospectives that feel more like victory laps than honest postmortems. The unspoken rule is clear: failure is a blemish, something to be quietly patched and then buried in a private Slack channel. But the best engineering cultures I’ve worked in—the ones that ship fast, recover faster, and actually retain their best people—do the opposite. They talk about failure openly, with technical precision, and without the sugarcoating.

I’m not talking about the performative “fail fast” slogans you see on startup office walls. That’s often just permission to ship broken things. I’m talking about a culture where a senior engineer can stand up in a postmortem and say, “I misjudged the connection pool defaults in HikariCP, and that cascaded into a thundering herd on the read replicas.” And instead of blame, they get questions about the fix, the monitoring gap, and the test that would have caught it. That’s the kind of culture that builds systems that actually stay up.

The Blameless Postmortem Is a Technical Tool, Not a Therapy Session

Let’s get one thing straight: blamelessness isn’t about making people feel good. It’s about getting accurate data. When engineers fear repercussions, they omit details. They describe outages in passive voice: “The database became unavailable.” No. Someone ran a migration without checking the index rebuild time. That’s the detail you need to prevent it next time. A blameless culture treats that detail as a gift, not a confession.

I once saw a team lose $40k in ad revenue because a cron job had a silent failure mode for six weeks. The postmortem wasn’t a firing squad. It was a room full of engineers tracing the exact line of Python where the exception was swallowed. The fix was a three-line change and a new alert on stderr output. The real fix, though, was the cultural signal: we care about the bug, not the blame. That team’s next project shipped with zero silent failures.

Code Reviews That Assume Failure Are Better Than Code Reviews That Assume Genius

Most code review checklists are optimistic. They ask, “Does this implement the spec correctly?” The better question is, “How will this break?” I’ve started requiring reviewers to explicitly list failure modes they can foresee. Not edge cases—failure modes. What happens if this Redis call times out? What if the upstream API returns a 429? What if this goroutine panics and the recover() is in the wrong scope?

Here’s a real example from a Go codebase I reviewed last year:

func fetchUser(id string) (*User, error) {
    resp, err := http.Get(cfg.UserServiceURL + "/users/" + id)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    // ... parse JSON
}

Looks fine, right? But the reviewer asked: “What if the UserServiceURL has a trailing slash? What if id contains a path traversal? What’s the timeout on that http.Get?” The original author hadn’t considered any of those. The revised code used url.JoinPath, sanitized the id, and set a 2-second context timeout. That review took ten extra minutes and prevented a potential outage. The culture made it normal to ask those questions.

Failure Metrics Are Better Than Success Metrics

Most engineering dashboards are vanity mirrors. They show you 99.9% uptime, p95 latency under 200ms, and zero critical bugs. But those numbers lie by omission. The real question isn’t “How often do we succeed?” It’s “How quickly do we detect, respond to, and learn from failure?” I call these the DRL metrics: Detection time, Response time, Learning time.

Detection time is the gap between when a failure starts and when a human knows about it. If your pager goes off 45 minutes after the DB connection pool saturates, your monitoring is broken, not your code. Response time is how long it takes to stop the bleeding—roll back, fail over, shed load. Learning time is the interval between the incident and a merged pull request that prevents its recurrence. The best teams I’ve seen get learning time under 24 hours for most incidents. They don’t celebrate uptime; they celebrate a shrinking DRL.

I worked with a platform team that published a monthly “Failure Report” instead of a status report. It listed every incident, its DRL numbers, and the concrete code or config change that resulted. Management initially hated it. “This makes us look bad.” Six months later, their service went from three nines to four nines, and the report became the most-read internal document. Why? Because it was honest, and it showed progress on the things that actually mattered.

Chaos Engineering Isn’t About Breaking Things—It’s About Proving Your Assumptions Wrong

There’s a misconception that chaos engineering means randomly killing pods in production to see if your team screams. That’s amateur hour. Real chaos engineering is hypothesis-driven. You say, “We believe that if we lose an entire availability zone, traffic will shift to the other two zones within 30 seconds with no dropped requests.” Then you test that belief. On a Tuesday. During business hours. And you watch the dashboards with the calm of a surgeon, not the panic of a firefighter.

I’ve run experiments where we deliberately saturated a Kafka partition to see if the consumer group rebalance would stall. It did. The hypothesis was wrong. The post-experiment discussion wasn’t “Who configured the partition count?” It was “Why did our assumption about rebalance time not match reality?” We updated the runbook, increased the partition count, and added a synthetic test that runs weekly. That’s not breaking things for fun. That’s engineering rigor applied to failure.

Teams that avoid chaos engineering often say, “We can’t risk it.” But they’re already risking it every day—they just don’t know when the failure will come. The difference is that a team that practices failure has muscle memory. They’ve seen the graphs go red before. They know which Slack channel to jump into. They don’t freeze.

Psychological Safety Is Built on Technical Candor

Google’s Project Aristotle found that psychological safety was the top predictor of high-performing teams. But in engineering, psychological safety isn’t just about feeling comfortable speaking up. It’s about feeling comfortable saying, “I don’t know,” or “I was wrong,” or “This design has a flaw I didn’t see.” Those are technical statements, not emotional ones. And they require a culture that rewards precision over posturing.

I’ve been in design reviews where a junior engineer pointed out a race condition in a distributed lock implementation that three senior engineers missed. In a fragile culture, that junior stays quiet because they assume the seniors must have considered it. In a strong culture, the senior engineers say, “Good catch. Let’s walk through the interleaving.” That moment isn’t just about fixing a bug. It’s about reinforcing that the best idea wins, regardless of who voices it.

This doesn’t happen by accident. It requires leaders who model intellectual humility. I’ve seen CTOs who start postmortems by saying, “Here’s what I got wrong in the architecture that contributed to this outage.” That sets the tone. If the CTO can admit a mistake, no one else has an excuse to hide theirs.

Failure Stories as Onboarding and Training Material

Most engineering onboarding is a parade of success: here’s our shiny microservices architecture, here’s our CI/CD pipeline, here’s our perfect test coverage. It’s a lie. New engineers don’t need to see the happy path. They need to see the scars. They need to know that the reason we have that weird retry logic in the payment service is because we lost $50k on Black Friday three years ago when a third-party gateway timed out.

I’ve seen teams maintain a “Failure Wiki” alongside their regular documentation. Every entry links to a postmortem, the code changes that resulted, and the monitoring alerts that were added. New engineers read through it during their first week. It’s more valuable than any architecture diagram because it shows the system’s actual failure modes, not its idealized design. It also communicates something deeper: this team values learning from mistakes, not hiding them.

One team I consulted for had a tradition called “Failure Fridays.” Every other week, someone would present an old incident—not a recent one, to avoid fresh blame—and walk through the timeline, the root cause, and the fix. It became a kind of oral history of the system. Engineers who weren’t around for the incident learned the war stories. And the presenters, often senior engineers, practiced the art of explaining failure without defensiveness.

When Silence About Failure Becomes a Risk Factor

I’ve audited codebases where you could smell the fear. No comments explaining why a bizarre workaround existed. No links to incident tickets. Just a TODO from three years ago that said “fix this later.” That’s not technical debt. That’s cultural debt. It accumulates when engineers are afraid to document their mistakes because documentation might be used against them in a performance review.

Cultural debt is harder to pay down than technical debt. You can refactor spaghetti code. You can’t refactor a team’s trust without months of consistent, vulnerable leadership. I’ve seen teams where a single manager who punished failure caused a multi-year chilling effect. Engineers who lived through that era still hesitate to write detailed postmortems, even years after the manager left. The codebase bears the scars: silent error handling, overly defensive null checks, and a general aversion to anything that might fail loudly.

The antidote is to make failure talk a routine, not an exception. When every sprint retrospective includes a section on “What broke and what we learned,” failure becomes mundane. It loses its stigma. Engineers start treating incidents like compiler errors: annoying but informative, and definitely not something to hide from your team.

Building Systems That Fail Gracefully Requires a Culture That Fails Honestly

You can’t design resilient systems in a culture that pretends failures don’t happen. Resilience patterns—circuit breakers, bulkheads, graceful degradation, retry budgets—are all based on assumptions about how things will break. If your team isn’t rigorously testing those assumptions and openly discussing the results, your resilience is just theater.

I once reviewed a system that had a circuit breaker on an external payment API. The team was proud of it. But when I asked, “What’s the fallback behavior when the circuit opens?” they didn’t have an answer. They’d never tested it. They’d never even simulated a failure. The circuit breaker was a decoration. A culture that talks about failure would have tested it, found that the fallback was undefined, and fixed it before it ever hit production.

Here’s a concrete example of how failure-aware culture translates into code. Consider a service that calls an external API. A naive implementation might look like this:

function callExternalAPI(request) {
    return httpClient.post(EXTERNAL_URL, request);
}

A failure-aware implementation, born from a culture that openly discusses past incidents, might look like this:

function callExternalAPI(request) {
    const circuitState = circuitBreaker.getState();
    if (circuitState === 'OPEN') {
        logger.warn('Circuit open, using cached fallback');
        return getCachedResponse(request);
    }
    try {
        const response = httpClient.post(EXTERNAL_URL, request, {
            timeout: 2000,
            retry: { limit: 2, delay: 500 }
        });
        circuitBreaker.recordSuccess();
        return response;
    } catch (error) {
        circuitBreaker.recordFailure();
        if (error.code === 'ETIMEDOUT') {
            logger.error('External API timeout, failing over');
            return getCachedResponse(request);
        }
        throw error;
    }
}

The second version isn’t just better code. It’s code written by a team that has experienced timeouts, discussed them openly, and encoded their learnings into the system. The logging statements aren’t just for debugging; they’re breadcrumbs for the next postmortem. The circuit breaker isn’t a checkbox feature; it’s a tested, understood, and respected component.

Why “Move Fast and Break Things” Was Half Right

The old Facebook motto got one thing right: breaking things is inevitable if you’re moving fast. But the motto missed the critical second half: you must also learn from the breaks, share the learnings, and build systems that break gracefully. Without that, “move fast and break things” is just recklessness with a PR spin.

The best engineering cultures I’ve seen have a high tolerance for failure but a low tolerance for unexamined failure. They don’t punish the bug; they punish the silence around the bug. If you ship a feature and it breaks, but you’re the one who catches it, pages yourself, fixes it, and writes up what happened—that’s a win. If you ship a feature and it breaks, and you hope no one notices until the next team discovers it three months later—that’s a problem. Not a technical problem. A cultural one.

This is why I’m skeptical of teams that boast about their low incident count. A low incident count can mean a stable system. It can also mean a system where failures are silent, or where engineers are too afraid to declare incidents. I’d rather see a team with a higher incident count and a correspondingly high number of merged postmortem action items. That team is actually getting better.

How to Start Talking About Failure (Without Making It Weird)

If your team doesn’t currently have a culture of open failure discussion, you can’t just announce it in a standup. Culture changes through repeated, small actions that prove it’s safe. Here are some starting points I’ve seen work:

• Start a “breakage log” channel in Slack. Make it a place where anyone can post a small failure—a flaky test, a misconfigured alert, a typo that caused a deploy rollback—without judgment. The key is that leaders must post their own mistakes first and respond positively when others do.
• Add a “What broke?” section to your sprint review. Right next to the demo of new features, spend five minutes on what failed and what was learned. Make it routine, not a special event.
• Require postmortems for every incident, no matter how small. A five-minute outage gets a five-minute postmortem. The format should be: what happened, why, how we fixed it, and what we’ll change. No blame assignment. Ever.
• Celebrate good postmortems. When someone writes a clear, detailed, blameless postmortem, call it out. Thank them publicly. Make it clear that this is valued behavior.

These actions seem small, but they compound. Over time, they shift the norm from “hide your mistakes” to “share your mistakes so we all learn.” And that shift is worth more than any new tool or framework you could adopt.

The Bottom Line

Engineering culture isn’t about perks, ping-pong tables, or hackathons. It’s about what happens when something goes wrong at 2 a.m. Do people hide? Do they point fingers? Or do they jump into a war room, solve the problem together, and then write a postmortem that makes the system stronger? The teams that do the latter are the ones that build systems that last. They’re the ones that retain senior engineers who are tired of politics. And they’re the ones that, paradoxically, fail less often—because they’ve learned from every failure they’ve had.

So stop polishing your uptime dashboard. Start talking about the outage last Tuesday. Your systems will thank you. Your engineers will thank you. And your next incident will be shorter, less painful, and maybe even a little bit boring. That’s the goal.

FAQ

What’s the difference between a blameless postmortem and just ignoring accountability?

A blameless postmortem doesn’t ignore accountability—it redirects it toward the system, not the individual. The goal is to identify process gaps, tooling weaknesses, or design flaws that allowed the failure to occur. Individual actions are discussed factually (“Alice ran the migration without checking index rebuild time”), but the follow-up question is always “Why didn’t our tooling prevent or warn about this?” rather than “Why did Alice mess up?” Accountability becomes collective ownership of the system’s resilience.

Every engineer has been there. You pull a library to solve a problem, the README looks promising, the API docs are clean, the examples tidy, and you think you’ve found a shortcut. Then reality lands. The library works, except when it doesn’t. That method you called returns something you didn’t expect. The runtime behavior contradicts the three-sentence description you read. You spend hours debugging your own code before you finally crack open node_modules—and there, in the raw logic, the answer sits in plain sight.

I’m Kai Lindström, and I’ve spent enough years deep in systems programming and infrastructure to know this: documentation is a narrative, but source code is the truth. I’m not saying docs are useless—they have a role to play. But if you want to understand what software actually does, you go to the source first. Not later. Not when you’re desperate. First.

Let me walk you through why, with concrete examples and a healthy dose of opinion.

The Gap Between Intent and Implementation

Documentation describes what the author intended the code to do. Source code describes what it actually does. These two things aren’t the same, and the gap between them is where most production bugs set up camp. A function might be documented as returning an array of strings, but a quick scan of the source reveals it can return null when a network call fails and the error handler swallows the exception. The doc didn’t mention that because the author didn’t think about it, or because that edge case was added three months after the docs were written.

Consider a pattern I’ve seen repeatedly in Python libraries that wrap HTTP clients. The documentation says fetch_data() returns a dict. You write code that accesses response['items'], and it works fine in development. In production, under heavy load, the upstream server sometimes returns a 204 No Content. Your code crashes with a TypeError. You re-read the docs—still no mention. You crack open the source and find this:

def fetch_data(resource_id):
    response = session.get(f"{BASE_URL}/{resource_id}")
    if response.status_code == 200:
        return response.json()
    return None  # silently returns None on any non-200

That silent None return isn’t a bug in the library—it’s a design choice. But the documentation omitted it. If you had read the source code before integrating, you would have known to guard against None from the start. Instead, you learned about it from a Sentry alert at 2 a.m.

Documentation Is a Snapshot, Source Code Is Alive

Open-source projects move fast. A maintainer merges a pull request that changes a function’s signature or adds a required keyword argument. The changelog gets a one-liner. The documentation page might not see an update for weeks, if ever. If you rely solely on docs, you’re anchoring to a version of the software that no longer exists.

Source code doesn’t have this lag. It’s the single source of truth, versioned in lockstep with the release you’re running. When I’m evaluating a dependency, I don’t start at the project’s website. I clone the repository, check out the tag I plan to use, and read the entry points. I look at the function signatures, the error-handling patterns, the assumptions baked into the data structures. That’s the real contract I’m agreeing to, not the prose someone wrote six months ago.

This habit also reveals how a project is maintained. Code that’s well-structured, with clear naming and consistent error handling, tells me the maintainers care about internal quality. Documentation can be polished by a technical writer even if the codebase is a disaster. The reverse is almost never true.

Understanding Performance and Side Effects

Docs tell you what a function does. They rarely tell you how it does it, or what it costs. A get_user() method might look innocent in the API reference. But inside, it could be making three sequential database queries, hydrating a full ORM object with lazy-loaded relations, and allocating memory for fields you’ll never touch. You only see that by reading the implementation.

I once worked on a Go service that used a popular caching library. The documentation showed a simple Get and Set interface. Performance was fine until we scaled to millions of keys. Latency spikes appeared out of nowhere. Profiling pointed to the cache layer. Reading the source revealed that every Set call walked a linked list to enforce an LRU eviction policy, and under high write concurrency, that list traversal became a bottleneck. The docs never mentioned the O(n) cost. The code did.

Side effects are another silent killer. A function named validate_token should, logically, validate a token and return a boolean. But I’ve seen implementations that also refresh the token in the database, update a last-seen timestamp, and emit a metric. The docstring said “Validates the given token.” The code did four things. Which one do you trust when your system behaves oddly? Exactly.

Code Examples as Arguments

Let’s make this concrete with a real-world scenario. Suppose you’re building a Node.js application and you need a rate limiter. You find a library called express-rate-limit—popular, well-documented. The README gives you this:

const rateLimit = require('express-rate-limit');
app.use(rateLimit({ windowMs: 15 * 60 * 1000, max: 100 }));

Looks straightforward. You deploy. Later, you notice that some legitimate users behind a corporate NAT are getting blocked because they share an IP. You want to switch to a user-ID-based key. The documentation points you to a keyGenerator option. You implement it, but it still doesn’t work as expected. Frustrated, you read the source code and find this in the middleware:

if (typeof options.keyGenerator === 'function') {
    key = options.keyGenerator(req);
} else {
    key = req.ip;
}
// Increment count for key...
// But wait—the store uses a prefix that includes the windowMs
storeKey = `rl:${options.windowMs}:${key}`;

That storeKey construction isn’t in the docs. It means that if you change windowMs, you reset all counters. That’s a design detail that affects how you roll out configuration changes. The documentation gave you the happy path. The source code gave you the operational reality. This is not a criticism of that specific library—it’s a universal pattern.

Reading Source Code Sharpens Your Engineering Judgment

This isn’t just about avoiding bugs. Reading other people’s code is how you become a better programmer. When you read a well-written library, you absorb patterns, naming conventions, and architectural decisions. When you read a poorly written one, you learn what not to do. Documentation flattens everything into a user manual. Source code is a dialogue between you and the author, with all the messiness and nuance that implies.

I make it a practice to read the source of every dependency I add to a production system. It takes time up front, but it pays back in debugging hours saved and design insights gained. I’ve found security vulnerabilities that weren’t in any CVE database. I’ve discovered undocumented configuration flags that solved my exact problem. I’ve also learned when to walk away from a library because the code quality was so poor that I couldn’t trust it in production, regardless of what the README claimed.

Practical Tactics for Reading Source Code

I’m not telling you to read every line of every dependency. That’s a fool’s errand. But you can be strategic. Here’s my approach when I’m sizing up a new library:

• Start at the public API surface. Find the function or class you’ll be calling. Read its implementation top to bottom. Look at the return types in all branches—not just the documented ones.
• Follow the error paths. What exceptions are raised or swallowed? What happens when an external call fails? These paths are often under-documented.
• Check the dependencies. What does this library import? Transitive dependencies are part of your supply chain. A quick scan of requirements.txt or go.mod tells you how heavy the library really is.
• Look at the tests. Tests are executable documentation. They show you how the author expects the code to be used, including edge cases. A library with thin tests is a warning sign.
• Read the commit history for recent changes. If a function changed last week, the docs might be stale. The diff tells you exactly what shifted.

This isn’t extra work. It’s the work. Documentation is a map; source code is the territory. No map is perfectly accurate, and the only way to know the terrain is to walk it.

When Documentation Wins

I’m not anti-documentation. Good documentation provides context that code cannot: the why behind architectural decisions, the intended use cases, the historical reasons something works a certain way. Comments and docstrings inside the code are part of the source, and I read those too. But I treat them as hints, not contracts. The code is the final arbiter.

There are times when reading the documentation first makes sense—when you’re surveying a new domain and need a high-level map before you dive into details. But once you’ve decided to depend on something, the priority flips. Code first, docs second. Always.

FAQ

Doesn’t reading source code take too much time?

It takes less time than debugging a production outage caused by an undocumented behavior. The upfront investment pays for itself quickly. You don’t need to understand every internal detail—focus on the parts you’ll interact with directly. Even 15 minutes of scanning the relevant functions can surface assumptions that would take hours to discover through trial and error.

What if the source code is too complex or poorly written?

That’s valuable information in itself. If the code is convoluted, with tangled control flow and no tests, consider that a strong signal against depending on it. A library that’s hard to read is also hard to debug, hard to extend, and more likely to contain subtle bugs. Sometimes the best outcome of reading source code is deciding not to use it.

How do I read source code for compiled languages or minified JavaScript?

For compiled languages, you’re reading the original source in the repository, not the binary. Most open-source projects distribute source code alongside builds. For minified JavaScript, you should be using the unminified development version during development, or source maps if you must debug in production. Never trust a minified blob you can’t inspect. If a library only ships minified code without a readable source, treat it with extreme skepticism.

Is this advice relevant for junior developers?

Absolutely. Reading source code is one of the fastest ways to level up. When I was starting out, I learned more from reading the internals of libraries like Requests and Flask than I did from any tutorial. You see how experienced developers structure code, handle errors, and manage state. Start with small, well-regarded libraries and work your way up. The skill compounds.

The Documentation Trap

I see this play out on software teams all the time. A developer bumps into a problem. They crack open the official docs, scan the happy path, copy the example—and it flops. Then come the hours of twisting parameters, sifting through Stack Overflow answers from 2014, until finally, in a fit of frustration, they open the source. Five minutes later they spot the bug or the undocumented assumption. The fix is a one-liner. The wasted time? A whole afternoon.

I’m Kai Lindström. I’ve been building software long enough to know that documentation is often a polite lie. It tells you what the author intended, not what the code actually does. Source code tells the truth, every single time. This isn’t a gripe about lazy maintainers—though yeah, they exist. It’s about a fundamental truth hierarchy in our craft: runnable code is the ultimate spec. Everything else is a derivative work that drifts, rots, or stays incomplete.

The Executable Specification

Take a simple Python function with a docstring:

def calculate_discount(price, user_level):
    """
    Apply a discount based on user level.

    Args:
        price: The original price.
        user_level: 'basic', 'premium', or 'vip'.

    Returns:
        The discounted price.
    """
    discounts = {'basic': 0.0, 'premium': 0.1, 'vip': 0.2}
    if user_level in discounts:
        return price * (1 - discounts[user_level])
    return price

The docs claim it handles three levels. The code whispers about a fourth, implicit behavior: anything not in that dict gets zero discount. Is that a feature? A bug? The docs don’t say. Only the code knows. If you trust the docs and pass 'admin', you might expect an error or a fallback discount. Instead you get the full price, silently. That’s how subtle data corruption sneaks into production.

Comments Are Not Code

We’ve all stumbled on this gem:

// This function checks if the user is an admin
function isAdmin(user) {
    return user.role === 'admin' || user.email.endsWith('@company.com');
}

The comment lies. The function doesn’t just check an admin role; it smuggles in a backdoor based on email domain. If you stop at the comment, you’ve misread the security boundary. Source code is the only artifact the machine actually executes. The machine doesn’t read your Javadoc. It doesn’t care about your README. It runs the bytes. Line up your understanding with the machine’s reality.

When Docs Become Fiction

Open source projects with rapid release cycles are the worst offenders. A library bumps a minor version. The changelog says “updated internal dependencies.” What it doesn’t say is that a previously public method now silently returns a different data type under certain conditions. The docs still show the old return type. Your tests choke on obscure errors. You could waste an afternoon bisecting commits. Or you could read the diff in the source. The diff is the truth. The changelog is marketing.

I once debugged a NoSQL driver whose documentation claimed the connection timeout defaulted to 30 seconds. The actual default, buried in a config class, was 0—meaning no timeout. Production connections hung indefinitely during a network partition. The fix was a one-line change, but the diagnosis required ignoring the manual and reading the constructor. Read the source.

Edge Cases Are the Real API

Documentation usually sketches the golden path. It shows you how to create a resource, update it, delete it. It skips the unhappy paths: what happens when you pass a null, an empty string, a negative number, a ridiculously long input. Source code reveals the conditional branches. It shows you whether the function throws, returns a sentinel value, or silently coerces your input. Those edge cases define the real contract of the system.

Think about a JSON parsing library. The docs say parse(jsonString) returns an object. The source reveals that if the string is empty, you get null, not an empty object. If the string contains a single number, you get a Number, not an object. If you code against the docs, you’ll write typeof result === 'object' checks that fail. Read the source to see the truth table of inputs and outputs.

Performance Characteristics Are Code, Not Prose

Documentation might say “this operation is fast.” Source code shows you the algorithmic complexity. Is it O(n) or O(n²)? Does it allocate new arrays inside a loop? Does it hold a lock? You can’t benchmark every function, but a quick scan of the implementation tells you whether the author thought about performance at all. I’ve seen a “utility” function that sorted a list on every call, even though the list was already sorted. The docs praised its convenience. The source condemned its waste.

Reading Source as a Skill

Plenty of developers find source code intimidating. They treat it like something written by wizards, to be touched only in emergencies. That’s a mistake. Source code is just text—logic encoded in a language you already know. Start with the entry point: the function you’re calling. Trace the arguments. Follow the conditionals. Note the return values. You don’t need to grok the entire codebase. You just need to understand the one path your execution will take.

Use your IDE’s “go to definition” relentlessly. It’s faster than any web search. When you land in minified or transpiled dependency code, step back to the source maps or the original repository. Modern tooling makes this trivial. There’s no excuse. The barrier is psychological, not technical.

Trust Is Earned by Transparency

When I evaluate a new library, I don’t start with the landing page or the quick-start guide. I open the GitHub repo. I look at the source of the core module. I check how errors are handled. I see if there are massive, untested conditional blocks. I gauge the author’s discipline. You can tell more about a library from reading 200 lines of its source than from 20 pages of its documentation. Documentation is a sales pitch. Source code is the product.

This applies to proprietary code too. Internal microservices often have stale wiki pages. The only reliable documentation is the service’s current code and its integration tests. If the tests pass, they demonstrate the actual contract. Read the tests. They’re executable documentation that can’t drift out of sync without breaking the build.

When You Must Read the Docs First

I’m not arguing for zero documentation. High-level architecture, intent, and known limitations have real value. Read the README to understand what problem the library solves. Skim the getting-started guide to see the overall shape. But the moment you need precise behavior—a parameter’s valid range, the order of callback execution, thread-safety guarantees—switch to the source. Docs are a map; source is the territory. The map is not the territory, and when they conflict, the territory wins.

The Debugging Accelerator

Time is the scarcest resource in engineering. Source-code literacy saves it. When a bug surfaces, you can build hypotheses from documentation and test them in production, or you can read the source and know. I’ve watched colleagues write elaborate reproduction scripts for bugs that were obvious from a three-line function body. They trusted the docs. The docs were wrong. The function had a clear off-by-one error visible to anyone who looked.

Learn to love the stack trace. It isn’t a wall of noise; it’s a map of execution. Each frame is a file and a line number. Click through. Read the code at that line. Understand the state. You’ll solve problems in minutes that others chase for days.

Conclusion

The hierarchy is immutable: source code, then tests, then official documentation, then community forums. Invert that order at your own risk. Next time you reach for a library’s docs because something isn’t working, stop. Open the source. Find the function. Read the logic. You’ll become a better engineer—not because you memorized more APIs, but because you started treating software as a transparent system instead of a black box. The code is always honest. Be someone who can hear it.

FAQ

What if the source code is too complex to understand quickly?

Zoom in on the specific path that matters to your problem. You don’t need to grasp the whole architecture. Use your IDE’s navigation to follow only the branches your input would trigger. Complexity is often an illusion created by volume; the single execution path is usually manageable.

Does this mean I should not write documentation for my own projects?

No. Write high-level docs that explain intent, architecture, and non-obvious decisions. But treat them as guides, not specs. Make sure your code is readable and your tests are comprehensive—those are the artifacts other developers will ultimately trust.

How do I handle closed-source or proprietary dependencies where I cannot read the source?

Treat them as high-risk components. Demand thorough documentation and build strong integration tests on your side to pin down behavior. If the vendor can’t offer transparency, you have to build your own safety net through extensive testing and defensive coding.