Protocols Are the Unseen Infrastructure of Software
Every engineer knows that moment. Two components disagree on what a message looks like, and the whole thing grinds to a halt. The product might have a slick UI, snappy buttons, a clean REST API. But underneath, a protocol is doing the real work. A good protocol is simply an agreement between systems—a formal spec that says: here are the fields, this is the encoding, this is the order of operations. It doesn’t give a damn about your brand identity or your onboarding flow. It cares about correctness, efficiency, and staying power.
I’ve spent a decade watching protocols evolve and products die. TCP has survived 40 years. Chat apps, file-sharing tools, game servers built on top of it? They come and go. The difference isn’t just technical. It’s philosophical. A good protocol is designed for machines first. A good product is designed for humans first. Those two goals often collide head-on.
What Makes a Protocol “Good”
I evaluate protocols on three axes: unambiguity, composability, and extensibility. Unambiguity means someone can build a conforming implementation from the spec alone, no guessing about the original author’s intentions. Composability means you can layer, wrap, or pipeline the protocol without breaking its core guarantees. Extensibility means it can absorb new use cases without spiraling into a version-negotiation hellscape.
Unambiguity: A Parser Should Never Guess
Picture a protocol where the payload length is a 16-bit integer, but the spec never mentions endianness. That’s not a protocol—that’s a suggestion. I’ve debugged embedded systems where a temperature sensor randomly spit out values in the millions because the firmware developer assumed little-endian and the server assumed big-endian. A protocol has to define the wire format down to the bit. JSON gets called a data format, but it becomes a protocol only when you nail down the schema, field ordering, and error-handling semantics. Skip those and every implementation turns into a game of telephone.
// Bad: JSON without a strict schema
{
"temp": 23.5
}
// Good: CBOR with a defined CDDL schema
{
1: 23.5 // key 1 is always Celsius, float32
}
The CBOR example is unambiguous because the binary encoding leaves no wiggle room: the field is a numeric key, the value is a 32-bit float. You can’t accidentally send a string or a nested object. The parser doesn’t have to be forgiving—it just follows the rules. That’s the hallmark of a good protocol.
Composability: The LEGO Block Principle
TCP is composable because it gives you a reliable byte stream, and you can layer TLS on top for a secure stream, then HTTP on top for a document transfer protocol. Each layer adds guarantees without wrecking the guarantees of the layer below. Compare that to something like FTP, which tangles control and data channels in a way that breaks horribly behind NATs. FTP isn’t composable—it assumes a specific network topology that hasn’t been common for two decades.
When I design a protocol, I ask: can someone drop this into a proxy without rewriting the whole stack? If the answer is no, the protocol is too tightly coupled to its transport. That tight coupling might get the product out the door faster, but it’ll rot the moment the network environment shifts.
Extensibility: Versioning Without the Nightmares
Extensibility is the hardest to get right. The naive move is to slap a version number at the front of every message and hope for the best. That just creates a combinatorial explosion of supported versions. A better approach is optional extension points—fields that new clients can emit and old clients can ignore. Protocol Buffers handle this well: unknown fields get skipped during deserialization. A service written in 2015 can talk to one written in 2025 without a human translating the diff.
I once inherited a binary protocol that used a fixed-length header with no reserved bits. When we needed to add a priority flag, we had to break backward compatibility. The product team said, “Just force everyone to upgrade.” That’s not a protocol—that’s a hostage situation. A good protocol plans for the unknown. It reserves space. It uses TLV (type-length-value) encodings. It doesn’t pretend the first version is the last version.
What Makes a Product “Good”
A good product solves a specific problem for a specific person at a specific moment. That’s it. It doesn’t need to be general-purpose. It doesn’t need to be composable. It needs to be usable, discoverable, and fast in the hands of a tired user on a Monday morning.
I’ve watched engineers—myself included—try to turn products into platforms too early. We’d expose every internal protocol as a public API, assuming developers would want the same control we had. They didn’t. They wanted a single button that did the thing. The product that wins is the one that hides the protocol, not the one that celebrates it.
Usability: The 10-Minute Rule
If a user can’t get value from your product in 10 minutes, you’ve lost them. This is where protocol thinking falls apart. A protocol designer says, “They just need to read the spec.” A product designer says, “They just need to click this button.” The product designer is right—for the user. The spec is a liability to the user, not an asset. It’s the implementation detail that should be bolted under the floorboards.
Take the Signal messaging app. The underlying Signal Protocol is a cryptographic masterpiece: double ratchet, prekeys, forward secrecy. But the user never sees any of that. They see a chat window that looks like SMS. The product is good because it takes an extremely complex protocol and makes it feel like nothing. The protocol is good because it doesn’t leak its complexity into the user experience.
Discoverability: Don’t Make Me Read
Users don’t read documentation. They poke at the interface until something happens. A good product makes that exploratory poking safe and informative. If deleting a message requires a long-press, the interface should hint at it—a subtle animation, a contextual tooltip. A bad product buries actions behind invisible gestures and calls it “minimalism.” That’s not minimalism; that’s obscurantism.
Protocols, by contrast, are inherently undiscoverable. You can’t poke a TCP socket and learn what it expects. You need the spec. That’s fine—protocols are for machines and the engineers who program them. But a product that inherits a protocol’s undiscoverability is a product that will fail. The translation layer between the two is where most software projects die.
The Collision: When a Protocol Becomes a Product
The most dangerous moment in a software project’s life is when someone says, “The protocol is the product.” This happens in developer tools, in blockchain projects, in any domain where the users are also engineers. The logic seems sound: our users understand the protocol, so we can expose it directly. The result is almost always a product that only the original team can use.
I worked on an IoT platform where the product was essentially a thin wrapper around our internal MQTT topics. The documentation was a list of topic strings and payload formats. We couldn’t figure out why adoption was slow. The answer was obvious in hindsight: no one wants to learn a new publish-subscribe hierarchy just to turn on a light. They want an API that says light.turnOn(). The protocol was good—MQTT is a solid, composable, extensible protocol. The product was bad because it forced the user to think like the protocol.
The fix wasn’t to abandon the protocol. It was to build a product layer on top: a REST API with sensible defaults, an SDK that handled topic management, and a UI that generated the right commands. The protocol stayed as the internal backbone. The product became the external face. That separation is not a compromise—it’s the correct architecture.
Case Study: HTTP/2 vs. gRPC
HTTP/2 is a protocol. It gave us multiplexed streams, header compression, and server push. It’s a good protocol—unambiguous, composable (TLS is practically mandatory), and extensible through frames. But HTTP/2 alone is not a product. You can’t download HTTP/2 and build an app. You need libraries, tools, and a mental model.
gRPC is a product built on HTTP/2. It takes the protocol and wraps it in a code generator, a schema language (protobufs), and a set of client libraries that feel native in a dozen languages. The product is good because a developer can define a service in a .proto file, run a command, and have working client and server stubs. The protocol is still there—you can inspect the wire format with Wireshark—but the product hides it until you need it.
The lesson: a good protocol is a prerequisite for a good infrastructure product, but it’s not enough. The product has to translate the protocol’s raw power into a form that fits a developer’s workflow. That translation layer is the hard part.
How to Spot the Difference in Your Own Work
I use a simple test: If you removed the UI, would the thing still have value to a machine? If yes, you have a protocol. If no, you have a product. That distinction helps me decide where to invest my effort. Protocols need rigorous specification, conformance testing, and a commitment to backward compatibility. Products need user research, interface iteration, and a willingness to throw away features that don’t click.
Another test: Can a competent engineer reimplement this from the spec alone? If the answer is no, the spec is incomplete—or you’ve accidentally built a product that you’re calling a protocol. I’ve seen startups publish “protocol” specs that are really just descriptions of their current server behavior. That’s not a protocol; that’s a memoir. A protocol spec should be enough to build an interoperable alternative without ever talking to the original author.
When I’m building a product, I deliberately avoid making the internal protocol visible. The product’s API is not the protocol—it’s a curated, simplified, versioned interface that might map to multiple internal protocols over time. When I’m building a protocol, I deliberately avoid thinking about the product. I think about the wire, the parser, the error states. Keeping those two mindsets separate has saved me from some truly awful architectural decisions.
FAQ
Q: Can a single system be both a good protocol and a good product?
A: Rarely. The goals pull in different directions. A protocol aims for generality; a product aims for specificity. You can have a product that wraps a good protocol (like gRPC), but the protocol itself has to stay product-agnostic. The moment you tailor the protocol to a single product’s UI flow, you’ve compromised its reusability.
Q: What’s the most common mistake when designing a protocol?
A: Ignoring extensibility. Teams ship a protocol that exactly meets the current product’s needs, with no reserved fields, no version negotiation, and no tolerance for unknown data. Six months later, they’re stuck with a breaking change. The fix is to always include a mechanism for forward compatibility—even if it means reserving a few bytes you don’t use yet.
Q: Why do open protocols outlast commercial products?
A: Because protocols solve a problem at the infrastructure level, not the user-experience level. TCP, HTTP, TLS—these don’t depend on a company’s design trends or quarterly targets. They become part of the environment, like electricity. Products, by their nature, are tied to a moment in time: a specific user need, a specific technology stack, a specific business model. That’s not a flaw; it’s just a different category of thing.
Q: How do I convince my team to invest in protocol quality when the product deadline is looming?
A: Show them the cost of a breaking change after launch. The 30 minutes you save by skipping a proper versioning mechanism will cost you weeks of migration work and customer frustration later. Protocols are the part of the system that’s hardest to fix once deployed. Investing early is not perfectionism—it’s risk management.