Understanding the CAP theorem

The CAP theorem states that a distributed system can't have all three properties at the same time:

• Consistency
• Availability
• Partition resiliency

In practice, you get to pick if you want a CP system or AP system. A CP system (consistent and partition resilient) is always consistent and will not serve queries or make changes if there is a network partitioning between components. It will function only when the system is fully connected. This obviously means that you don't have availability. On the other hand, an AP system (available and partition resilient) is always available and can operate in split-brain fashion. When the system splits, each part may continue to operate normally, but the system will be inconsistent because each part is unaware of transactions happening in the other part.

AP systems are often referred to as eventually consistent systems because, when connectivity is restored, some reconciliation process ensures the entire system syncs up again. An interesting variant is frozen systems, where, when a network partitioning occurs, they degrade gracefully and both parts continue to serve queries, but reject all modifications to the system. Note that there is no guarantee that, at the moment of partitioning, both parts are consistent because some transactions in one part may still not be replicated to the other part. Often, it is good enough because the divergence between the split part is small and will not increase over time because new changes are rejected.