ACID Compliance

To ensure streams of data are accurate and consistent, ACID compliance might be a requirement placed on our database management system. To that end, let’s review the ACID acronym:

Atomicity

Make all changes or none.

Consistency

All changes obey database rules.

Isolation

Apply all changes step by step before showing any changes.

Durability

Changes will be permanent.

Note: Some systems will claim “ACID” but ACID across one document is not the same as ACID across the dataset. Likewise, eventual consistency is not the same as formal consistency.

The choice of DBMS will determine whether the database can be trusted to provide the necessary reliability, or whether the developer will need to account for this in the application code.

Minimal Schema Changes

For DBAs, schema changes are critical transactions that need planning and careful execution. Frequent and reckless changes in schemas are both difficult to manage in RDBMSs and propagate risk in ways that are sometimes difficult to mitigate. Any NoSQL solution will offer ways to store documents with different schemas into the same table. These tools do not, however, mitigate issues raised by application linkage to particular data schemas. The key from the administrator’s view is to ensure that the integrity of the data is not compromised in ways that adversely affect which data is delivered to end users.

High Availability

DBAs set up database software in such a way that the information will always be available to users. This can include configuration and maintenance of high-availability (HA) features, as well as disaster recovery (DR) features.

Fault Tolerance

Fault tolerance defines the ability of your application to maintain responsiveness in the face of infrastructure failures. This requires that a DBA replicate databases, and that facilities for “failing over” from one copy to another are available in whatever DBMS is selected.

Questions to Ask Moving Forward

If we are choosing to use a multi-model data management solution, our DBAs will want to know whether we have a standard operating procedure for dealing with existing data following each implicit or explicit schema change. Will your applications be able to handle the expected mixture of schemas at every juncture?

What about the availability for our application? Does the DBMS we are considering support HA, and what ongoing management will be required to maintain it? If our application must maintain some critical minimum level of continuous performance, does the DBMS support fault tolerance?

With so many concerns expressed here, one might legitimately ask how software developers can possibly address them all. The answers proposed here are organizational in nature. Developers might want to learn the specific features of the multi-model database products that are available, but that knowledge requires a collaborative approach from the organization implementing the solution to deliver the potential value of a data-driven approach.

To lay the groundwork for this type of cooperation, I’m going to systematically visit several areas of potential synergy. In many organizations, these functions are managed by separate groups and full adoption of a data-driven paradigm could involve transformation changes to the way organizations are, in fact, organized.

System administration, compliance and risk management, data analysis, business analysis, and the development process itself are addressed next.