How to scale...
I received an email this morning from another Oracle employee and the gist was
"what percentage of the questions regarding scalability, reliability or performance you get are as a result of an 'elegantly' designed middle tier implementation that just took for granted a well tuned, well designed database"
Meaning, the presumption was made that the database will just scale and perform well without any serious thought. This question came about as a result of a discussion he was having with another internal employee who had (wisely) written (emphasis is mine)
I think some of the successes that I had in development projects as I started doing more with middle tier development always came from my early work and knowledge of the database and its capabilities. I found with Scada systems, billing systems, etc. that no matter how well your architect a system, if the database is not used effectively, then your systems will never scale or be reliable. Architects put so much time into creating what they perceive to be a perfectly architected middle tier (or SOA), without realizing that much of their success will come from the Database tier.
It has been my experience that too many of the “middle tier” experienced people now days have no knowledge of how to effectively use the DB, how to architect systems with a large middle tier component around a DB, and how to take advantage of DB features as part of a middle-tier architecture. Middle tier people have become DB illiterate.
They wanted to know what I thought about this concept. Most of you can already probably guess what my response was I am sure :) I wrote back simply:
all of them, 100%. I am not being sarcastic, just agreeing 100% with what was said. Chapter 1 of my books start with:
Developing Successful Oracle Applications
I spend the bulk of my time working with Oracle database software and, more to the point, with people who use this software. Over the last 18 years, I’ve worked on many projects—successful ones as well as complete failures—and if I were to encapsulate my experiences into a few broad statements, they would be
- An application built around the database—dependent on the database—will succeed or fail based on how it uses the database. Additionally, in my experience, all applications are built around databases. I cannot think of a single useful application that does not store data persistently somewhere.
- Applications come, applications go. The data, however, lives forever. In the long term, the goal is not about building applications; it really is about using the data underneath these applications.
- A development team needs at its heart a core of database-savvy developers who are responsible for ensuring the database logic is sound and the system is built to perform from day one. Tuning after the fact (tuning after deployment) typically means you did not give serious thought to these concerns during development.
To me, the ability to scale (perform, be reliable) starts with and many time ends with the database implementation and how the application interacts with the database.