Thursday, September 29, 2005

Most incredible statement I heard this week

Was this:

I might be naive, but Oracle is supposed to be this big powerful RDBMS engine but it can't handle anything. I think we're overpaying for it. The design should have nothing to do with performance. Oracle must be able to handle everything.

The design should have nothing to do with performance....

Think about that statement for a moment.

The design (the most fundamental thing, the cornerstone of the 'system', the key piece, the thing were much time should be spent...) should have nothing to do with performance.

The design should have nothing to do with performance....

I keep saying it over and over and it still sounds, well, not smart.

Think about a car: "the design should have nothing to do with performance".

Think about anything, "the design should have nothing to do with performance".

It is all about the design, from day 1, the key component to performance is the design - maybe the only component in fact.

Two others have commented on this as well.  First was Pete-S and second was Tim Hall. I agree with their sentiment.

The following is the opening section of both Expert One on One Oracle (modified from the original a bit) and Expert Oracle Database Architecture and sums up my thoughts on this (which are 180 degrees opposite of "design should have nothing to do with performance":

I spend the bulk of my time working with Oracle database software and, more to the point, with people who use this software. Over the last 18 years, I’ve worked on many projects—successful ones as well as complete failures—and if I were to encapsulate my experiences into a few broad statements, they would be

  • An application built around the database—dependent on the database—will succeed or fail based on how it uses the database. Additionally, in my experience, all applications are built around databases. I cannot think of a single useful application that does not store data persistently somewhere.
  • Applications come, applications go. The data, however, lives forever. In the long term, the goal is not about building applications; it really is about using the data underneath these applications.
  • A development team needs at its heart a core of database-savvy developers who are responsible for ensuring the database logic is sound and the system is built to perform from day one. Tuning after the fact (tuning after deployment) typically means you did not give serious thought to these concerns during development.

These may seem like surprisingly obvious statements, but I have found that too many people approach the database as if it were a black box—something that they don’t need to know about. Maybe they have a SQL generator that they figure will save them from the hardship of having to learn the SQL language. Maybe they figure they will just use the database like a flat file and do keyed reads. Whatever they figure, I can tell you that thinking along these lines is most certainly misguided: you simply cannot get away with not understanding the database. This chapter will discuss why you need to know about the database, specifically why you need to understand

  • The database architecture, how it works, and what it looks like.
  • What concurrency controls are, and what they mean to you.
  • That performance, scalability, and security are requirements to be designed into your development efforts, not something to hope you achieve by accident.
  • How features are implemented in the database. The way in which a specific database feature is actually implemented may not be the way you might envision. You have to design for how the database works, not how you think it should work.
  • What features your database already provides for you and why it is generally better to use a provided feature than to build your own.
  • Why you might want more than a cursory knowledge of SQL.
  • That the DBA and developer staff members are fighting for the same cause; they’re not two enemy camps trying to outsmart each other at every turn.

This may initially seem like a long list of things to learn, but consider this analogy for a second: if you were developing a highly scalable, enterprise application on a brand-new operating system (OS), what would be the first thing you would do? Hopefully, your answer is, “Find out how this new OS works, how things will run on it, and so on.” If you did not do this, your development effort would fail.

 

Consider, for example, Windows versus Linux. Both are operating systems. Each provides largely the same set of services to developers, such as file management, memory management, process management, security, and so on. However, they are very different architecturally. Consequently, if you’re a longtime Windows programmer and you’re asked to develop a new application on the Linux platform, you would have to relearn a couple of things. Memory management is done differently. The way in which you build a server process is considerably different. Under Windows, you develop a single process, a single executable, with many threads. Under Linux, you do not develop a single stand-alone executable; instead, you have many processes working together. In summary, much of what you learned in the Windows environment doesn’t apply to Linux (and vice versa, to be fair). You have to unlearn some old habits to be successful on the new platform.

What is true of applications running natively on operating systems is also true of applications that will run on a database: you need to understand that the database is crucial to your success. If you do not understand what your particular database does or how it does it, then your application is likely to fail. If you assume that because your application ran fine on SQL Server, it will necessarily run fine on Oracle then, again, your application is likely to fail. And to be fair, the opposite is true: a scalable, well-developed Oracle application will not necessarily run on SQL Server as is without major architectural changes. Just as Windows and Linux are both operating systems, but fundamentally different, so Oracle and SQL Server (pretty much any database could be noted here) are both databases, but fundamentally different.

POST A COMMENT

53 Comments:

Anonymous Robert Lockard said....

Tom,

Sadly I am currently working with a java developer who believes learning about how oracle works is not important. Infact he has told me many times, "I don't want to know anything about Oracle." This has caused many problems with the application and the ablility to deliver a product to the customer. I have also heard this comment from the same developer. "I don't care about performance, our users are ... so, they can wait." I even purchased your book "Effective Oracle by Design" and gave it to the developer only to have it thown back at me. The problem is not ignorance (well that could be debated) The problem is schools putting these people into the workforce. Some take encapulation to the extreme, and believe everthing needs to be a black box and knowledge of the black box is not needed.

-Rob

Thu Sep 29, 07:56:00 AM EDT  

Anonymous Kim Berg Hansen said....

I agree 111% - absolutely. And should I design an application from scratch I hope I can live up to those ideals.

Unfortunately I have to work within an ERP application where the designers in the vendor development team decided that it should be "portable" and "database independent", so it can run on Oracle, SQL Server, DB2 and their own legacy database with as few changes to the code as possible.

What happens? They treat the base as a black box with nothing but tables and indexes.
"Sequences? Well the databases does it differently so we can't use the database features - let's create a one-record table with an integer column for each sequence and update that, that'll work in any database."

Makes you want to wish that there were Kyte books available when these designers make those decisions 12 years ago. Those decisions continually haunt my daily work life :-(

Thu Sep 29, 08:07:00 AM EDT  

Blogger Rudy said....

Maybe once he came across this, or just heard of it.
Maybe he couldn't just understand it.
Surely he went wrong :-)

Anyway, I can't have enough of reading your articles - thank you.

Rudy (Italy)

Thu Sep 29, 08:55:00 AM EDT  

Blogger David Aldridge said....

While we're opn this topic, you'll enjoy this thread: http://discuss.joelonsoftware.com/default.asp?joel.3.218447.10

"I'd say RDBMS' are more complicated to use because you have to actually design the database on top of everything else. With an OODBMS the database simply stores whatever business objects you tell it to store; no database design (and skill) is required. That also means you won't need any skilled database folk on your team, just business and interface level programmers (major bonus)."

I smell a magic bullet. I'm sure that FP would enjoy that posting, also.

Thu Sep 29, 09:04:00 AM EDT  

Blogger Jeff Hunter said....

It is all about the design

And here I thought it was all about bind variables. ;)

Thu Sep 29, 09:05:00 AM EDT  

Anonymous Jason McIntosh said....

Speaking of concurrency, I'm kinda curious as to the best methods to handle concurrency in Oracle. I know there's the select for update, but in a webserver environment that's stateless, that doesn't necessariliy work the best - i.e. you may not keep the state from request to request, plus, if they don't do anything for a while, other users can't update said records.

One method I've seen/used is to have another unique index field that is used for updates, and a trigger to update that index field after update. This works well, but I thought that there might be another solution.

Any suggestions? Good books/articles on the subject? I'm developing a web based app which uses an interface layer, then object/objectfactory java objects which talk directly to the database instead of using entity beans or similar constructs - so I'm doing direct SQL queries/manipulations.

Thanks for any advice on this! It seems to be one of the less discussed topics out there.

Thu Sep 29, 09:43:00 AM EDT  

Blogger Joel Garry said....

An application built around the database—dependent on the database—will succeed or fail based on how it uses the database.

My experience has been it will succeed or fail based on how useful it is for the users tasks, or more precisely, the users perception of usefulness.

Applications, like OS's, do not succeed based on technical superiority. They succeed based on a combination of functional adequacy, marketing, timing and luck.

Many projects have failed because of Obsessive Technical Disorder. Note that I think one of the reasons linux is succeeding is because it recognized this early; see the original discussions about microkernels v. monolithic debate. Once it got going it was able to be layerized. And one could argue it is now suffering from OTD.

I'm not saying this is how it should be, just describing what I've seen. In a nutshell.

I'm wishing the next big "paradigm shift" will be towards building apps correctly for the ages, but I'm not holding my breath.

Thu Sep 29, 09:51:00 AM EDT  

Blogger shrek said....

A wise person once told me "it is far easier to design performance in than to add it afterward."

Thu Sep 29, 09:54:00 AM EDT  

Anonymous Rob H said....

I got an idea, I'll take my VW bug, drop a 700HP turbocharged V10 in it. I'm SURE I can win daytona with that much power.

It would rediculous to think otherwise ....

Thu Sep 29, 10:16:00 AM EDT  

Blogger Javier Morales said....

In the datawarehouse I'm working on now, I fought a lot over the need for FK's, NN's and PK's. They said it was too late to change the design.

Now, cartesian joins, outer joins and non profitable rewrites over materialized views are just dropping the server, flooding temp tablespace space, etc.

They say now, "let's change design after it get's into production environment". Ouch!

Thu Sep 29, 10:49:00 AM EDT  

Anonymous Enayet said....

May not be relevent, but quite interesting...

Microsoft's Midlife Crisis
"What has gone wrong? Microsoft, with $40 billion in sales and 60,000 employees, has grown musclebound and bureaucratic. Some current and former employees describe a stultifying world of 14-hour strategy sessions, endless business reviews and a preoccupation with PowerPoint slides; of laborious job evaluations, hundreds of e-mails a day and infighting among divisions so fierce that it hobbles design and delays product releases."
http://www.forbes.com/home/technology/2005/09/12/microsoft-management-software_cz_vm_0913microsoft.html

Sound to me typical Bill Gates' Design problem ...?

Thu Sep 29, 10:53:00 AM EDT  

Anonymous Anonymous said....

who said that quote?

Thu Sep 29, 11:09:00 AM EDT  

Blogger Mark G said....

I think all too often companies/people get so wrapped up in tool sets it's often forgotten tool sets although valuable analysis/design still play the most important role in IT systems.

Thu Sep 29, 11:39:00 AM EDT  

Blogger Jan said....

Attending a presentation about Oracle at a BIG financial institution (my wife works there :-) one of the attending technical people (not the presenter luckily) told us he thought sql performance tuning is a waste of time because he'd just buy a bigger machine to run the slow app.

Amazing...

Thu Sep 29, 11:54:00 AM EDT  

Blogger scubajim said....

When I read that comment the other day my immediate thought was that the speaker had a political or religios axe to grind. That since Oracle "costs so much" it should automaticaly solve performance problems. It was a denial of their participation and responsibility in the building of the application. Unfortunately this sillyness happens all to often.

I've encountered this attitude numerouse times in my professional career. For example, I was working on customized pension administration systems (defined benefit plans) and the common calulation was highest average pay. (eg highest 60 consecutive months out of the last 120 months). The company myth was if we changed from an inteprerated scripting language to VB (version 1 had just been released like 3 months earlier) or compiled C code then the calculations "would be orders of magnitude faster".

Fortunately, the proprietary scripting language we were working in had a very nice profiler. I would come on site, start the profle, start the pension calculation and look at the results. It was not unusual that I could take a 3 to 4 minute calculation down to 6 to 9 seconds with about 1 hour of work. Why? They would retrieve the COMPLETE pay information during the calculation from the database 120 times! The problem wasn't the scripting language. You could do the same thing in C and sure the calculation might not take 3 minutes but it would take 2 minutes 53 seconds. (not much of an improvement)

Yes, design is vital.

As a senior coworker told me once:
Perfomance is part of the job. Not something you bolt on afterwards.

How right he was.

Thu Sep 29, 12:10:00 PM EDT  

Blogger Tim... said....

We've all seen situations where buying more CPU, memory or faster disks has provided a short-term performance improvement. A fool assumes the problem is solved and moves on, while a sensible person uses the time (s)he's bought to refactor the application so it doesn't happen again. Of course the wise person doesn't get into that sort of mess in the first place :)

Thu Sep 29, 12:20:00 PM EDT  

Anonymous Anonymous said....

The link to Scott Spendolini's blog hasn't been updated to the new location:

http://spendolini.blogspot.com/

Thu Sep 29, 01:24:00 PM EDT  

Anonymous Gabe said....

Jason McIntosh said:

Speaking of concurrency, I'm kinda curious as to the best methods to handle concurrency in Oracle. I know there's the select for update, but in a webserver environment that's stateless, that doesn't necessariliy work the best …


You may want to get Tom’s latest book and read chapter 6 … it covers pessimistic vs. optimistic locking strategies pretty well.

Thu Sep 29, 01:26:00 PM EDT  

Anonymous Anonymous said....

Well,
Oracle is one of the best. but there are better for specific uses, like sybaseIQ for data warehouse. or timesten for realtime.

Thu Sep 29, 02:04:00 PM EDT  

Blogger Alberto Dell'Era said....

... or timesten for realtime

But TimesTen has now joined the Oracle family (Larry found a couple of spare coins after his shopping frenzy ;)

http://www.oracle.com/database/timesten.html

Thu Sep 29, 02:42:00 PM EDT  

Anonymous noname said....

The design should have nothing to do with performance....

Hmmm... this guy certainly works for Micro$oft...

Thu Sep 29, 04:43:00 PM EDT  

Anonymous NR said....

I am not surprised by this comment because similar kind of comment once made by one of the director " I know their performance is not good but it's scalable" in my department. Also I don't understand this comment made by so many people design your application in way that will have minimum dependency on a technology (specially oracle). I think that way you are not making an independent application but restricting yourself not to use one of the best technology out there. But so far they are trying to listen with hands on their ears.

Thu Sep 29, 05:10:00 PM EDT  

Anonymous Kashif said....

Hi Tom and all -

Software without design... hmm, probably a management-type without much experience in developing software, that's all I can think of.

On a somewhat related note, I was curious to know your thoughts about this posting I stumbled across. There are some merits to the argument presented in that discussion. Anyway, thought I'd put it out there for you and others to see...

http://discuss.joelonsoftware.com/default.asp?joel.3.217131.40

Kashif

Thu Sep 29, 05:24:00 PM EDT  

Blogger William Robertson said....

Maybe one day it'll be like the movies, where you can just mumble some vague requirements and the ship's computer designs a new holoprogram for you. That seems to be what everyone wants and expects software vendors to provide. The idea that sometimes you might actually have to think through what it is you are asking for is just an inconvenience that should be dealt with by faster processors and better software.

Thu Sep 29, 06:10:00 PM EDT  

Blogger Noons said....

"design has nothing to do with performance"

Only a java-head could come up with such a cretin concept.

And yet, it's exatcly these people that the entire Oracle marketing panders to.

So there you go folks: it's Tom, Lex, the Oakies, Fabian, Date, Codd, just about any other Oracle developer around for more than 3 years, etc, that are wrong.

It's people who utter idiocy like this who are right.

One wonders how much longer before the inevitable crash...

Fri Sep 30, 01:38:00 AM EDT  

Blogger Tarry said....

I don't know Tom. Not that I want to punture anyone's baloon here but a statement like that need not be taken seriously.

Period.

Fri Sep 30, 03:43:00 AM EDT  

Blogger Tarry said....

:-|Gnarr Typo's :*puncture*, *balloon*

Fri Sep 30, 03:45:00 AM EDT  

Blogger Thomas Kyte said....

"I'd say RDBMS' are more complicated to use because you have to actually design the database on top of everything else.

I read through that discussion and in my opion, one of the more intelligent comments was:


The main problem with OO databases is that the objects are based on the needs of the application, but the data is likely to outlive the application (and thereby the object model) and be used for other purposes outside the given application.


I would change the "the data is likely" to "the data WILL".

How many of you use only applications written 10 years or more ago?

How about 5 years or more?

I've a feeling that in 5 years, what is state of the art today will be laughed at and rewritten using some new paradigm...

but the data will live on. It has in the past.


My experience has been it will succeed or fail based on how useful it is for the users tasks, or more precisely, the users perception of usefulness.

Agreed, that is another point. My point was - if you treat the database (operating system, development tool - whatever) as a black box, I don't care how cool the gui is, how sexy it is, how functional it is - it won't fly any more than a pig will.




I still cannot figure out why many developers will spend years learning the in's and out's of the language du-jour (I got tired of it after learning PL/I, Rexx, Exec 2, JCL, SAS, C, C++, Ada, Cobol, Snobol, Lisp, Prologue, Java, ........) but won't spend a day on the database.

I say many, not all, because I've been a developer from day one.



On a somewhat related note, I was curious to know your thoughts about this posting I stumbled across. There are some merits to the argument presented in that discussion. Anyway, thought I'd put it out there for you and others to see...


I'll blog on that soon - it is something that I have a definite opinion on.



but a statement like that need not be taken seriously.

that, I don't I agree with. It is a sentiment I've heard far too often. If it were isolated - ok, but it is a recurring theme.

Fri Sep 30, 07:01:00 AM EDT  

Anonymous Anonymous said....

How many applications are "design once" type of applications? Today, I design one application very well. By the time this application goes into production, the managers are not sitting idle. They are gathering new requirements from the client, adding new functionality which no one had thought of in the first phase. Most of the business rules in the first phase of application have now changed. What are the options? Either redo the entire first phase and incur the expense and delay of rewrite and retest or do a patch up job and somehow merge the new functionality into the existing one? Most project managers will take the second approach because they know that second phase is not the end. There will be phase three, four, five...for the next ten years till the client runs out of money.

New requirements will come in which had never been seen before, never been imagined before, business rules will change drastically. The functionality provided by the application has raised the user expectation. They want more from the application. So, once the business rule said that there will one and only one address for a person. Now there can be three or four addresses. What do the project managers do? Will they rewrite the entire previous portions of the application? No. In this whole melee, where is the time to think about design? It is all about getting the new version out and getting money from the client. Performance and design are not part of the contract.

Fri Sep 30, 08:53:00 AM EDT  

Blogger Thomas Kyte said....

Performance and design are not part of the contract.

I'll let our developers know that, they'll be very relieved. Imagine the time we could save not worrying about those two things.

Life just got better....

not

Sorry, but from my perspective performance and design are the contract.

Fri Sep 30, 09:15:00 AM EDT  

Anonymous Anonymous said....

The poster is partially correct. On most projects the managers are not the least bit interested in hearing about performance until there is a problem, then they want a quick fix. You can't put a gun to a managers head and say you will do it this way! They typically don't listen.

Development managers typically are dominated by GUI developers and database people rarely get promoted to prominent positions on a team. Most of the time database developers are marginalized and used to create tables, write sql, and left in after the fact. You can say all you want about it, but that is the way it works.

I have found that it takes alot of effort to build up respect for your skills on a project if you are database centric specialist.

I have found that it is very hard to get anyone to listen to a database person about much of anything. To be fair, most technical people are incompetent which translates to most database developers being incompetent. So alot of managers have had bad experiences working with database people who blow off their mouths about performance and have no idea what they are talking about.

Fri Sep 30, 12:00:00 PM EDT  

Anonymous Anonymous said....

The best quote I ever heard about development:

'Customers never have enough money for us to do it right, but they always have enough money for us to do it again.'

Fri Sep 30, 12:04:00 PM EDT  

Blogger Pete_S said....

Development managers typically are dominated by GUI developers and database people rarely get promoted to prominent positions on a team.
Maybe true in some environments - but the original post that Tom (and I) wrote about was a for a Data Warehouse - I am at a loss to see how anyone could "design" a data warehouse with any regard to perfromance. Perhaps my 'duty of care' is too strong for the real world!

Fri Sep 30, 12:20:00 PM EDT  

Blogger Thomas Kyte said....

I have found that it takes alot of effort to build up respect for your skills on a project if you are database centric specialist.

I've been lucky then - for I've worked on projects where the database is sort of the central key thing. And I do find many people are aware of that (that they database is sort of "key", the application is pretty).

And they are in fact becoming more aware of that as they mature. It was only 12/14 years ago I realized the end goal was not my application - and that was when they started retiring my first applications (and tried to figure out what the heck they did and how they did it)...


And don't get me wrong, I'm talking about database applications. If you are making a word processor - that is one thing. Making an HR system - something else entirely.


And I agree with Pets_S about the data warehouse thing - although I have seen DW's created not with a database centric view but rather on how pretty the screens could be made.

Those only end up with pretty screen shots and no deployments however. So they are short lived (late/never to deliver and over budget)... Unfortunately, I get to see these after the fact - to try and triage them.


If I had to guess as to the major reason for switching databases -- it would be "version 2".

What I mean by that is "well, version 1 was so bad - it must be the database, we ran a benchmark and database X ran this query faster so we'll go with that one and it'll be better"

But, it isn't. Not unless they matured a lot during this process.

(I try to change the parameters of these benchmarks - to get them to let someone who knows the database actually look at the processes and *change* them. Not always successful getting them to do that "database independence" and all)

Fri Sep 30, 12:32:00 PM EDT  

Anonymous Anonymous said....

Welcome to the world of consulting. When big consulting companies bid for multi-million dollar projects, who draws up the contracts? Not the DBA, not the developer. Attorneys. Do they care about design? Do they care about performance? Do they even know about design and performance? All they care about is ensuring the consulting company gets its money and they get their fat fees. In none of the big contracts, 10 times the size of a phone book, that you will find performance and design. In project manager's lexicon, performance is something to be made barely acceptable when users start screaming and issue escalates to someone who directly controls writing the check.

Sorry for sounding bitter, but this is the way I have been seeing things work for the last 7 years.

Fri Sep 30, 12:50:00 PM EDT  

Blogger Thomas Kyte said....

Welcome to the world of consulting.

I've always wondered why "we" subject databases, application servers, and maybe even operating systems to involved, detailed "technology inspections" before buying them...

(I know how long, involved and detailed they are - I have to do them)

and then plop on top of them something with a really pretty screen...

That no one evaluated from a scalabilty perspective.

Or a "how does it use the technology" perspective.

Or can it even do what we need to do user wise (eg: has anyone actually used it with 400,000 users yet?)

Why are applications not held to the same standard - or why hold that standard for the database/app server if the application is never going to achieve it anyway.


I do see "second system" effects coming in however - if you have one big failed one, that adds (usually, hopefully) to the knowledge set of the implementation team and they do better next time.

I honestly quit my first job for these reasons. It was driving me nuts to see what was happening. (then it was all about database independence - I was like "what do you mean I have to *write* a two phase commit, invent my own "stored procedure" language, make ftp of flat files look like a table....)

Fri Sep 30, 12:59:00 PM EDT  

Blogger Doug Burns said....

Tom said ...

"I've been lucky then - for I've worked on projects where the database is sort of the central key thing. And I do find many people are aware of that (that they database is sort of "key", the application is pretty)."

I think you *have* been lucky then (or maybe not so much lucky as determined about the type of environment you want to work in).

I think there are two sides to this thread :-

1) The side that says ... this stuff is both important and obvious.

2) The side that says ... Yeah, we know that, but it isn't anything like what we come across at work.

I think both are true. It occurs to me that none of the posters who are talking about 'how it is' misunderstand the importance of up-front design so it's pushing against an open door to reiterate it to them. They're just saying 'not at my work, pal...'

And, sorry Pete, but I've worked in DW environments where only lip service was paid to performance, until long after the **** hit the fan. Where the focal point seemed to be functionality over any solid technical foundations.

Really, I despair at some of the systems and attitudes I come across, but once you've pointed this out (a few times), where do you go from there? We can't all resign our positions because of industry-wide stupidity ;-)

(Oh, btw, those sites where you get respect because people understand how central the database is to the entire system? Feel free to let me know about them because those are the places I'd like to work. At least once)

Cheers,

Doug

Fri Sep 30, 01:57:00 PM EDT  

Anonymous Eddie Awad said....

How come no one has mentioned "testing"?! A very important part of developing an application (any application) is to test it again and again until the tests produce the desired results.

If the design is good or bad, if the performance is good or bad... all of this *should* be caught during the testing stage. So, I cannot stress enough the importance of building good test cases to iterate over all possible test scenarios *before* your application is migrated to production.

Fri Sep 30, 02:50:00 PM EDT  

Blogger Pete_S said....

Doug:
Perhaps I'm too much of an idealist, perhaps it is because I have a lot (too much?) of control over the type of work I do. I am fortunate in that I work for a company that used to have slogan of "invent, build and manage" We design DW systems and undertake to support (even host) them for as long as the customer wants - I could never work for a company that treats development in same way as a lobing a dead cat over the neighbour's fence. Guess I'm lucky - -cheers Pete

Zogit - yet another real word!

Fri Sep 30, 03:46:00 PM EDT  

Blogger Doug Burns said....

Pete said ...

"Guess I'm lucky ..."

Yeah, you are or, as I suggested with Tom, you're committed and :-

"I could never work for a company that treats development in same way as a lobing a dead cat over the neighbour's fence."

I was just saying that although the DW crowd generally take a more rigorous approach (because of the dangers of not doing so) I've seen half-hearted disasters in that arena too.

As I said, I think this stuff is important and obvious, but I was trying to point out the reality at many sites.

But, just because reality sucks, it doesn't mean I accept it blindly ;-)

Cheers,

Doug

Fri Sep 30, 03:56:00 PM EDT  

Anonymous Anonymous said....

While we are on the subject of design - your thoughts on "why does oracle allow rows and even whole tables to be locked when a user has ONLY select access to it"? I wanted to post this question on asktom - been waiting for an oppertunity for several weeks to post a new question.

Mon Oct 03, 10:14:00 PM EDT  

Anonymous Alex said....

The design should have nothing to do with performance....
This result is from hiring not professionals is development but handy helpers in digestions of another people money.

Tue Oct 04, 09:08:00 AM EDT  

Anonymous Kompella said....

I am not sure why everyone is jumping on the person making the comment. The person making the comment is speaking non-technical lingo -- he is the enduser and his requirement is to decouple design from performance.

While design is one the way to affect(improve) performance, it aint the only way. Reality of life says that the latter is more commonly used.

Who in IT hasn't heard bad-performance being fixed with more CPU. Many times it works.

Second, Oracle partitioning is a great example of how design is decoupled from performance.

End-user wants something. It is not achievable 100%, but Oracle (& others) can definitely work towards that goal.

Wed Oct 05, 09:27:00 AM EDT  

Blogger Thomas Kyte said....

Kompella,

you should have followed the link to the entire quote which started with:


...
Anyway, we're in the meeting yesterday and one of the developers says: "I might be naive,....


one of the DEVELOPERS

not Joe end user.



My reality differs from yours - most improvement comes from the physical design.

You cannot throw more cpu at a hard parse problem (well, you can but it won't do anything).

.... Who in IT hasn't heard bad-performance being fixed with more CPU. Many times it works....

sometimes it works, sometimes it don't.

but the developer ATTITUDE of "don't have to design, just whip it out there"

is outrageous.

Wed Oct 05, 09:46:00 AM EDT  

Anonymous Anonymous said....

Besides, Oracle is a SQL DBMS not an "RDBMS engine", ( If there is such a thing )

Thu Oct 06, 02:21:00 PM EDT  

Anonymous Anonymous said....

Thomas Kyte said...

"but the developer ATTITUDE of "don't have to design, just whip it out there"

is outrageous. "

Couldn't agree more. I was a developer for a coulple of years and now endeavoring to be an Oracle DBA(love it!). I learned programming on my own, through web articles, example code, etc. One thing I read about in all these "Googlings" of mine was a recurring message of having complete(or as complete as possible) requirements, and making sure the thing is fast and reliable.

Unfortuneatly the small company I worked for during my apprentice years as it were, had an attitude of don't have to design, just whip it out by x date.

It saddens me to think that there are other people(companies) that operate with 1/2 a brain like my former employer. Someone said in a previous post about managers with no technical knowledge or experience in systems, db's, programming, or any other needed skill areas, making decisions on a whim or promising the end user a product by x date without knowing what the heck they are talking about.

Our team heard many times after a roll-out from end users that "it's slow!" We knew it but can't fight management all day when they threaten you to "get er done!"

Hillbilly management kills, lol

Thu Oct 06, 04:01:00 PM EDT  

Blogger Oracle Guy said....

Tom,
After 25 years in IT and 19 years of Oracle, I have now heard it all.
It is the thought that design having nothing to do with performance which keeps me in business.

Scott

Fri Oct 07, 11:19:00 AM EDT  

Blogger jsndba said....

We had one Java program who told me .... "Oracle is a necesary evil until we can get all the data moved into XML documents." No I'm not joking! This guy refused to learn anything Oracle, for that matter anything database. He wouldn't take advice from any of us that had 5-10 years experience. According to him, we were all dinosaurs! Thankfully he finally quit! After he left other more capable developers got the misfortune to inherit his code mess. He had actually pulled back data from multiple tables into java, then looped in java to join the data. His code to so long to run, he had put a coffee button on the screen. So that when the users were ready for a coffee break, they could click this button to do his java code. After his departure from the company, a more opened mind developer revamped this code to have the database join the result set before returning the rows. After this fix, the users said... "This window must be broken, it came up in 2 seconds, we didn't even have time for coffee!"

I could not understand where his hatred for Oracle or databases came from. Then I attended a Java conference... which cleared things up a little. While attending one of the session the speaker said... "SQL is the worse evil known to man" Because of it's syntax differences between databases. Suddenly it was clear who are new college java graduate had been listening too. After the session, I went to talk to the presenter. Asked him if he was actually suggesting java development not use Oracle, or a database. He laughed, and said of course not! That is not my message. However, I've seen more then one java programer with that attitude. They take some of the java teaching to literally. I explained to the presenter the problems we were having with our new college graduate java programmer. He could not believe they were anti-oracle anit-database. He said he would change the way he delivered his message. I certainly hope he has.

Tom, you are the Greatest! If only more developers would read your works.

Thank you for all your efforts and time.

Fri Oct 07, 04:23:00 PM EDT  

Anonymous Nik said....

I fully agree that design in critical for performance, however I feel I have to put in a word of understanding for the "young, cocky javaheads".

There are so many issues nowadays for an application developer to consider, starting from browser compatibility to J2EE deployment issues, databases etc. that "blackboxing" has become a way of survival in the world of ever changing specs.

The Java developer will probably prefer to use a ORM solution such as Hibernate to abstract away the CRUD operations (and these should not be a problem for any database anyway). The problems occur probably occur when the ORM fails to do complex joins and ends up with something that can't really be optimized by the DB.

Of course it is outrageous for a project manager to demand and change of DB vendors in mid-project buy should it happen, guess who takes the heat? Yup. The coders. "What, you hardcoded it for Oracle!? You have all those tools and wizards and you hardcoded it!?!1"

The solution is probably (as usual) somewhere in the middle.

Tue Oct 11, 04:22:00 AM EDT  

Anonymous Anonymous said....

Don't you wonder if the problem is a case of the "telephone game"? I think most people would agree that designing in optimization is a waste, since it can be hard to see ahead of time where the load will be on a system. But some people seem to hear that as performance, which is an entirely different animal. Working against minimal amounts of data and assuming performance will magically scale is a recipe for disaster. Worrying about the execution time of "if" versus "case" globally before having working SQL probably means you'll never have a chance to cook...

Mon Oct 17, 12:52:00 PM EDT  

Anonymous Anonymous said....

Referring to jsndba's java coder:

Perfect example of why you thoroughly interview and tech out a potential employee/consultant. Let them learn the hard way by hitting them in the pocket.

Fri Nov 03, 12:58:00 PM EST  

Blogger Ayisha said....

Nice and knowledgeable gifts for everyone-
booksshelf
knowledge
books
liberary
kitaben
Books and references
books

Fri Aug 24, 09:31:00 AM EDT  

Blogger Dana said....

Gee, why doesn't software do what we WANT it to, instead of what we TELL it to do? :-)

Mon Mar 10, 07:50:00 PM EDT  

POST A COMMENT

<< Home