Doctors...

This recent blog post by Seth Godin reminded me a lot of my introduction to Effective Oracle by Design of a few years ago. What was true then is still so true today...

Here is an excerpt from my book that mirrors what he just wrote:

I will use yet another analogy to describe how this book will present information.  Pretend for a moment that the developer is instead a medical doctor and the application is the patient.  There are many types of MD’s:

• The emergency room (ER) doctor. They do “triage” – separating the hopeless from the ones that can be helped.  Performing quick fixes to keep patients alive for as long as possible.  They strive for short term band-aids to fix up the patient.  They will take a patient with a heart attack induced by smoking, bad diet and no exercise and get them stabilized.

• The operating room (OR) doctor.  They get the patient after the ER doctor has triaged them and patched them up.  They strive for long term fixes to keep the patient not only alive but as fully functioning as possible.  They perform the by-pass operation on that heart attack attempting to clear the arteries.

• The physical therapist (PT).  They get the patient after the operating room doctor is finished and begin a long and painful (not to mention expensive) process of rehabilitation.

• The preventative medicine doctor.  They strive to avoid the above three doctors at all costs.  They counsel the patient to quit smoking, eat a healthy diet, and exercise – developing a phased plan to get them in shape.  If they do their job right – with the exception of unfortunate accidents (like a car accident), the patient will never see the ER, OR or PT doctors.

Now, the world needs all types of doctors – accidents do happen after all.  But one of the most important types of doctors is that last one, the preventative medicine doctor.  The one that tries hard to avoid having their patient need the other three.

It is my belief (experience) that most people and books approach tuning using the mindset of the first three doctor types above.  They are in support of the hero developer; e.g. the ER or OR doctor.  Perhaps that is partially due to my observation that pre-emptive good design and implementation is mostly a thankless exercise.  These developers seem to get all of the fame as they snatch the patient from the grasp of death (save the system by doing something miraculous).  They get called in at the last moment; work horribly hard for an extended period of time trying to keep the patient alive (and get paid handsomely as well).  The physical therapists are the unlucky souls that get the system after the ER/OR doctor has patched it up.  They are the ones responsible for keeping this system going.

I feel I am well equipped to speak from that perspective.  I am in fact one of those “heroes”.  I am called in to “lay hands on” systems and make them better.  I could write that book, I’ve been told I should write that book – but I won’t.

What is missing is the comprehensive approach that includes the preventative medicine doctor training.  There are some out there – my favorites being Guy Harrison’s developer book, as well as Jonathan Lewis’s DBA book.  These books, including my own “Expert One on One Oracle” work to remove the need for the hero.  Remember – firefighters are heroes when they do their job, we all just hope we never need them personally!

When is a foreign key not a foreign key...

I learn or relearn something new every day about Oracle.  Just about every day really!

Last week I was in Belgrade Serbia delivering a seminar and an attendee reminded me of something I knew once but had totally forgotten about.  It had to do with foreign keys and the dreaded NULL value.

Many of you might think the following to be not possible, we'll start with the tables:


ops$tkyte%ORA11GR2> create table p
  2  ( x int,
  3    y int,
  4    z int,
  5    constraint p_pk primary key(x,y)
  6  )
  7  /
Table created.


ops$tkyte%ORA11GR2> create table c
  2  ( x int,
  3    y int,
  4    z int,
  5    constraint c_fk_p foreign key (x,y) references p(x,y)
  6  )
  7  /
Table created.

Looks like a normal parent child relationship - a row may exist in C if and only if a parent row exists in P.  If that is true - then how can this happen:

ops$tkyte%ORA11GR2> select count( x||y ) from p;

COUNT(X||Y)

-----------

          0

ops$tkyte%ORA11GR2> select count( x||y ) from c;

COUNT(X||Y)

-----------

          1

There are zero records in P - none.  There is at least one record in C and that record has a non-null foreign key.  What is happening?

It has to do with NULLs and foreign keys and the default "MATCH NONE" rule in place.  If your foreign key allows NULLs and your foreign key is a composite key - then you must be careful of the condition where by only SOME of the foreign key attributes are not null.  For example - to achieve the above magic, I inserted:

ops$tkyte%ORA11GR2> insert into c values ( 1, null, 0 );

1 row created.

The database cannot validate a foreign key when it is partially null.  In order to enforce the "MATCH FULL" option of a foreign key - you would want to add a constraint to your table:

ops$tkyte%ORA11GR2> alter table c add constraint check_nullness

  2  check ( ( x is not null and y is not null ) or

  3          ( x is null and y is null ) )

  4  /

Table altered.

That will ensure either:

• All of the columns are NULL in the foreign key
• None of the columns are NULL in the foreign key

As long as that constraint is in place - your foreign key will work as  you probably think it should work.

See also: http://docs.oracle.com/cd/E11882_01/appdev.112/e25518/adfns_constraints.htm#ADFNS273

UKOUG 2012...

The call for papers for the UKOUG 2012 conference ends in less than three short weeks!  If you were planning on going to the conference (and even if not) - you should consider submitting a paper.

I've been a long time supporter of all of the user groups and their conferences and I can attest to the quality of the UKOUG event.  The conference is chock full of technical talks with hundreds of sessions to choose from.  There is something for everyone there.

If you've never presented before, don't let that deter you from submitting a paper.  No one knows the anxiety that public speaking can bring better than I - I've written about it before. You'll find the conference to be an entirely different experience on the other side of the podium.  In addition to the experience of presenting, the networking and exposure that comes with being a speaker won't hurt you at all.  Whether you are a DBA or developer - having good public speaking skills is a necessity today - and using the conference as a way to build those skills is a great way to start.

Additionally - what you have to say is important and relevant to the user community as a whole.  A good conference needs a lot of speakers, from many diverse disciplines, with diverse backgrounds - the more speakers the merrier.  Don't think you don't have anything to offer - everyone does.  And don't feel that your topic wouldn't be interesting to someone else - it will be.  There are a lot of people out there trying to do some of the same things you've done and they'd love to hear how you did it.

That is one of the things about user groups I really like - they bring together a lot of people doing similar things - but in a different way.  You'll learn something new - and they will too.  

The UKOUG is one of the larger and well run conferences out there - don't be afraid to talk.  Challenge yourself to get up there and just do it.  You won't be sorry (ok, maybe in the minutes leading up to it you will be - but you'll get over that :) ) 

Hope to see you there - and don't chicken out!

Pokemon and When Others...

Question: What do Pokemon and When Others in PL/SQL have in common?

For many developers the answer is:

Asnwer: You gotta catch 'em all.

I saw this other blog post on "Dodgy Coder" and was greatly amused by the comparison of Pokemon and catching all exceptions and basically making them disappear.

I've written about this "worst" practice a lot.  You can see some of them at:

• http://tkyte.blogspot.com/2007/03/dreaded-others-then-null-strikes-again.html
• http://tkyte.blogspot.com/2008/06/when-others-then-null-redux.html
• http://www.oracle.com/technetwork/issue-archive/2007/07-jul/o47asktom-092692.html
• http://www.oracle.com/technetwork/issue-archive/2011/11-nov/o61asktom-512015.html (why you really want to let exceptions propagate)

You really DON'T gotta catch em' all!  And if you do - you really gotta throw them again (re-raise them)

Another debugging story...

There is a saying "correlation is not causation" meaning - just because you observe A and then observe B, it does not mean that A causes B - even if every single time you observe A - you see B as a 'side effect'.

Here is a great story to back that up - a debugging session where the end users had determined that they could not send email more than 500 miles.  Read it and note how much work the end users had gone into 'proving' that email cannot be sent more than 500 miles...

I use a similar story when talking about bind variables and bind variable peeking.  In the story - I make the claim that "when ever it rains on a Monday morning - you have to restart the database in the afternoon to make it perform correctly".  That is - rain on Mondays implies poor performance, and the only fix is to reboot.

This conclusion was reached via empirical observation - every single time, *every single time*, it would rain on Monday mornings - the database would respond with really poor response times to certain critical queries in the afternoon.  Every single time, without failure.  It must be that rain affects database performance somehow.

But here is what really was happening.  The database in question was a database that was backed up using cold backups (DBA's were of the archaic type).  These cold backups took place Sunday nights - late at night.  That meant that every Monday morning all of the caches would be virtually empty.  No parsed SQL for the applications in particular would be present.  So - Monday morning was the morning of hard parses every single week.

Now, in the critical application - there was a query that went against some very skewed data.  99.9999% of the time - the queries would be very selective, they would return 1 or 2 rows.  The other times - they would be very non-selective returning much of the table - but these queries were rare.

Now, the main user of this application was a person that came in early most of the time - they would get into the office at 6am and start working.  They always used bind inputs that returned very few rows - hence the problem queries would hard parse and tend to use indexes.  When this happened - everyone was happy.

The other set of users - they came in later, usually around 9am or so.  They would run the non-selective query and were OK with the index performance.  It might not have been the *best* possible performance - but it worked OK for them.

So, what was the link to rain?  Well, the office where everyone worked was located in Washington DC - when it rains in DC it doesn't matter if you leave home at 6am - it'll been noon before you get to work.  If you leave at 7am - it'll be noon when you get to work.  If you leave at 8am - it'll still be noon before you get to work.  In short - it really didn't matter when you left for work - if you are driving into the city - it is going to take a long time.  So, when it rained, the person that did the selective queries would just hit the snooze button on the alarm and go back to sleep.  Not worth driving in yet.

However, the people that did the non-selective queries - they lived in an apartment across the street from the  office.  They were not affected by the rain.  They came in at 9am regardless.  Since they ran the non-selective queries and the cache was empty - they would hard parse the queries and result with full scan plans.  This group of people was in fact a little convinced that rain *helped* performance a bit - when it rained on Mondays - they saw slightly better performance in the morning.

When the person that slept in finally got to work - and ran the application - they would definitely notice the full scans and how slow the database was.  Since it was raining - and they had observed this hundreds of times before - they call the data center - tell them "It is Monday, it is raining, you know what to do", after the reboot, everything is running fast again.

But - did it have anything to do with rain - or was it something entirely different :)

Getting to the root cause (bind peeking issue) can lead you to an efficient, effective corrective action (perhaps query plan stability for a certain set of queries for example).

Going down the empirical path of "correlation must imply causation" will lead to never actually fixing the problem - and the invention of many myths...