Wednesday, May 04, 2005

Quality

As for you working on the second edition, I would think it would be very tough to not edit almost everything in the previous edition. Even if it was perfect to you back then, you're not the same man you were then. I know I have that trouble with my writing. A suggestion - In your next blog will you cover something related to quality and processes. What you follow and make your team follow to ensure you get top most quality. For e.g. - Are you very harsh on people who do not indent their code properly.

While working on the second edition, I am mostly pleased with what I did over four years ago, it has held up well. And due to the plethora of examples in the book - it is very much self correcting. One of the problems a second edition like that could seriously run into is that the author knew the first time how something worked and as they are reading it - still believe it to work that way. Only it doesn't and they don't catch that -- because they didn't demonstrate it. Things change. It does makes it hard to plot how long it will take - because what seems like "this will be a simple couple of pages" turns into an entire afternoon.

Case in point, in Expert One on One Oracle, pages 177-180 I describe this phenomena "delayed block cleanout". Then on page 191, I come back and show how it can trigger an ORA-01555. Only it doesn't anymore. I mean, it should in theory - but I cannot get it to anymore. I try everything - a 2 MB undo tablespace (I wanted to use automatic undo management because I believe we should be, I dig that feature a lot) of fixed size. A 4 MB buffer cache (about 500 blocks). An 8,000 block table - update the first row on every block (guaranteeing myself that I'll have delayed block cleanout). Open a cursor but don't fetch. Do the small transactions off to the side - overwrite the undo many times. Fetch a row and.. bummer, it works. It still works (which means it succeeds, but I want it to fail) if I do the transactions in another session. It just keeps working and working. I cannot get it to 1555. Wrote to Jonathan Lewis to get some ideas. Finally - got it, do the above, dirty the table, open the cursor and then in FIVE other sessions do some work, come back to the original session and fetch a row - bam, ORA-01555 (never was so glad to see one in my life!).

Turns out in 10g, they've changed some things - ORA-01555's will be rarer and less frequent - the cases where they come about due to delayed block cleanout are fewer - takes more things happening at once (so they are not gone). But, had I just described a phenomena and been done with it - and not had an actual method that said "if you do A, B, and C you'll see D" - the book would have been in error. So many of the examples are like that.

So yes, working on the second edition is tough. I've found one or two technical inaccuracies in the book (poorly phrased sentences, in hindsight I know what I mean but it could be read wrong). They are getting corrected but I'm sure new ones are being added. And, so much has changed. Many new background processes. Memory management is radically different.

But the "quality process" I follow is "it isn't so until I see it". I know I'm going to have really smart people reading this under a microscope, I cherry picked my review team this time - so peer review is big in this 'quality' thing. Criticism is something that has to be accepted willingly and freely. Oh - I'll put up my side of the argument when appropriate (don't worry about that) but definitely concede the point more often than not. And the criticism can be brutal, but you cannot take it too personally. Actually on the technical material, I tend to not take it personally at all. You can say "that's not a very smart way to show that", "that's way overly complicated, you should do it like this", "here is a better way to show that", "that is just totally wrong and here is why" (and believe me, they do, I do myself when reviewing - you cannot just say "that is totally wrong" though, you have to follow it up with "and here is why")

So, I think quality is about going the extra mile to verify that what you say or do is mostly "correct". Off the cuff work is the low quality stuff - I think the stuff we print and publish needs to take a long time to produce (must be tested), must be peer reviewed (that is what I love about asktom - that is a peer review by 10's of thousands of people), must be criticized. You would think for example that text that was verbatim from the first edition of the book - already printed and accepted - and still true would be sacrosanct, beyond reproach. Not a chance, it receives as much criticism as the rest. Chapter Two - over 100 comments (criticisms) and it was small (13 pages, boy, I cannot wait for the Tables chapter to come back - at 74 pages)

On the "Why" blog - I received this comment:

How to "Question authority" - Without screwing up your appraisals?

That is a clear indication that there is a less than quality process going on there. If you fear questioning authority, that is a big problem. Anyone in a technical environment who says "Because I said so" (caveat, as a parent of a 12 and 9 year old, we are of course exempt from this rule applied to our children until they are living on their own!) or "I don't need to justify why", "I don't need to show you why, I'm experienced, just do it the why I say" - well, they are full of it. Run, don't walk, to the next project that comes along. That is not someone you want to learn from. I must admit, I've been rather lucky that way - I've always had good leadership to learn from, leadership that not only was willing to be criticized, but would say "ok, now, what's wrong with our plan".

I wish every web page had a link on the bottom that let us add a comment -- anonymously or not. Every web page. There are so many out there with just provably wrong statements on it that I'd love to leave behind a 3 or 4 line snippet of something that shows "no, that isn't quite right -- or isn't always right -- or is never even close to right".

Quality isn't something you get the first time all of the time, only through incremental refinement. Quality in our business, if I had to boil it down to an equation, might be:

Q = the SUM of ( hard work + peer review/criticism ) over incremental refinement

meaning it is a summation of (work+review)+(rework+review)+(rework...)... but is a series that converges rapidly if your review team is good.


In closing - am I harsh on people that don't indent their code ;) Hopefully in person I come off less harsh than I must in some writing. Yes, I believe indenting is important but even more so are my two rules:

1. The subroutine (also known as a "method") must fit on the screen from top to bottom (the declarations are allowed to scroll off the top but the entire body must fit). Anything bigger than that is too big to digest. I work on a 1600x1200 and a 1650x1080 screen - if I cannot fit your code on screen from top to bottom, I just won't even look at it.
2. There had better be some sort of logic explanation/comment in there unless it is so obvious my mother could figure it out. That is why when 'tuning' a query, I want the 'question' - not the query. I want to know what the goal is and not be influenced by your past attempts. I want to know what it is supposed to do before I look at how you did it.
POST A COMMENT

22 Comments:

Anonymous Anonymous said....

Yup, I work for one of those "just do it because I said so" bosses and he is definitely not someone I am learning anything from. Unfortunately in a small city with 3 IT shops, there is not much room to move. I choose instead to publish solid tested and written proof that I know what I am doing and my peers respect me and value my opinion because of this. Eventually I will have his job!

Wed May 04, 08:58:00 AM EDT  

Blogger Ajay said....

if I cannot fit your code on screen from top to bottom, I just won't even look at it.
What about complex reports? I have some that run a single 700 line query, and thats after compressing them substantially by using a subquery factoring clause. They perform well, so there is no reason to break them down farther into views. As for complexity, once you understand the subquery (which are about half the query), the rest is obvious.

Wed May 04, 09:31:00 AM EDT  

Blogger David Aldridge said....

>> Anything bigger than that is too big to digest. <<

I found that getting a rotatable LCD screen was a work-altering experience. For so many applications (web, documents, code etc) it is much more natural to use a portrait screen, and it's really tough going back.

Wed May 04, 09:50:00 AM EDT  

Anonymous Anonymous said....

A review (on Connor McDonalds book list) of Expert 101 shows another way to ensure quality in Oracle projects

(quote)
... and the author's eyes on the cover following you around the room ...
(/quote)

you MUST provide good quality ;-)

Wed May 04, 09:52:00 AM EDT  

Blogger Jan said....

Quality... must be one of my pet peeves! I think coding today is done under the umbrella of "rapid application design", in other words, throw some code at the problem and see what sticks! Haven't seen development teams around here do code inspections for years!
I remember when I first was confronted by code inspections. That was 20 years ago at the Dutch Steelworks (Hoogovens). I was doing programming, Cobol, IMS. You had to start with creating a program flow (pseudo code) from the (very good) spec. Then you had a flow walkthrough. That meant sitting in a room with the development team lead, the analyst, another developer and a developer from the support team. Put the spec and the flow side by side and explain what you're doing and why. Quite intimidating in the beginning I tell you! Then after the necessary updates and more reviews, you got the green light to start the actual programming. Then once that was finished, you'd have do a code review...
Did I hate it in the beginning? Yeah! Did I love it after a while? Heck yeah! Did it improve my programming? A lot!! We would have so much less problems if this process was still being followed during development. But everything has to be done in a hurry, so we don't have time for that. Of course, it will all come right back to you in the form of bug fixes and lots of patches, but I guess the powers that be don't care about that...
Getting off my soapbox now...

Wed May 04, 10:14:00 AM EDT  

Blogger Thomas Kyte said....

What about complex reports?

Well, I was talking about procedural code -- but the same concept applies with SQL.

You used subquery factoring. Guess what each of those subqueries is -- to me, the logical equivalent of a "subroutine" -- each:

with X as ( query ),
Y as ( query ),
Z as ( query )
select ...
from x, y, z


each inline view should be semi-digestable, each with should be.

I take big queries and break them down like that.

You can sort of see it in the way alot of my queries are presented:

select ...
from
(
select ...
from
(
select ...
from
)
)
)

each layer building on the next -- the WITH factoring helps alot.

and the most important thing in front of a big query, a big comment :)

I found that getting a rotatable LCD screen

I'm spoiled now, my laptop is
1680x1050 -- short but wide
and I connect it to a 21" lcd and stretch the display onto
1600x1200 - tall and skinny

I have the identical setup at home and work -- perfection. And on the road (which is where I am much of the time), I have a screen that is capable of doing actual work ;)


Quality... must be one of my pet peeves!

We talked about this (code reviews) just recently here. You are not alone in your wistful rememberances of code walkthrus.

Wed May 04, 10:23:00 AM EDT  

Blogger Peter K said....

How to "Question authority" - without screwing up your appraisals?
Easy, make sure that you have solid evidence to back up your claims and that they are not far-fetched.

I have an employee who told me that it would take him a whole two weeks to install and create an Oracle database. When asked on why, his response was a suggestion that I read the Oracle manuals especially the installation guides. Now, that's not THE WAY to question authority nor is that the way to get your point across.

Wed May 04, 10:59:00 AM EDT  

Anonymous Anonymous said....

I don't have a problem with quality processes but don't like the way it's followed. Lot of times code reviews are done by finding out where code is not aligned and then filling up a form. I think its done because that is the easiest thing to find. I try to break my code while testing and want the testers to do the same. But I normally don't believe code reviewers :-)

Wed May 04, 12:08:00 PM EDT  

Blogger bernice said....

Peter - I would agree with your coworker if ...
it's his first time installing the version of oracle on the flavor of operating system. Main reason is spending the time researching on metalink to see if there are any bugs and necessary patches (security and patch bundles) that need to be installed after installing the base oracle release. In addition, need to researching the patches that needed to be applied on the OS (I have to do that myself here also).
That is what I usually do - I dedicate about one week to research and read. Then, second week to install and patch. It doesn't mean I will take all 2 weeks. But it's a buffer I would provide to my boss.
But for subsequent installs of the same setup, it would be an one-day job (or less since I have everything documented and scripted).

Wed May 04, 12:11:00 PM EDT  

Anonymous denni50 said....

Tom..

love your equation:

Q = the SUM of ( hard work + peer review/criticism ) over incremental refinement

As you hone and polish your skills
and knowledge the degree of quality
increases exponentially.

I believe most have the inherent
intention to produce work that is
of good standard and good quality..however the degree of quality is relative to an individual's current level of skill
and knowledge.

In my case I look back on code I
wrote 3 years ago and say to myself
"what the heck is this?" and have
rewritten alot of old code. As my
skills have increased so has the
degree and level of quality.

Someone with more expertise and skill may judge another's work as substandard and inferior..therefore of no quality,
when in fact the work produce was
the best quality that that particular person was able to produce at that particular stage of
their development.

Flowcharting:
In today's terms: Data Flow Diagrams...is this done anymore?

Back in the 80's no code was written until a flowchart was produced and reviewed by the Data Processing Mgr and Systems Analyst.

The entire system's logic had to be
presented to the dept heads before
any programmer could begin the actual programming.

Wed May 04, 12:27:00 PM EDT  

Blogger David Aldridge said....

PeterK

>> I have an employee who told me that it would take him a whole two weeks to install and create an Oracle database <<

Do you work for DISA?

Wed May 04, 01:26:00 PM EDT  

Blogger Peter K said....

Peter - I would agree with your coworker if ...
it's his first time installing the version of oracle on the flavor of operating system.

Maybe but I wouldn't let a junior DBA do an installation from scratch but would have him/her shadow a senior DBA the first time around. In this particular case, the DBA claimed to have done 14 installations. Well, that shouldn't take more than 3 to 4 days then. :D

Do you work for DISA?
Nope. :D Now that I've peed off the Cherry Sisters, I wouldn't want DKB to be phoning my employer demanding that I be re-assigned.

I don't have a problem with quality processes but don't like the way it's followed. Lot of times code reviews are done by finding out where code is not aligned and then filling up a form.
That more or less speak to a lack of understanding and training for the QA folks. Code reviews are done by peers who have an idea of the function of the code under review and the purpose is to catch undesireable consequences. How many times have you seen comments in code that stated Don't understand what the following section does but removing it causes the program to not function? When I review code, there several things to look for including (a) meaningful names are being use, (b) code is self-documenting, (c) all conditional statements will be executed, (d) error handling are enabled for all possible conditions including a catch-all with the default being an error condition, (e) no buffer overflow problems, and (f) code is efficient.

Code review is not meant to be a vehicle for critism but rather a process for ensuring that the code function as intended and that it function efficiently and properly.

Wed May 04, 01:44:00 PM EDT  

Blogger Christopher Beck said....

I know all about Quality and Criticism from Tom. I have worked with him for most of the last 14 years. He was my first boss out of college and for the last 10 year, I have worked with him here at Oracle. Let me share a little story with you about Tom, quality, criticism and software development.

I was a junior analyst, fresh out of college, working on my first assignment in Ada. It was a govt contract, and Tom was my boss. I had just gotten the program working and was excited to show him. So I call Tom over to my desk and told him it was done. I gave him directions on how to run it. "start the program, input A, return, input B, return, input C return" and it would work great. So Tom sit down at my xterm, runs the program, types A, return, types B, return and then drags his hand across the keyboard and hits return. *Segmentation Fault*. Tom gets up, looks at me, smiles and says, "Its not done." and walks away. “But, but… it works great with the proper inputs.” *sigh* I was crushed.

Some people would have been upset with Tom for that but I was not. I was mad at myself for not considering bogus input as a possibility. But I learned. I learned that it is important to write quality code, and code that is complete. It’s not enough that it works, but that it works all the time. I learned that critical peer review is important. I learned you have to accept it as professional criticism, and not take it personally. It was bit of a harsh way to learn that lesson but in the end, made me a better professional; one with a thicker skin when it comes to criticism. Today I welcome the reviews and criticisms of my peers. I am still crushed when they find a bug or a better way to do something, but hey, that’s all a part of learning.

So, I spent the rest of that day bullet proofing that program and when I went back to show Tom this time, it worked as designed and he was unable to break it. Victory was mine! And from that day on, I strived to make sure my code was 'Tom Proof' before I declared it finished.

And on a side note, yes it’s true, if code and SQL are not formatted properly and do not fit on a screen, they will not work. Period. But don’t ask me for a proof; it’s just that way cause Tom says so ;-)

Wed May 04, 01:53:00 PM EDT  

Blogger Thomas Kyte said....

I know all about Quality and Criticism from Tom.

hehehehehe (well, he was a bit sure of himself right out of college and all...)

Another story about young Christopher, way back when....

When he filled out the application for GRC, the company we worked at -- it was a very generic application. A programmer would fill out the same form as the guy working on the loading dock.

Now, most people applying for technical jobs, when getting to the part of the application that said:

"Are you will to lift heavy objects -- Yes or No"

"If you answered yes, how heavy of an object would you be willing to lift"

"If you answered yes, how often"


would skip it -- instead, he answered,

Yes, 100 pounds, frequently


And to this day, anytime a box or computer or monitor needs to be moved, I give Chris a call.....

Wed May 04, 02:05:00 PM EDT  

Blogger David Aldridge said....

>> Tom gets up, looks at me, smiles and says, "Its not done." <<

Well, at least he smiled.

Wed May 04, 02:27:00 PM EDT  

Blogger Christopher Beck said....

Hey, what can I say, Im not most people. I was not going to lose a job because the application was not completely filled out. I consider that a good thing. Notice how I put a positive spin on it. I have had 14 years to work on this spin. Tom, has never let me forget that 100lbs frequently checkbox ;-) Its all good fun.

And yea, his monitor is still sitting in his old office at GRC too.

Wed May 04, 02:35:00 PM EDT  

Blogger Joel Garry said....

if I cannot fit your code on screen from top to bottom, I just won't even look at it.

I evaluated a system once where whoever set up the standards wanted lots of white space. Lots and lots. Like one BASIC statement per page. Normally I would reject any arbitrary statement like "it must fit on a screen" in favor of some, any, standardization, but that system got rejected, based a lot on the ridiculous white space. But so did writing it custom in Oracle 3.

I have an employee who told me that it would take him a whole two weeks to install and create an Oracle database. When asked on why, his response was a suggestion that I read the Oracle manuals especially the installation guides. Now, that's not THE WAY to question authority nor is that the way to get your point across.


I worked for a place once where they would rcp all the (O8) executables and datafiles from one unix box to another and call that an installation. By the way, the boxes weren't identical. I'd rather the two weeks.

Yes, 100 pounds, frequently

I once posted some humor about how to read DBA job ads. The funniest part that no one would get was, that the one for a DBA who's duties would including pulling cable was an actual job listing, not one I made up (and I knew the place that posted it, it was real - and they eventually lost all their DBA's to a competing contractor).

I work for one of those "just do it because I said so" bosses and he is definitely not someone I am learning anything from.

My first boss was one of those, and many of the bosses and customers I've had do that. What you learn from them is how to deal with difficult people, patience and diplomacy. Certainly worthwhile skills (and still not something I'm particularly good at). A positive spin can be put on it - since this is the status quo at many places, people that don't or can't deal with it are not competition. Which leads to scarcity, and hence, value.

Working around roadblocks is a useful skill in itself. Often, they can be manipulated easily, by simply making a suggestion and then waiting a few days until they decide they thought of it.

Wed May 04, 03:00:00 PM EDT  

Anonymous Anonymous said....

Recently, I was asked to generate explain plan for each and every query in a module that is being developed. When I asked why, I was told that "we want to see how the queries are preforming like how many buffer gets the queries are doing". It also turned out that attaching explain plan of every query to the deliverable was part of a "quality checklist".

Wed May 04, 03:08:00 PM EDT  

Anonymous Anonymous said....

> The funniest part that no one would
> get was, that the one for a DBA
> who's duties would including pulling
> cable was an actual job listing,
> not one I made up (and I knew the
> place that posted it, it was real -
> and they eventually lost all their
> DBA's to a competing contractor).

Whereas I have spent the last 10 years working for small manufacturing companies which have chosen to keep their operations in the United States rather than outsouce to [low income country of your choice]. In such an environment flexibility, adaptability, and the willingness to do anything to get the job done are what counts. I would terminate the DBA who refused to help me pull a cable.

sPh

Wed May 04, 05:16:00 PM EDT  

Anonymous Anonymous said....

"I wish every web page had a link on the bottom that let us add a comment -- anonymously or not. Every web page. "

I seem to recall years ago there was some add-in that was developed for this purpose. Whenever you viewed a page, it would go off to a certain website and see if any comments had been logged on that URL. If they had, you'd get them in a pop-up.

Probably a lot harder now with dynamic generation of pages.

Wed May 04, 09:00:00 PM EDT  

Blogger Connor McDonald said....

You use subquery factoring

although rewriting a lengthy SQL to use with-query might end up with a dramatically different execution plan and a dramatically different execution time (maybe better... maybe worse)

Wed May 04, 09:00:00 PM EDT  

Blogger Thomas Kyte said....

although rewriting a lengthy SQL...

agreed, that is why many of my sql's must be read from the "inside out"

but I am using the "with" clause more and more. it is really cool for many classes of problems.

Wed May 04, 09:05:00 PM EDT  

POST A COMMENT

<< Home