Saturday, November 05, 2005

Storage

Remember when 32MB hard drives seemed literally huge?  10 years ago in 1995, Oracle 7.3 was production and Oracle was doing its first terabyte test to scale tests.  Back then, a terabyte was huge.  Back then, a terabyte was expensive, heavy and consumed a ton of power (needed to be in a data center, not something you could roll out on stage and demonstrate).  These days – a terabyte or more is virtually mainstream, many people have worked or work on systems of that size..

Remember when 1 or 2 MB pen drives were cool – now they come in GB sizes.

A couple of weeks ago, I bought some disk drives and had them delivered to my house.  On my dining room table we had 2.5 terabytes of disk.  Small enough that the table could hold it all, light enough that anyone could carry them into the house.  Delivered straight from amazon.com by UPS.  Power consumption – nominal.  Price – well under $1 per GB.  Things have changed.

I saw this the other day.  A 3.5 inch 100TB drive in the works.  It’ll be a while, but just think about what could be done with that.

I still find that no matter how big the storage feels when you first get it, I’ve no problems filling it up with stuff.  That first 32MB drive seemed infinitely large – but it got full.  My 300GB drives at home are getting filled with stuff.  I travel with 140GB of storage and had to erase two database instances to make room for something else at UKOUG last week (getting tight on space).  100TB though, that might last a while – at least a week or two.

Anyway, just landed at Heathrow from Dublin on my way home.  It took one hour to get from the plane at terminal one to my gate in terminal three. Originally, they had me scheduled for a one hour twenty minute layover – based on my past experience – I knew that was quite possibly too short – so I opted for a three hour layover.  I’d rather be relaxed than rushed.

Besides the Red Carpet clubs have wireless T-Mobile hot spots, it’s just like being at work (but the coffee is better).

Looking forward to getting home, Ireland was fun – got to see some old friends – but it’ll be nice not to sleep in a hotel for a couple of nights.
POST A COMMENT

22 Comments:

Blogger Doug Burns said....

Tom,

Have a good trip home. 2 weeks away is too long and hopefully there's someone at home who doesn't want to talk about Oracle ;-)

We've enjoyed having you over here.

Cheers,

Doug

Sat Nov 05, 04:53:00 AM EST  

Blogger Doug Burns said....

Oh, and by 'we', I mean the entire European Oracle user community ... obviously ;-)

Sat Nov 05, 05:40:00 AM EST  

Anonymous Anonymous said....

RAID is getting cheap enough where Dell is selling it in high end gaming machines. We may see in the near future alot of smaller devices that add up to alot of storage using RAID at a price cheap enough to have at home. My cable company just increased bandwidth to 6MB/sec and I know Verizon is putting in a national fiber network where you can get 15 MB/Sec at an affordable price. You might need a RAID type device to be able to write this fast and do other things with your PC.

INPHASE technology has a 200 GB optical storage device coming out next year. Its write once, but its not much bigger than a CD/DVD. They plan to have a 1.6 TB Read/Write device by 2010. They have not updated their roadmap on their page. I will post a link to a news story about their timelines.

The 100 TB thing you posted is interested, but they are not even in development yet. This is more imminent.

http://www.inphase-tech.com/
http://news.google.com/news?hl=en&q=inphase%20technology&sa=N&tab=wn

Has anyone heard anything about Terabyte Hard Drives? We have been at 500 GB drives for atleast 2 years now with no increase? I have seen storage companies focus on mini-storage devices, so maybe their future plan is as I stated, several hard drives using a cheap RAID connection for home use?

Sat Nov 05, 08:59:00 AM EST  

Blogger scubajim said....

Apple has SAN storage device something like 7 terabytes for $12K. A little out of my for home use budget, but reasonable for even a small company.

Sat Nov 05, 10:24:00 AM EST  

Blogger Robert said....

My 300GB drives at home are getting filled with stuff. I travel with 140GB of storage and had to erase two database instances to make room...

:O
OMG, what..how..you manage to use up ALL that space !?!? (assuming you're not into graphics, video editing/ripping....) do you ever delete anything ? lol

Sat Nov 05, 02:52:00 PM EST  

Blogger David Aldridge said....

Now, if only the bandwidth could catch up with the capacity.

Sat Nov 05, 04:48:00 PM EST  

Blogger Tony said....

I remember when I had a "huge" 200MB hard drive and had just purchased 4MB of RAM for $160 bucks. I thought my system was pretty hot. I took about four months before the hard drive was full.

I wouldn't even need 100TB, just 10TB would be a nice start.

Sun Nov 06, 09:47:00 PM EST  

Blogger Noons said....

Tom, I thought scanning my slides and negatives at 20Mb each was bad enough! I must admit running out of space on a 160Gb disk for demo dbs beats that.

But then again, I can quickly jack up the rez on my scanner to make a humble 35mm negative take a whooping 170Mb. At that rez, I'd need the 100Tb drive next week!

Quite frankly, I like the way disk capacity is going. I can find things already to fill up a few Tb and I haven't even started with home video!

Ah yes: somewhere in there my test Oracle databases need to fit as well. A mere few hundred Gb at this stage.

Makes one wonder where things are going: we just upgraded our main data centre to around 10 Tb capacity and I can already put a date on when it will be needing upgrade.

Jim Gray from Microsoft research is absolutely right: we're gonna need something better than just plain old indexing to be able to effectively manage and leverage all this data!

Sun Nov 06, 11:41:00 PM EST  

Anonymous Anonymous said....

I think that no matter much disk space you there is, people will always want more.
Q. Do your slides/pictures need to be 20mb each?
A. No.
Q. Do your spreadsheets need Animated gifs?
A. No.
Q. Do you need to see the full accounts from the past 20 years in an online, multilayed reporting environment?
a. Probably not!

But we do these things anyway, just because we can!
"Something better than plain old indexing" ? How about "good design"???

Mon Nov 07, 09:15:00 AM EST  

Anonymous Anonymous said....

Has anyone heard anything about Terabyte Hard Drives?

LaCie offers one here:

http://www.lacie.com/products/product.htm?pid=10128

LaCie Big Disk Extreme with Triple Interface 1TB

USB 2.0, FireWire 400, FireWire 800

It's for $999, currently backordered due to increased demand.

Imagine! And I remember how excited I was when I bought my first 1 GB drive for under $500!

Bob Shepard

P.S. Anyone know how to turn the above into a "live" link?

Mon Nov 07, 11:32:00 AM EST  

Blogger Thomas Kyte said....

You use the

text to link to

html syntax. I have some LaCie drives - travel with their smaller ones. this one looks interesting (I have a bunch of seagate 300gb ones now)

Mon Nov 07, 11:39:00 AM EST  

Blogger Joel Garry said....

I went to show my 83-year-old father-in-law one of those newfangled keychain usb drives they gave away at OpenWorld.

But it's too small, I couldn't find it.

Mon Nov 07, 02:15:00 PM EST  

Anonymous Rob H said....

Whats interesting is that imagine walking into work with a ext hard disk or pen drive and walking out with an entire database.

Mon Nov 07, 04:09:00 PM EST  

Anonymous Anonymous said....

100TB? What is the point when Oracle XE only has 4GB... :(

Oh well, at least we can archive all those multimedia files, knowing that *if* we have time to watch them, we could more easily locate them!

Mon Nov 07, 08:27:00 PM EST  

Anonymous Anonymous said....

What is the point when Oracle XE only has 4GB...

But it has dblinks...

Tue Nov 08, 07:46:00 PM EST  

Anonymous Roderick said....

Just a couple of days ago, I was reminiscing with someone about a customer visit in 1995 where a customer was looking for a way to keep MTTR below 24 hours. Problem was this was a 600GB OLTP database and it took 30 hours just to do a full db backup to tape. Disk was triple mirrored, and 1.2TB was about all they could attach to one of the largest Unix servers available at the time. Now thanks to a very kind gentleman, I now have a 300GB drive sitting next to a 5 year old PC that's barely fast enough to play any new video games. The rate of progress in IT has been amazing.

Wed Nov 09, 03:18:00 AM EST  

Anonymous Roderick said....

Typo in my previous post. Should be 300GB not 600GB at a customer site. V6 defaulted to a 2k block size with 10k extent size and maxextents of 121 and for a few months anyway, everyone thought it was more than enough. So in 1989, I remember one customer bragging about having a 3GB Oracle database running on ... MS-DOS. I was blown away. Now I barely lift an eyebrow if I hear about a database being 10TB in size.

Wed Nov 09, 12:25:00 PM EST  

Anonymous Carl said....

Realising we could fit one of our production databases into one guy's mobile phone was quite freaky (Well, the compressed export file anyway!)
Even though I'm not very old I still find the concept of running oracle on a laptop disturbing.

You can get carried away though, I often hear the 'diskspace is cheap' motto, yes it is, but managing that space, bandwidth issues, backup and recovery windows etc do not mean that we can forget about data size as a constraint.
We have a 33Gb database which is nothing in modern terms but there is a big issue because of a requirement to perform an export/import of the data on a regular basis (Don't ask!)

Fri Nov 11, 10:25:00 AM EST  

Anonymous Anonymous said....

Hi Tom,

I have one question about lobs.
I have a table which consist of one clob column inline.
Lob segment is about 360 Gb.
Table is not partitioned, so everything is in one large lob.
We will be very short with disk space soon.
We archive old clobs to disk and update column to empty_blob.
But lob segment continue to grow.
I was thinking that new clobs will use claimed space.
What do you suggest ?

Regards,

Shone

Thu Nov 17, 12:02:00 PM EST  

Anonymous herodt said....

"Conspiracy Theory" coming up :)

Some software manufacturers must be owned by or own storage manufactures.

We recently went through an aquistion. "They" decided to change out our email systems to save a few sheckles on licensing per user.

Our email went from 1.5 terabytes for 2000 users, to 7.8 terabytes for - get this - 3000 users!

What type of a system keeps a copy of an item for every single record having access to that record?

In this case attachments - you send out an email to 1800 internal people with a 1 meg attachment. Our old system, 1 meg file with a pointer.

This new system 1800 1 meg database records. Guess what, more drives, more bays, more fibre connections , more backup tapes, more stress on our poor system admins.

AAARGH.

Yet, our financials system, for 11 years of data is 42 gb of oracle space.

Fri Nov 18, 09:48:00 PM EST  

Anonymous Anonymous said....

The best idea is to buy generic viagra online for your problems

Sat May 06, 07:01:00 PM EDT  

Anonymous document storage said....

If you think about it, today's terabytes will be considered small after 5 years. Technology made a lot of leap in storage and will still continue to.

Mon Jan 16, 04:52:00 AM EST  

POST A COMMENT

<< Home