Database architecture: September 2007 Archives

Despite massive increases in capacity and major reductions in cost, today's high capacity disk drives and their decreasing effective transfer bandwidth will drive users to more efficient column stores. Contributor David DeWitt explains how this transfer bandwidth challenge actually makes it slower to fully scan a typical 10 TB data warehouse today then it did years ago. Continue reading "Disk trends will drive the need for column stores" »

In a recent post, a reader made the following comments: OODBs failed because RDBMS vendors added OO functionality; data warehouse users are not in pain; column stores only beat row stores on poorly designed warehouse schemas. In Mike Stonebraker's opinion, all three claims are false. Continue reading "Stonebraker comments on OODB market failures, data warehouse pain, and column advantages" »

In regards to an earlier post about the advantages of compression in column-oriented databases, a commenter asked about maintaining the correspondence between columns in a column-oriented database. Sam Madden discusses how the issue isn't really "getting the data back in the original order" as much as it is ensuring that the system has a way of accessing records in other columns that correspond to a particular record in a sorted column. Continue reading "Follow up on compression post: Columns, indices, and sorting" »

One of the key performance features of column databases that Mike mentioned in his previous post was the aggressive use of data compression. In this post, we'll discuss how column-oriented databases are able to more effectively exploit compression than a typical row-oriented system. There are two key points: 1) Columns of data from the same attribute compress better than rows of tuples in a table; and 2) A well-architected database engine using appropriate compression techniques can operate directly on the compressed data, without decompressing it. Continue reading "Good things come in small packages: The advantage of compression in column databases" »

Since the 1970s, new DBMS application areas have emerged with very different requirements than OLTP, but none of the major vendors have performed a complete redesign to deal with this changed landscape. Mike Stonebraker discusses how users can now experience blindingly fast data warehouse performance using column-oriented databases. Continue reading "One size fits all: A concept whose time has come and gone" »