www.infinidb.org

I'll be presenting "The Thinking Person's Guide to Data Warehouse Design" at the upcoming MySQL User conference. While a lot of people think that bad SQL code is the #1 wrecking ball of data warehouses and marts, the fact is that poor database design is the first cause of both downtime and bad performance. In my presentation, I'll do my best to show how up-front work in a data model pays off and how to take that model into a MySQL physical design, with topics like scale-up/out designs, storage engine decisions, partitioning schemes, indexing issues, and much more being discussed. I'll then wrap up with tips on monitoring and tuning of the design.

Hope to see you there!

Comment (0)

I was privileged to lead a MySQL University session last week on MySQL-based column databases - what they are, how they work, why you should use them, cases where you shouldn't use them, benchmarks showing their performance, and more. I covered the main MySQL column databases - Infobright, Kickfire, and InfiniDB - and discussed the various features, benefits, and limitations of each. You can find the slide deck and the recording of the presentation complete with the slide presentation at: http://forge.mysql.com/wiki/MySQL_Column_Databases.  Let me know what you think and if you have any questions about what I presented.

Comment (0)

Right now, we support InfiniDB on a couple of different Linux flavors. As we march toward a couple of more releases this year, we want to expand our operating system support. We could really use your input on which OS's we should support next. Please go to http://www.infinidb.org/index.php. On the lower left corner you'll see our OS poll - please vote and let us know which OS you use for your production databases. Thanks!

Comment (1)

Last week, I pointed you to a paper I recently wrote on how to use InfiniDB with Microsoft Excel. Of course, I realize there are plenty of people who don’t use Microsoft office tools and use OpenOffice instead, so I’ve just posted another paper on how to use InfiniDB with OpenOffice Calc (OO’s complement of Excel) and Base (OO’s twin of MS Access). FWIW, I found OpenOffice easier and more straightforward to use with InfiniDB and other RDBMS's than Microsoft (and I'm pretty much an MS Office poweruser).

Again, you’ll have to register on our .ORG site to download and read the new paper, but again, don’t worry about us hitting you with a bunch of emails or anything else – we just like to know who’s interested in what on our site.

If you’d like to see other papers written on how-to-use-infinidb-with-, please let me know. And thanks again for your support of InfiniDB.

Comment (0)

A recent survey found that the most used BI tool isn’t what you might think (e.g. Business Objects, Pentaho, Cognos, Jaspersoft, etc.) – it’s Microsoft Excel. And I suppose that’s not so surprising given spreadsheets are the main communication medium for financial and general business analysis.

If you use Excel in this manner, you should know that you can use Excel’s data import capabilities to directly query (via the MySQL ODBC driver) and pull data from InfiniDB into a spreadsheet. But there is one thing you should watch for: for whatever reason, when building a join between two or more tables, Excel’s table/column selection wizard will send a Cartesian join statement to MySQL as it steps through its process. With InfiniDB, we disallow Cartesian joins because (1) most Cartesian joins are a coding mistake; (2) a large Cartesian join that’s executed can kill performance on a database.

So if you want to use Excel and query InfiniDB, you need to cancel out of Excel’s table/column selection wizard when constructing a join, and instead use Excel’s query editor. With Excel’s query editor, you can select tables, visually do your joins, and select the columns you want with no Cartesian join statements being sent to the server.

I’ve written a new white paper on using InfiniDB and Microsoft Excel that’s filled with step-by-step examples, which you can download and read. Given that it’s a white paper, we do require registration on infinidb.org to get it, but rest assured we won’t give out your email address or spam you with correspondence.

Thanks again for your support of InfiniDB!

Comment (0)

One aspect of my positions with Calpont, MySQL, and other companies I’ve worked for has been to interact with the various analysts who cover the database scene. It’s definitely an interesting part of the job, especially when you get to query some of the experts who have been around a long time and are good at making solid technology calls on where things are headed.

IDC is one such group of experts and they recently released an interesting set of predictions on where databases – and specifically data warehouses and analytic DB’s – are going. You can read the quick update they issued HERE.

A couple of their ‘within 5 year’ predictions I found particular interesting for MySQL data warehouse users are:

• Most data warehouses will be stored in a columnar fashion
• Most large-scale database servers will achieve horizontal scalability through clustering

This is helpful information for MySQL users who want to ensure they’re making the right bets on their data warehouse designs and the storage engines they’re choosing. It’s also good news because MySQL users now have a choice of column databases they can try out and see which is right for them.

I’ve written a new article on the MySQL dev zone on why you should check out a column database if you’re planning to implement a data warehouse, reporting database, or any read-intensive application. It talks about the why and why not’s of column DB’s, discusses why they matter, and provides some interesting benchmark tests of a leading row database vs. InfiniDB.

You can find the new article HERE. Please let us know what you think and what plans you have for your data warehouses.

Comment (0)

We’ve had a good and successful launch of InfiniDB 1.0 (Feb 1), and while we’re pleased with that accomplishment, we’ve now turned our attention to the next release of InfiniDB, which will be 1.1. We’ve got some nice new features, capabilities, and changes planned for you that aren’t very far away. In addition, we plan on backporting some of these changes to a 1.0 patch release so current users can benefit from them now.

You can find the new, updated roadmap available here. You’ll find each new committed feature briefly described, features we’ll implement if time permits, and a schedule showing what new builds will contain what new features, along with when our beta and RC cycles are expected. If all goes as planned, the FINAL InfiniDB 1.1 version will be released the week of July 1.

We’re committed to a release early/often framework, and we look forward to your participation in our upcoming release. If you have any questions about our plans, please let us know. Thanks for your support of InfiniDB!

Comment (0)

We're getting quite a few people now using InfiniDB and the top open source BI tools. As we're not experts in these tools ourselves, we are looking for help to do product viability testing to determine what issues (if any) we need to correct in InfiniDB to properly work with these tools. If you know one of the tools below well and are interested in tackling this type of project, we'll certainly compensate you for the time you spend. Contact me at rschumacher AT calpont DOT com.

• Pentaho
• Jaspersoft
• BIRT

Comment (0)

Are you interested in working with an outstanding development team that is creating some of the finest database technology that’s out in the analytic/data warehouse marketplace? If so, let us know because we’re hiring! We’re continuing to grow our team and are in need of the following:

Senior Developer

This position will work directly with our current engineering team in developing our database server. To be considered, you’ll need the following:

• Strong C++ skills in a Linux environment, ideally also in Windows.
• Experience with SQL databases.
• Thorough understanding of MPP engineering techniques.
• Strong OO design and engineering skills.
• Good team player.

Senior Support Engineer/Manager

This role will take the lead in ensuring our customer’s technical support needs are taken care of.  For this spot, you should possess the following skills:

• Experience with SQL databases and ideally experience with MySQL.
• Top-notch technical troubleshooting capabilities (receive, diagnose, and help resolve technical problems).
• Ability to smartly interface with engineering and quality assurance teams to replicate and solve customer problems.
• Excellent customer interface skills mandatory.
• Quality assurance and data warehouse background helpful.
• Experience on key OS platforms (Linux, Windows, etc.)

Neither of the positions above requires relocation (the current team is located in Dallas, TX). And you'll be paid well, have a nice benefits package, and also get some equity in the company to boot.

Interested? If so, shoot a note with your background information to rschumacher AT calpont DOT com.

Comment (0)

We received some nice feedback on our care and feeding of InfiniDB blog entry, and we appreciate all of you who were kind enough to respond. We did fail, however, to communicate a few other intentions we have regarding how we plan to release and label the InfiniDB software so here’s some more thoughts from us on this important matter:

For new releases, we plan to follow the traditional alpha, beta, RC framework. Alpha means an upcoming release is not yet feature complete and more features are coming; beta means an upcoming release is now feature complete and we are now only in bug-fix mode; and RC means we think we’ve got the quality right for the new release but may still have to iron out a few last wrinkles. Of course, a new release is developed in a separate and different branch of our codeline than the maintenance done on our most current FINAL version. Anything fixed in our most current FINAL version will naturally be included in the upcoming release as well.

With respect to how we will number our releases, we plan on using the standard MAJOR.MINOR.PATCH numbering scheme. How the numbers are incremented are as follows:

For a PATCH release, these are just incremented normally as new bug fix builds are produced and released to the community. Again, our goal is to have monthly builds for the community for our most current FINAL version and quarterly service packs for the commercial version.

A new release will contain a MINOR number increase when one of two things (or both) are true: (1) when enhancements are made to already existing functionality that shows marked improvement in how the existing feature functions or performs; (2) new features are added that are not large scale in nature, but enhance the overall product’s capabilities. So, for example, in our next release, we will deliver high-speed subquery support for InfiniDB. You can use subqueries today in InfiniDB by setting our vtable mode to pass everything to MySQL, but trust me: you don’t really want to send subqueries against TB’s of data and billions of rows directly to MySQL for processing unless you plan on waiting a while for your response. Because we technically do have subqueries now, the addition of high-speed subqueries would qualify for a MINOR number bump. Also – whether it’s a single number bump (e.g. 1.0 to 1.1) or multi-number bump (e.g. 1.0 to 1.5) - will depend on the number of additions added to the new release and their sophistication level.

A new release will contain a MAJOR number increase when one or more large-scale new features and functionality that do not currently exist are added to the product. These new features allow brand new things to be accomplished with the server that currently weren’t possible or required major workarounds. Of course, a new MAJOR number release will likely contain enhancements to existing functionality and small new features. The vast majority of the time, a MAJOR number bump will be sequential (e.g. 1.0 to 2.0) but it is possible to go more than one level between releases – such a thing is rare but does happen (e.g. Sybase went from version 12 to version 15). As an example, today we formally support a shared-disk configuration for our MPP commercial version. While you can do a shared-nothing setup, we don’t recommend it for production nor support it as the server currently does not have built-in redundancy and failover between shared-nothing nodes today. When we do formally deliver shared-nothing with the appropriate feature set, this could qualify for a multi-MAJOR version bump (e.g. 1.5 to 3.0). We certainly don’t want to artificially inflate MAJOR version numbers so, again, this type of multi-number bump will be rare and most times simple one digit sequential increases will be done.

So that’s our current thinking, but we’re certainly open to your thoughts and will eagerly accept better ways of doing things, so let us know what you think. And thanks again for your support of InfiniDB.

Comment (0)