InfiniDB load rate trended through 23.9 billion rows
Posted by: jtommaney
on Dec 09, 2009
Just a quick graphical presentation of the load rate with InfiniDB importing 23.9 billion rows in about 12 1/2 hours. Rows per second is just north of .5 million rows per second.
Your milage will obviously vary based on row width, disk configuration, etc., but the expectation would be that the load rate is stable over a very wide range of cardinalities.

This table was created and optimized as the fact table for a star schema data model originally implemented on a popular row based dbms. As a result, this table may be narrower than some others. The import used the default cpimport (bulk load) parameters, so 4 total threads are allocated for the bulk load process. The total seconds were 45,247 and 23,963,603,232 rows were loaded in 6305 individual jobs.
create table pos_day
(
period_id int,
location_id int,
product_id int,
upc_product_id int,
adj_first_order_date_id int,
recmd_item_flag char(1),
new_item_flag char(1),
promo_activity_code char(1),
item_source_code char(1),
sales_unit decimal(7,2),
sales_retail decimal(7,2),
gross_profit decimal(7,2),
promo_unit decimal(7,2) ) engine=infinidb;
Example syntax to import:
/usr/local/Calpont/bin/colxml point_of_sale -t pos_day -j 1129
/usr/local/Calpont/bin/cpimport -j 1129
Example load data: 6306|5139|124910|98548|1009|Y|N||I|1|0.99|0.41||
Let us help you put your data to work, and also what additional features are of interest.
Thanks - Jim Tommaney
