InfiniDB load rate trended through 23.9 billion rows

Posted by: jtommaney

Tagged in: Untagged 

Just a quick graphical presentation of the load rate with InfiniDB importing 23.9 billion rows in about 12 1/2 hours.  Rows per second is just north of .5 million rows per second. 
Your milage will obviously vary based on row width, disk configuration, etc., but the expectation would be that the load rate is stable over a very wide range of cardinalities. 





This table was created and optimized as the fact table for a star schema data model originally implemented on a popular row based dbms. As a result, this table may be narrower than some others. The import used the default cpimport (bulk load) parameters, so 4 total threads are allocated for the bulk load process.  The total seconds were 45,247 and 23,963,603,232 rows were loaded in 6305 individual jobs. 

  create table pos_day
(
  period_id        int,
  location_id      int,
  product_id       int,
  upc_product_id   int,
  adj_first_order_date_id  int,
  recmd_item_flag  char(1),
  new_item_flag    char(1),
  promo_activity_code   char(1),
  item_source_code      char(1),
  sales_unit       decimal(7,2),
  sales_retail     decimal(7,2),
  gross_profit     decimal(7,2),
  promo_unit       decimal(7,2)   ) engine=infinidb;

Example syntax to import:
     /usr/local/Calpont/bin/colxml point_of_sale -t pos_day -j 1129
     /usr/local/Calpont/bin/cpimport -j 1129

Example load data:  6306|5139|124910|98548|1009|Y|N||I|1|0.99|0.41||

Let us help you put your data to work, and also what additional features are of interest. 
Thanks - Jim Tommaney



Comments (0)Add Comment

Write comment
You must be logged in to post a comment. Please register if you do not have an account yet.

busy