3093 Posts in 875 Topics by 654 members
|
Page:
1
|
Go to End | |
| Author | Topic: | 1827 Views |
-
Dataobject insert about 4000 quearies
20 August 2009 at 9:11pm Last edited: 20 August 2009 9:17pm
I have a problem with inserting into the database. It should be at the same time to insert about 5000 records. I use ecommerce module. How to accelerate the insertion, as DataObjects takes a very long time ... For ongoing connection to the SOAP server, and information acquisition and storage of the ecommerce module. Received errors on the memory utilization, but hold it up to 128MB to avoid errors, execution time increases over the 300S.
Maybe you have a recipe?code sample:
function GetProductList($Name, $ID, $Parent){
$results = $this->cacheToFile('getResultsProductList', 3600, false, array($Name, $ID));
if (isset($results))
foreach ($results->GetProductListResult->ProductList->Product as $key => $item){
$isNew = DataObject::get_one('AccProduct', '`SupplierCode` = "'. Convert::raw2sql($item->SupplierCode) .'"');
if ($isNew){
unset($this->GetProductList->GetProductListResult->ProductList->Product[$key]);
continue;
}
//$Parent = DataObject::get_one('AccProductGroup', "`ClassId` = '$item->ClassId'");
//
if (!$Parent){
unset($this->GetProductList->GetProductListResult->ProductList->Product[$key]);
continue;
}
$product = new AccProduct();
//
$product->MenuTitle = $item->Name;
$product->Title = $item->Name;
$product->URLSegment = strtolower(trim(str_replace(' ', '-', $item->Name)));
$product->MetaTitle = $Parent->Title . " - " . $item->Name;
$product->MetaDescription = $item->Name;
$product->MetaKeywords = trim(str_replace(' ', ',', $item->Name));
$product->ProvideComments = 0;
$product->Sort = $this->Sort++;
$product->ParentID = $Parent->ID;
$product->Content = '';
$product->Price = $item->Price;
$product->Warranty = $item->Warranty;
$product->Quantity = $item->Quantity;
$product->FeaturedProduct = 0;
$product->AllowPurchase = 1;
$product->Weight = $item->GrossWeight;
$product->DateExpected = $item->DateExpected;
$product->RecommendedRetailPrice = $item->RecommendedRetailPrice;
$product->IsNewProduct = $item->IsNewProduct;
$product->SupplierCode = $item->SupplierCode;
$product->PartNumber = $item->PartNumber;
$product->Segment = $item->SegmentId;
$product->Group = $item->GroupId;
$product->Class = $item->ClassId;
$product->Series = $item->SeriesId;
// print "<pre>";
// //var_dump($this->getProductClassification($item->ClassId));
// //Debug::show($this->getProductClassification($item->ClassId));
// print "</pre>";
if (!$isNew){
$product->writeToStage('Stage');
$product->publish('Stage', 'Live');
}else {
$isNew->destroy();
}
unset($this->GetProductList->GetProductListResult->ProductList->Product[$key]);
$product->destroy();
$Parent->destroy();
}
//Sunaikinam gauta sarasa produktu
unset($results);
return true;
} -
Re: Dataobject insert about 4000 quearies
20 August 2009 at 10:32pm
Hiya,
I've had the same problem and found that the easiest thing to do was to just run it three of four times - until all the products are entered. You are already checking if it's new - and if you output some debugging information then you can also see which ones have been done.
I found it was quicker to just run it 3-4 times than actually re-write the code in different ways. You could, for instance, use MySQL's LOAD DATA INTO function to do it.
If you code a quicker way - and I'm sure there are quicker ways - please do let me know.
Cheers,
Josh
-
Re: Dataobject insert about 4000 quearies
21 August 2009 at 8:13am Last edited: 21 August 2009 8:20am
Throughout the day, trying to find out how to insert a large flow of records in the database. The decision is simple: insert without sitetree versions, and the stage version, and then the insertion is very fast ... But the versions of all hinders ...
I use:
$product->writeToStage('Stage');
$product->publish('Stage', 'Live');
When I use $product->writeToStage('Live'); and $product->writeWithoutVersion(); I get in sitetree table record with sitetree class and empty variables...
Maybe is other magic way.... -
Re: Dataobject insert about 4000 quearies
28 August 2009 at 12:45am
I dont try, but found solution insert/update, less memory usage becouse its array dealing soliution check here http://doc.silverstripe.com/doku.php?id=manipulation-arrays
| 1827 Views | ||
|
Page:
1
|
Go to Top |
