Jump to:

3460 Posts in 1064 Topics by 739 members

Data Model Questions

SilverStripe Forums » Data Model Questions » DataObject::write() optimize memory usage

Moderators: martimiz, Sean, biapar, Willr, Ingo, swaiba, simon_w

Page: 1
Go to End
Author Topic: 2296 Views
  • imsas
    Avatar
    Community Member
    22 Posts

    DataObject::write() optimize memory usage Link to this post

    function write() dont frees all memory, then using large data inserting into database... Maybe some one try solve this optimization?

  • imsas
    Avatar
    Community Member
    22 Posts

    Re: DataObject::write() optimize memory usage Link to this post

    Im using curl connection ang inserting data into catalog, but I find out problem in my writed function CatalogData(). Im inserting ProductGroup() and Product() objects, memory usage looks good (can insert up to ~2 000 inserts and memony usage about 32MB), but then Im trying write ProductSpecification() its Product has many relation. Thats one cycle takes ~3mb memory usage and I cant find out how frees memory, thats happen whet I using write function....

  • imsas
    Avatar
    Community Member
    22 Posts

    Re: DataObject::write() optimize memory usage Link to this post

    Okey Im testing object insertation. Here memory usage when executing only controler and writing ProductGroup() 726 objects:

    -[ 21.763416MB iteration start
    -[ 71.006488MB iteration end

    -[ 71.0268MB iteration start
    -[ 77.80532MB niteration end

    -[ 77.826208MB iteration start
    -[ 79.442648MB iteration end

    -[ 79.4634MB iteration start
    -[ 93.414464MB iteration end

    -[ 93.434848MB iteration start
    -[ 95.415392MB iteration end

    -[ 95.436168MB iteration start
    -[ 95.559544MB iteration end

    -[ 95.58028MB iteration start
    -[ 95.604808MB iteration end

    -[ 95.625592MB iteration start
    -[ 96.061632MB iteration end

    Maybe have some ideas how to frees used memory back....?

  • dalesaurus
    Avatar
    Community Member
    283 Posts

    Re: DataObject::write() optimize memory usage Link to this post

    I think the easiest solution would be to break up your inserts (or just use plain SQL and mysql from the CLI).

    If you have 5000 entries and you can enter 1000 at a time, break the data up into 5 pieces.

  • Ingo
    Avatar
    Forum Moderator
    801 Posts

    Re: DataObject::write() optimize memory usage Link to this post

    Its hard to give you any advice without having reproducible code - I don't know what your CatalogData() method does, so can't say anything about its memory usage.

  • imsas
    Avatar
    Community Member
    22 Posts

    Re: DataObject::write() optimize memory usage Link to this post

    problem is here:
    $data - count ~ 10 000 entries. I use simple ecommerce module. I thinking write method searching is changed fields or something else.... When I debuging i see memory usage increasing by cycle iterations...

    foreach ($data as $item) {
    $catalog = new ProductGroup();
    $catalog->Title = $item['title'];
    ...
    $catalog->writeToStage ( 'Stage' );
    $catalog->publish ( 'Stage', 'Live' );

    foreach ($item['Products'] as $elem){
    $product = new Product();
    $product ->Title = $item['title'];
    ...
    $product ->writeToStage ( 'Stage' );
    $product ->publish ( 'Stage', 'Live' );
    }

    }

    DB::Manipulate() is one of the fastest data insertions in the database mode. Maybe I something dont know...

    2296 Views
Page: 1
Go to Top

Want to know more about the company that brought you SilverStripe? Then check out SilverStripe.com

Comments on this website? Please give feedback.