Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

We've moved the forum!

Please use forum.silverstripe.org for any new questions (announcement).
The forum archive will stick around, but will be read only.

You can also use our Slack channel or StackOverflow to ask for help.
Check out our community overview for more options to contribute.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, Ed, biapar, Willr, Ingo, swaiba

Running out of memory creating db record with SortableDataObject


Go to End


7 Posts   1871 Views

Avatar
ec8or

Community Member, 18 Posts

13 March 2012 at 5:17am

Hi,

I'm running out of memory every time I try to create a new record for my Book class. This seems to be because I try to create a set of default BookPage records every time a new Book is saved, which causes SilverStripe to run out of memory:

Fatal error: Allowed memory size of 167772160 bytes exhausted (tried to allocate 91 bytes) in /usr/www/users/ukibum/sapphire/core/model/DataObject.php on line 2758

The first thing I tried was simply to allocate more memory but even with more than 1GB eventually the same thing happens (it just takes a lot longer). This only happens in production where I got more than 90,000 BookPage entries, I tried to purge the db and run the same code on a test server without any problems so it almost seems like SilverStripe is looping through all the records when creating a new one? At first I thought this only happened in Modeladmin but turns out the same thing happens if I try to create and write a new Book entry in any controller.

Not really sure if this is to be expected or what is going on so any advice would be greatly appreciated. Please check this sspaste for what should be all the relevant code:

http://www.sspaste.com/paste/show/4f5e2332a121e

The script times out while trying to perform the write on line 80.

Cheers

Avatar
swaiba

Forum Moderator, 1899 Posts

13 March 2012 at 6:00am

Edited: 13/03/2012 6:35am

Hi,

EDIT...

I think this may help in your loop http://www.silverstripe.org/general-questions/show/16988#post303671

Note you'll need PHP 5.3 to use garbadge collection.

Avatar
ec8or

Community Member, 18 Posts

13 March 2012 at 6:44am

Edited: 13/03/2012 6:47am

Thanks for your comments Swaiba.

I've removed the DOM stuff but still get the same error. It also happens when I try to create a new Book using a method on a custom controller:

class BookController extends Page_Controller {

...

    public function new_template() {
        if (Permission::check("ADMIN")) {
            $book = new Book();
            $book->Title = 'New book';
            $book->write();
        }
    } 

}

so doesn't seem to be Modeladmin related after all (I've changed the title of the post). That being said, there is another table with around 100,000 records with has_one relationships to both BookPage and Book so might be that. I can't understand why all those records are fetched every time I do a simple db write though?

Actually, it seems I can't even just write a BookPage record without running out of memory:

            $bookPage = new BookPage();
            $bookPage->write();

So something is definitely up... writing a BookElement works fine and is near instant.

Avatar
swaiba

Forum Moderator, 1899 Posts

13 March 2012 at 6:49am

yeah, it is being done in a loop though... or is it failing on the first write?

As in the post I've refernced there doesnt' seem to be any garbadge collection by default (it was only introduced in PHP5.3) so if you are doing a lot of new/write it could easily be that. Do some logging of the memory at that point if you are in a loop, then if this is the issue then I can only recommend the garbadge collectin.

Avatar
ec8or

Community Member, 18 Posts

13 March 2012 at 6:57am

Edited: 13/03/2012 6:58am

It is just the first write, no loop.

I've tried removing the onAfterWrite() method on both BookPage and BookElement as well, but no luck. So without the Book object being called this is basically what I've got:

<?php
class BookPage extends DataObject {

	public static $db = array(
        'Type' => 'Enum("cover, blank, title, copyright, back, content", "content")',
        'UniqueID' => 'Varchar',
        'Background' => 'Varchar',
        'MotivationText' => 'HTMLText',
        'Editable' => 'Boolean'
    );

	public static $has_one = array(
        'Book' => 'Book',
        'MemberBook' => 'MemberBook'
	);

    public static $has_many = array(
        'Elements' => 'BookElement'
	);
    
    public static $default_sort = 'SortOrder ASC, Number ASC';

    public static $summary_fields = array(
        'Type',
        'MotivationText',
        'Editable',
    );

    function onAfterWrite() {
        if (!($this->UniqueID)) {
            $uniqueID = uniqID();
            $this->UniqueID = $uniqueID;
            $this->write();
        }
    }

class BookElement extends DataObject {

	public static $db = array(
        'Type' => 'Enum("image, text")',
        'UniqueID' => 'Varchar',
        'PosX' => 'Int',
        'PosY' => 'Int',
        'Width' => 'Int',
        'Height' => 'Int',
        'FontSize' => 'Int',
        'FontColor' => 'Varchar',
        'LineHeight' => 'Varchar',
        'FontFamily' => 'Varchar',
        'FontShadow' => 'Varchar',
        'FontAlign' => 'Varchar',
        'ZIndex' => 'Varchar',
        'Content' => 'HTMLText'
    );

	public static $has_one = array(
        'Page' => 'BookPage',
        'Book' => 'Book',
        'MemberBook' => 'MemberBook',
        'Picture' => 'BookImage'
	);
    
    static $searchable_fields = array(
        'ID',
        'Content'
    );
    
    function onAfterWrite() {
        if (!($this->UniqueID)) {
            $uniqueID = uniqID();
            $this->UniqueID = $uniqueID;
            $this->write();
        }
    }

and here is the method that runs out of memory (on BookController.php):

    public function new_template() {
        if (Permission::check("ADMIN")) {
            $bookPage = new BookPage();
            $bookPage->write();
            echo "wrote: " . $bookPage->ID;
        }
    }   

Avatar
swaiba

Forum Moderator, 1899 Posts

13 March 2012 at 7:06am

maybe there is some kind of circular loop, saving the book?

find the line (deep int eh SS code) that errors and then add an SS_Backtrace::backtrace() before it (you'll have to find something unique about it or it'll be called tons of times...)

Avatar
ec8or

Community Member, 18 Posts

13 March 2012 at 7:25am

Edited: 13/03/2012 7:29am

Think I found it! BookPage was also in a SortableDataObject array...

            $bookPage = new BookPage();
            $bookPage->SortOrder = '4';
            $bookPage->write();

So when setting the SortOrder manually it all works fine, obviously leaving that blank is causing the DOM module to count all the BookPages in the db. Maybe this could be done in a more efficient way using raw SQL?

Will change the title again if anyone else runs into a similar problem. Thanks a lot for your time Swaiba!