Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

DataObjectManager Module /

Discuss the DataObjectManager module, and the related ImageGallery module.

Moderators: martimiz, UncleCheese, Sean, biapar, Willr, Ingo, swaiba, simon_w

Memory Issue with Sortable Data Objects


Reply


3 Posts   614 Views

Avatar
elgordo

Community Member, 70 Posts

20 June 2011 at 8:58pm

I've recently being trying to create a large SilverStripe site by importing all of my Flickr photographs as DataObjects (around 60000). This is primarily so I can do things like database queries on the Exif data, e.g. find all photographs taken in the morning with an aperture of less than f8.

My import process kept crashing with a memory issue, and the symptom I was seeing was memory jumping from 29M to > 128M with the saving of a single image. I eventually tracked it down to the onBeforeWrite method of File, and looking through the DataObject decorators, the one that was failing was SortableDataObject.

The problematic method is below:

public function onBeforeWrite()
   {
      if(!$this->owner->ID) {
         if($peers = DataObject::get($this->owner->class))
            $this->owner->SortOrder = $peers->Count()+1;
      }
   }   

The line

         if($peers = DataObject::get($this->owner->class))

is loading *all* of the DataObjects of a particular class (in my case, Image), and then setting the SortOrder of the new one to be one more than the total count of these DataObjects. As a result as a site grows, more memory will be used up when saving a new DataObject, eventually resulting in a save becoming impossible due to memory constraints.

Cheers

Gordon

Avatar
elgordo

Community Member, 70 Posts

20 June 2011 at 8:59pm

I've registered this as a bug on github, https://github.com/unclecheese/DataObjectManager/issues/11

Avatar
elgordo

Community Member, 70 Posts

21 June 2011 at 3:06pm

I've now imported 26000 Dataobjects representing flickr images with no apparent memory issues, the import process is sitting around 30M of memory use.