Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

DataObjectManager Module

Discuss the DataObjectManager module, and the related ImageGallery module.

Moderators: martimiz, UncleCheese, Sean, biapar, Willr, Ingo, swaiba, simon_w

Memory Issue with Sortable Data Objects


Reply

3 Posts   592 Views

Avatar
elgordo

20 June 2011 at 8:58pm Community Member, 70 Posts

I've recently being trying to create a large SilverStripe site by importing all of my Flickr photographs as DataObjects (around 60000). This is primarily so I can do things like database queries on the Exif data, e.g. find all photographs taken in the morning with an aperture of less than f8.

My import process kept crashing with a memory issue, and the symptom I was seeing was memory jumping from 29M to > 128M with the saving of a single image. I eventually tracked it down to the onBeforeWrite method of File, and looking through the DataObject decorators, the one that was failing was SortableDataObject.

The problematic method is below:

public function onBeforeWrite()
   {
      if(!$this->owner->ID) {
         if($peers = DataObject::get($this->owner->class))
            $this->owner->SortOrder = $peers->Count()+1;
      }
   }   

The line

         if($peers = DataObject::get($this->owner->class))

is loading *all* of the DataObjects of a particular class (in my case, Image), and then setting the SortOrder of the new one to be one more than the total count of these DataObjects. As a result as a site grows, more memory will be used up when saving a new DataObject, eventually resulting in a save becoming impossible due to memory constraints.

Cheers

Gordon

Avatar
elgordo

20 June 2011 at 8:59pm Community Member, 70 Posts

I've registered this as a bug on github, https://github.com/unclecheese/DataObjectManager/issues/11

Avatar
elgordo

21 June 2011 at 3:06pm Community Member, 70 Posts

I've now imported 26000 Dataobjects representing flickr images with no apparent memory issues, the import process is sitting around 30M of memory use.