23450 Posts in 18899 Topics by 2877 members
Page: 1 2
|Go to End||Next >|
12 November 2009 at 1:58pm
I'm migrating a site to SS and I've used the CSV bulk uploader to get 6,000+ dataobjects loaded and that all works fine with no significant performance hit. The problem though is that each of those objects has 2-6 images associated with them. I've copied all the images to the assets folder and that automatically synced up fine and loaded all of the 16,000 File objects into the database but I cannot access the /assets section of admin and can not load any of the "select from file store" image selectors without hitting a PHP memory limit. I've bumped the memory limit in PHP up to 1gb and still no luck. I understand 16,000 files is a lot. But it shouldn't be impossible to implement this efficiently. Is there any workaround or known reason for this bottleneck?
As a note, the files are broken up into subdirectories something like this...
There are about 5,000 sub-directories inside CarParts, each containing a handful of images. Is it possible that having a certain ratio of sub-directories to files or a deeper nesting might at least let the asset manager load?
12 November 2009 at 11:15pm
This is a known problem, you can read up on :
For me, the latest stable version improved the situation somewhat , but I can't imagine it working properly with 10.000+ files.
12 November 2009 at 11:33pm Last edited: 12 November 2009 11:33pm
Good to know. So what this means is that you can not use Silverstripe for serious websites?
In the usenet discussion mentioned in the post above some people mention that you should be able to turn certain features off, for instance in the _config.php. Some of those things should be implemented in 2.3.x
Are there any recepies/code snippets/words of advice floating around?
13 November 2009 at 12:26am
>Good to know. So what this means is that you can not use Silverstripe for serious websites?
Well, I think that is a bit of an overstatement. Sure, there are some (performance) issues and no , if I'd be creating the new Facebook maybe Silverstripe would not be my weapon of choice, but all in all I think it is perfectly suited for large(r) websites.
IMHO, Having 10.000+ assets is not really an everyday use-case and I would love to see the problem fixed (since I have 1 site with the same problems as topic-starter).
13 November 2009 at 12:41am
Sorry for being a so negative, but it does trouble me.
The question is when the problem kicks in: does it already start when you have 5.000+ images? Or maybe even 3.000+?
And how are the images counted? How about a site with 3.334 images, who have thumbnails + larger versions + originals = 10.002? Or are we really talking about 10.000 images and the thumbnails and resized-images come on top of that?
13 November 2009 at 1:22am
We are running 1 failry large site with > 3000 files in the assets directory (including resampled etc.) and the site is starting to show slowdowns when accessing assets (either through files and images or by selecting an image through the content-editor).
The situation improved greatly with the 2.3.3 though.
13 November 2009 at 1:46am
The site is running on 2.3.3. Does anyone know the cause of the bottleneck, though? I only have 3 directories and 2 files in the top-level directory. It seems a little unnecessary to build the entire object tree in memory just to load this top-level directory. I'm 90% done the project now so it's unlikely I can scrap SS and go with something like symfony. Are there any ideas for a workaround? Maybe a way to exclude the asset manager from entering this directory?
13 November 2009 at 2:15am Last edited: 13 November 2009 2:17am
Fuzz10: "We are running 1 failry large site with > 3000 files in the assets directory (including resampled etc.) and the site is starting to show slowdowns when accessing assets (either through files and images or by selecting an image through the content-editor)."
So does this mean that this problem becomes noticeable at 1.000 - 1.500 images?... Which translates to a site with 500 product-pages/article-pages with 3 images per page ( and then add thumbs + larger images + original images)? And what about the cms previews and downloads like pdfs?
Page: 1 2
|Go to Top||Next >|