hm, you could try editing the fields directly in a utf8-savyy phpmyadmin, and see what comes out in the frontend. the urlencoding from tinymce (or our ajax-form-submission) might be a reason - does the same thing happen in normal varchar/TextFields?
We have the same problems with one of our sites. On our local test system everything works fine, but on the live server some characters aren't displayed correctly.
We will try another mysql server version (hopefully this week)...
Please post if you find a solution or workaround
@Ingo the same happens for title, menutitle etc.
Thanks for reply. I'm afraid I don't know how to edit the fields directly in phpmyadmin, tried though.
The incorrectly encoded strings are saved in normal varchar/TextFields too.
Possible reason of the issue:
In one of the Russian forums a user reported the same problem with TinyMCE unrecognizing one cyrillic character, though the string was saved in db correctly.
The user as well reported that he fixed the problem turning off the AutoEscaping in CGI.pm. AutoEscaping by CGI.pm is made with Latin encoding. AutoEscaping is no problem for all encodings, except for Unicode. In utf-8 it is "escaping" a certain part of the character, which leads to "unrecognized" characters in the output.
Could that be the reason? I've no idea how to deal with it. Where is CGI.pm on the server?
they're referring to another script in another language, silverstripe doesn't use CGI.pm. both php5 and mysql4 should be safe to handle utf8, we encode all our content in it (and didn't have problems in spanish/french translations so far). the problem happening in title/menutitle pretty much rules out tinymce as a culprit. i think looking for help in russian forums is your best shot, as its only two out of a dozen russian special characters, so a very specific problem
Ingo, they, actually, discussed another CMF. I just thought it coud be the reason. It's definitely not.
It looks like there's nothing in Russian forums about that. In Russia SS is not widely known yet. I couldn't find any Russian site running on SS.
Do you think the cause of the problem is somehow with db encoding? It's strange that the strings are not saved correctly, though the collation of the fields is set to utf8_general_ci.
Which script (file or module) in SS is responsible for saving the strings into DB?
Do you think it also might be some system templates that are not set to utf8 and causing the weird output?
The characters appear unrecognized after the edited content is saved.
And I tried to correct the strings in the db fields, after flushing everything, the whole content output in the CMS and on the site became unreadable.
you don't need to find russian silverstripe developers, just russians with knowledge about character handling in php5/mysql :)
I couldn't find the solution yet. If the site is on a shared hosting, there's not much that you can configure. I tried to create an empty db, then convert it to utf8 via phpmyadmin, then install SS. It doesn't help much. The server support guy said that I probably have to hack my php files (to insert mysql directives). Someth like:
mysql_query ("set character_set_client='utf8'"); mysql_query ("set character_set_results='utf8'"); mysql_query ("set collation_connection='utf8_general_ci'");
But to what file? I tried install.php before installing SS, works fine but no use.
Could you or anyone, please, look at my server variables after SS installation that are shown by phpmyadmin. This is a mess. Is there any ideas what I should do?
character set client utf8 (Global value) cp1251 character set connection utf8 (Global value) cp1251 character set database cp1251 character set filesystem binary character set results utf8 (Global value) cp1251 character set server cp1251 character set system utf8 character sets dir /usr/share/mysql/charsets/ collation connection utf8_general_ci (Global value) cp1251_general_ci collation database cp1251_general_ci collation server cp1251_general_ci