Jump to:

17452 Posts in 4473 Topics by 1971 members

Archive

SilverStripe Forums » Archive » Russian Unicode problem. What to do?

Our old forums are still available as a read-only archive.

Moderators: martimiz, Sean, biapar, Willr, Ingo, simon_w

Page: 1 2 3 4
Go to End
Author Topic: 72529 Views
  • Ingo
    Avatar
    Forum Moderator
    801 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    hm, you could try editing the fields directly in a utf8-savyy phpmyadmin, and see what comes out in the frontend. the urlencoding from tinymce (or our ajax-form-submission) might be a reason - does the same thing happen in normal varchar/TextFields?

  • SiSt
    Avatar
    Community Member
    5 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    We have the same problems with one of our sites. On our local test system everything works fine, but on the live server some characters aren't displayed correctly.
    We will try another mysql server version (hopefully this week)...

    Please post if you find a solution or workaround

    thx
    sist

    @Ingo the same happens for title, menutitle etc.

  • Craneway
    Avatar
    20 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    Thanks for reply. I'm afraid I don't know how to edit the fields directly in phpmyadmin, tried though.
    The incorrectly encoded strings are saved in normal varchar/TextFields too.

  • Craneway
    Avatar
    20 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    Possible reason of the issue:
    In one of the Russian forums a user reported the same problem with TinyMCE unrecognizing one cyrillic character, though the string was saved in db correctly.

    tinyMCE.init({entity_encoding: 'raw'});

    didn't help.

    The user as well reported that he fixed the problem turning off the AutoEscaping in CGI.pm. AutoEscaping by CGI.pm is made with Latin encoding. AutoEscaping is no problem for all encodings, except for Unicode. In utf-8 it is "escaping" a certain part of the character, which leads to "unrecognized" characters in the output.

    Could that be the reason? I've no idea how to deal with it. Where is CGI.pm on the server?

  • Ingo
    Avatar
    Forum Moderator
    801 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    they're referring to another script in another language, silverstripe doesn't use CGI.pm. both php5 and mysql4 should be safe to handle utf8, we encode all our content in it (and didn't have problems in spanish/french translations so far). the problem happening in title/menutitle pretty much rules out tinymce as a culprit. i think looking for help in russian forums is your best shot, as its only two out of a dozen russian special characters, so a very specific problem

  • Craneway
    Avatar
    20 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    Ingo, they, actually, discussed another CMF. I just thought it coud be the reason. It's definitely not.
    It looks like there's nothing in Russian forums about that. In Russia SS is not widely known yet. I couldn't find any Russian site running on SS.
    Do you think the cause of the problem is somehow with db encoding? It's strange that the strings are not saved correctly, though the collation of the fields is set to utf8_general_ci.
    Which script (file or module) in SS is responsible for saving the strings into DB?
    Do you think it also might be some system templates that are not set to utf8 and causing the weird output?
    The characters appear unrecognized after the edited content is saved.

    And I tried to correct the strings in the db fields, after flushing everything, the whole content output in the CMS and on the site became unreadable.

  • Ingo
    Avatar
    Forum Moderator
    801 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    you don't need to find russian silverstripe developers, just russians with knowledge about character handling in php5/mysql

  • Craneway
    Avatar
    20 Posts

    Re: Russian Unicode problem. What to do? Link to this post

    Hi, Ingo!
    I couldn't find the solution yet. If the site is on a shared hosting, there's not much that you can configure. I tried to create an empty db, then convert it to utf8 via phpmyadmin, then install SS. It doesn't help much. The server support guy said that I probably have to hack my php files (to insert mysql directives). Someth like:

    mysql_query ("set character_set_client='utf8'");
    mysql_query ("set character_set_results='utf8'");
    mysql_query ("set collation_connection='utf8_general_ci'");


    But to what file? I tried install.php before installing SS, works fine but no use.

    Could you or anyone, please, look at my server variables after SS installation that are shown by phpmyadmin. This is a mess. Is there any ideas what I should do?

    character set client    utf8
    (Global value)    cp1251
    character set connection   utf8
    (Global value)    cp1251
    character set database    cp1251
    character set filesystem    binary
    character set results    utf8
    (Global value)    cp1251
    character set server    cp1251
    character set system    utf8
    character sets dir    /usr/share/mysql/charsets/
    collation connection    utf8_general_ci
    (Global value)    cp1251_general_ci
    collation database    cp1251_general_ci
    collation server    cp1251_general_ci

    72529 Views
Page: 1 2 3 4
Go to Top

Want to know more about the company that brought you SilverStripe? Then check out SilverStripe.com

Comments on this website? Please give feedback.