Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, biapar, Willr, Ingo, swaiba, simon_w

SS 3: Cyrillic symbols and Dataobject Relationship Management


Reply


7 Posts   817 Views

Avatar
WebSpilka

Community Member, 83 Posts

14 November 2012 at 3:58am

Edited: 14/11/2012 4:00am

I get code from http://doc.silverstripe.org/framework/en/tutorials/5-dataobject-relationship-management
If I try Link Existing 'Student' that has name with cyrillic symbols: 'Цимбалюк'.
In

<input id="gridfield_relationsearch" class="text relation-search no-change-track ui-autocomplete-input ui-autocomplete-loading" type="text" placeholder="Find Students by Name" data-search-url="'admin/pages/edit/EditForm/field/Students/search'" name="gridfield_relationsearch" autocomplete="off" role="textbox" aria-autocomplete="list" aria-haspopup="true">


I enter the following characters 'Ци', SS create GET request

http://......../admin/pages/edit/EditForm/field/Students/search?Title=Content+personalization&URLSegment=content-personalization&MenuTitle=Content+personalization&Content=%3Cp%3EContent+personalization%3C%2Fp%3E&MetaTitle=&MetaKeywords=&MetaDescription=&ExtraMeta=&gridfield_relationsearch=%D0%A6%D0%B8&Students%5BGridState%5D=%7B%22GridFieldAddRelation%22%3A%5B%5D%2C%22GridFieldSortableHeader%22%3A%7B%22SortColumn%22%3A%5B%5D%7D%2C%22GridFieldFilterHeader%22%3A%7B%22Columns%22%3A%5B%5D%7D%2C%22GridFieldPaginator%22%3A%7B%22currentPage%22%3A1%7D%2C%22GridFieldSearchRelation%22%3A%5B%5D%7D&filter%5BName%5D=&gridfield_relationsearch=&Mentors%5BGridState%5D=%7B%22GridFieldAddRelation%22%3A%5B%5D%2C%22GridFieldSortableHeader%22%3A%7B%22SortColumn%22%3A%5B%5D%7D%2C%22GridFieldFilterHeader%22%3A%7B%22Columns%22%3A%5B%5D%7D%2C%22GridFieldPaginator%22%3A%7B%22currentPage%22%3A1%7D%2C%22GridFieldSearchRelation%22%3A%5B%5D%7D&filter%5BName%5D=&ClassName=Project&ParentID=7&SecurityID=da6568ad4a6d135ca97531800dc699d0211fe731&ID=9&AbsoluteLink=&LiveLink=http%3A%2F%2Fwebspilka.dev%2Favto-shrot%2Fprojects%2Fcontent-personalization%2F%3Fstage%3DLive&StageLink=http%3A%2F%2Fwebspilka.dev%2Favto-shrot%2Fprojects%2Fcontent-personalization%2F%3Fstage%3DStage&TreeTitle=%3Cspan+class%3D%22jstree-pageicon%22%3E%3C%2Fspan%3E%3Cspan+class%3D%22item%22%3EContent
+personalization%3C%2Fspan%3E&gridfield_relationsearch=%u0426%u0438

in framework/forms/gridfield/GridFieldAddExistingAutocompleter.php

public function doSearch ....
....
$stmts[] .= sprintf('"%s" LIKE \'%s%%\'', $searchField, Convert::raw2sql($request->getVar('gridfield_relationsearch')));
...


I get $request->getVar('gridfield_relationsearch')='%u0426%u0438'
And next query in mysql

SELECT DISTINCT "Student"."ClassName", "Student"."Created", "Student"."LastEdited", "Student"."Name", "Student"."University", "Student"."ProjectID", "Student"."ID", CASE WHEN "Student"."ClassName" IS NOT NULL THEN "Student"."ClassName" ELSE 'Student' END AS "RecordClassName" FROM "Student" WHERE ("Name" LIKE '%u0426%u0438%') AND ("Student"."ID" NOT IN (SELECT DISTINCT "Student"."ID" FROM "Student" WHERE ("ProjectID" = '9'))) ORDER BY "Student"."Name" ASC LIMIT 20


Why javascript codes variable in appearance '%u0426%u0438', not in '/u0426/u0438'?
How to decode the variable in php script?
Now MySQL receives encoded value:

"Name" LIKE 'Ци%'


I want send to MySQL decoded:

"Name" LIKE '%u0426%u0438%'

Avatar
WebSpilka

Community Member, 83 Posts

14 November 2012 at 11:20pm

Edited: 14/11/2012 11:27pm

a temporary solution for me was to add a function jdecoder to framework/forms/gridfield/GridFieldAddExistingAutocompleter.php

   //Decode Cyrillic symbols to utf8
   public function jdecoder($json_str) {
      $cyr_chars = array (
      '%u0430' => 'а', '%u0410' => 'А',
      '%u0431' => 'б', '%u0411' => 'Б',
      '%u0432' => 'в', '%u0412' => 'В',
      '%u0433' => 'г', '%u0413' => 'Г',
      '%u0434' => 'д', '%u0414' => 'Д',
      '%u0435' => 'е', '%u0415' => 'Е',
      '%u0451' => 'ё', '%u0401' => 'Ё',
      '%u0453' => 'ѓ', '%u0403' => 'Ѓ',
      '%u0454' => 'є', '%u0404' => 'Є',
      '%u0436' => 'ж', '%u0416' => 'Ж',
      '%u0437' => 'з', '%u0417' => 'З',
      '%u0438' => 'и', '%u0418' => 'И',
      '%u0456' => 'і', '%u0406' => 'І',
      '%u0457' => 'ї', '%u0407' => 'Ї',
      '%u0439' => 'й', '%u0419' => 'Й',
      '%u043A' => 'к', '%u041a' => 'К',
      '%u043B' => 'л', '%u041B' => 'Л',
      '%u043C' => 'м', '%u041C' => 'М',
      '%u043D' => 'н', '%u041D' => 'Н',
      '%u043E' => 'о', '%u041E' => 'О',
      '%u043F' => 'п', '%u041F' => 'П',
      '%u0440' => 'р', '%u0420' => 'Р',
      '%u0441' => 'с', '%u0421' => 'С',
      '%u0442' => 'т', '%u0422' => 'Т',
      '%u0443' => 'у', '%u0423' => 'У',
      '%u0444' => 'ф', '%u0424' => 'Ф',
      '%u0445' => 'х', '%u0425' => 'Х',
      '%u0446' => 'ц', '%u0426' => 'Ц',
      '%u0447' => 'ч', '%u0427' => 'Ч',
      '%u0448' => 'ш', '%u0428' => 'Ш',
      '%u0449' => 'щ', '%u0429' => 'Щ',
      '%u044A' => 'ъ', '%u042A' => 'Ъ',
      '%u044B' => 'ы', '%u042B' => 'Ы',
      '%u044C' => 'ь', '%u042C' => 'Ь',
      '%u044D' => 'э', '%u042D' => 'Э',
      '%u044E' => 'ю', '%u042E' => 'Ю',
      '%u044F' => 'я', '%u042F' => 'Я',
      '%r' => '',
      '%n' => '',
      '%t' => ''
      );

      foreach ($cyr_chars as $key => $value) {
         $json_str = str_replace($key, $value, $json_str);
      }
      return $json_str;
}


and change next code

//Convert::raw2sql($request->getVar('gridfield_relationsearch'))); //- original
Convert::raw2sql($this->jdecoder($request->getVar('gridfield_relationsearch')))); // - my


and how to make it right? the question is still open.
I think the problem in javaScript, which passes the request.
Unfortunately I can not yet find the JavaScript code that is responsible for this

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 9:46am

Edited: 15/11/2012 9:46am

I wonder if urldecode() is used in the server side PHP to decode the URL containing encoded Cyrillic characters?

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 9:49am

Does Convert::raw2sql(urldecode($request->getVar('gridfield_relationsearch'))) work as a replacement in the doSearch function of GridFieldAddExistingAutocompleter.php?

Avatar
WebSpilka

Community Member, 83 Posts

15 November 2012 at 1:16pm

Edited: 15/11/2012 1:19pm

urldecode not solve the problem at all ((.
But in [url=http://www.php.net/manual/en/function.urldecode.php#79595]urldecode documentation[/url] I find next solution:

public function utf8_urldecode($str) {
      $str = preg_replace("/%u([0-9a-f]{3,4})/i","&#x\\1;",urldecode($str));
      return html_entity_decode($str,null,'UTF-8');;
   }
...
...Convert::raw2sql($this->utf8_urldecode($request->getVar('gridfield_relationsearch')))

And yet there is written:
I had troubles converting Unicode-encoded data in $_GET (like this: %u05D8%u05D1%u05E2) which is generated by JavaScript's escape() function to UTF8 for server-side processing.

[url=https://developer.mozilla.org/en-US/docs/JavaScript/Guide/Functions?redirectlocale=en-US&redirectslug=Core_JavaScript_1.5_Guide%2FFunctions#escape_and_unescape_functions]Developer.mozilla.org[/url] said:
The escape and unescape functions do not work properly for non-ASCII characters and have been deprecated. In JavaScript 1.5 and later, use encodeURI, decodeURI, encodeURIComponent, and decodeURIComponent.

I think it is more logical in SS3 to stop using JavaScript's escape(), and begin to use encodeURIComponent ?
I wonder how much it takes a lot to make edits?
And where else can be a problem with the Cyrillic?

Avatar
WebSpilka

Community Member, 83 Posts

15 November 2012 at 2:04pm

Edited: 15/11/2012 2:05pm

To solve the problem with Cyrillic can fix php file as described above.
Or you can edit the framework/javascript/GridField.js

+escape(searchField.attr('name'))+'='+escape(searchField.val()),

fix on

+encodeURIComponent(searchField.attr('name'))+'='+encodeURIComponent(searchField.val()),

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 4:57pm

Thanks for this, I've done a pull request to get it fixed in the next version of 3.0:

[url]https://github.com/silverstripe/sapphire/pull/953[/url]

Sean