Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

We've moved the forum!

Please use forum.silverstripe.org for any new questions (announcement).
The forum archive will stick around, but will be read only.

You can also use our Slack channel or StackOverflow to ask for help.
Check out our community overview for more options to contribute.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, Ed, biapar, Willr, Ingo, swaiba

SS 3: Cyrillic symbols and Dataobject Relationship Management


Go to End


7 Posts   1781 Views

Avatar
WebSpilka

Community Member, 89 Posts

14 November 2012 at 3:58am

Edited: 14/11/2012 4:00am

I get code from http://doc.silverstripe.org/framework/en/tutorials/5-dataobject-relationship-management
If I try Link Existing 'Student' that has name with cyrillic symbols: 'Цимбалюк'.
In

<input id="gridfield_relationsearch" class="text relation-search no-change-track ui-autocomplete-input ui-autocomplete-loading" type="text" placeholder="Find Students by Name" data-search-url="'admin/pages/edit/EditForm/field/Students/search'" name="gridfield_relationsearch" autocomplete="off" role="textbox" aria-autocomplete="list" aria-haspopup="true">

I enter the following characters 'Ци', SS create GET request
http://......../admin/pages/edit/EditForm/field/Students/search?Title=Content+personalization&URLSegment=content-personalization&MenuTitle=Content+personalization&Content=%3Cp%3EContent+personalization%3C%2Fp%3E&MetaTitle=&MetaKeywords=&MetaDescription=&ExtraMeta=&gridfield_relationsearch=%D0%A6%D0%B8&Students%5BGridState%5D=%7B%22GridFieldAddRelation%22%3A%5B%5D%2C%22GridFieldSortableHeader%22%3A%7B%22SortColumn%22%3A%5B%5D%7D%2C%22GridFieldFilterHeader%22%3A%7B%22Columns%22%3A%5B%5D%7D%2C%22GridFieldPaginator%22%3A%7B%22currentPage%22%3A1%7D%2C%22GridFieldSearchRelation%22%3A%5B%5D%7D&filter%5BName%5D=&gridfield_relationsearch=&Mentors%5BGridState%5D=%7B%22GridFieldAddRelation%22%3A%5B%5D%2C%22GridFieldSortableHeader%22%3A%7B%22SortColumn%22%3A%5B%5D%7D%2C%22GridFieldFilterHeader%22%3A%7B%22Columns%22%3A%5B%5D%7D%2C%22GridFieldPaginator%22%3A%7B%22currentPage%22%3A1%7D%2C%22GridFieldSearchRelation%22%3A%5B%5D%7D&filter%5BName%5D=&ClassName=Project&ParentID=7&SecurityID=da6568ad4a6d135ca97531800dc699d0211fe731&ID=9&AbsoluteLink=&LiveLink=http%3A%2F%2Fwebspilka.dev%2Favto-shrot%2Fprojects%2Fcontent-personalization%2F%3Fstage%3DLive&StageLink=http%3A%2F%2Fwebspilka.dev%2Favto-shrot%2Fprojects%2Fcontent-personalization%2F%3Fstage%3DStage&TreeTitle=%3Cspan+class%3D%22jstree-pageicon%22%3E%3C%2Fspan%3E%3Cspan+class%3D%22item%22%3EContent
+personalization%3C%2Fspan%3E&gridfield_relationsearch=%u0426%u0438

in framework/forms/gridfield/GridFieldAddExistingAutocompleter.php

public function doSearch ....
....
$stmts[] .= sprintf('"%s" LIKE \'%s%%\'', $searchField, Convert::raw2sql($request->getVar('gridfield_relationsearch')));
...

I get $request->getVar('gridfield_relationsearch')='%u0426%u0438'
And next query in mysql
SELECT DISTINCT "Student"."ClassName", "Student"."Created", "Student"."LastEdited", "Student"."Name", "Student"."University", "Student"."ProjectID", "Student"."ID", CASE WHEN "Student"."ClassName" IS NOT NULL THEN "Student"."ClassName" ELSE 'Student' END AS "RecordClassName" FROM "Student" WHERE ("Name" LIKE '%u0426%u0438%') AND ("Student"."ID" NOT IN (SELECT DISTINCT "Student"."ID" FROM "Student" WHERE ("ProjectID" = '9'))) ORDER BY "Student"."Name" ASC LIMIT 20

Why javascript codes variable in appearance '%u0426%u0438', not in '/u0426/u0438'?
How to decode the variable in php script?
Now MySQL receives encoded value:
"Name" LIKE 'Ци%'

I want send to MySQL decoded:
"Name" LIKE '%u0426%u0438%'

Avatar
WebSpilka

Community Member, 89 Posts

14 November 2012 at 11:20pm

Edited: 14/11/2012 11:27pm

a temporary solution for me was to add a function jdecoder to framework/forms/gridfield/GridFieldAddExistingAutocompleter.php

	//Decode Cyrillic symbols to utf8
	public function jdecoder($json_str) {
		$cyr_chars = array (
		'%u0430' => 'а', '%u0410' => 'А',
		'%u0431' => 'б', '%u0411' => 'Б',
		'%u0432' => 'в', '%u0412' => 'В',
		'%u0433' => 'г', '%u0413' => 'Г',
		'%u0434' => 'д', '%u0414' => 'Д',
		'%u0435' => 'е', '%u0415' => 'Е',
		'%u0451' => 'ё', '%u0401' => 'Ё',
		'%u0453' => 'ѓ', '%u0403' => 'Ѓ',
		'%u0454' => 'є', '%u0404' => 'Є',
		'%u0436' => 'ж', '%u0416' => 'Ж',
		'%u0437' => 'з', '%u0417' => 'З',
		'%u0438' => 'и', '%u0418' => 'И',
		'%u0456' => 'і', '%u0406' => 'І',
		'%u0457' => 'ї', '%u0407' => 'Ї',
		'%u0439' => 'й', '%u0419' => 'Й',
		'%u043A' => 'к', '%u041a' => 'К',
		'%u043B' => 'л', '%u041B' => 'Л',
		'%u043C' => 'м', '%u041C' => 'М',
		'%u043D' => 'н', '%u041D' => 'Н',
		'%u043E' => 'о', '%u041E' => 'О',
		'%u043F' => 'п', '%u041F' => 'П',
		'%u0440' => 'р', '%u0420' => 'Р',
		'%u0441' => 'с', '%u0421' => 'С',
		'%u0442' => 'т', '%u0422' => 'Т',
		'%u0443' => 'у', '%u0423' => 'У',
		'%u0444' => 'ф', '%u0424' => 'Ф',
		'%u0445' => 'х', '%u0425' => 'Х',
		'%u0446' => 'ц', '%u0426' => 'Ц',
		'%u0447' => 'ч', '%u0427' => 'Ч',
		'%u0448' => 'ш', '%u0428' => 'Ш',
		'%u0449' => 'щ', '%u0429' => 'Щ',
		'%u044A' => 'ъ', '%u042A' => 'Ъ',
		'%u044B' => 'ы', '%u042B' => 'Ы',
		'%u044C' => 'ь', '%u042C' => 'Ь',
		'%u044D' => 'э', '%u042D' => 'Э',
		'%u044E' => 'ю', '%u042E' => 'Ю',
		'%u044F' => 'я', '%u042F' => 'Я',
		'%r' => '',
		'%n' => '',
		'%t' => ''
		);

		foreach ($cyr_chars as $key => $value) {
			$json_str = str_replace($key, $value, $json_str);
		}
		return $json_str;
}

and change next code
//Convert::raw2sql($request->getVar('gridfield_relationsearch'))); //- original
  Convert::raw2sql($this->jdecoder($request->getVar('gridfield_relationsearch')))); // - my

and how to make it right? the question is still open.
I think the problem in javaScript, which passes the request.
Unfortunately I can not yet find the JavaScript code that is responsible for this

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 9:46am

Edited: 15/11/2012 9:46am

I wonder if urldecode() is used in the server side PHP to decode the URL containing encoded Cyrillic characters?

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 9:49am

Does Convert::raw2sql(urldecode($request->getVar('gridfield_relationsearch'))) work as a replacement in the doSearch function of GridFieldAddExistingAutocompleter.php?

Avatar
WebSpilka

Community Member, 89 Posts

15 November 2012 at 1:16pm

Edited: 15/11/2012 1:19pm

urldecode not solve the problem at all ((.
But in urldecode documentation I find next solution:

public function utf8_urldecode($str) {
		$str = preg_replace("/%u([0-9a-f]{3,4})/i","&#x\\1;",urldecode($str));
		return html_entity_decode($str,null,'UTF-8');;
	}
...
...Convert::raw2sql($this->utf8_urldecode($request->getVar('gridfield_relationsearch')))

And yet there is written:
I had troubles converting Unicode-encoded data in $_GET (like this: %u05D8%u05D1%u05E2) which is generated by JavaScript's escape() function to UTF8 for server-side processing.

Developer.mozilla.org said:
The escape and unescape functions do not work properly for non-ASCII characters and have been deprecated. In JavaScript 1.5 and later, use encodeURI, decodeURI, encodeURIComponent, and decodeURIComponent.

I think it is more logical in SS3 to stop using JavaScript's escape(), and begin to use encodeURIComponent ?
I wonder how much it takes a lot to make edits?
And where else can be a problem with the Cyrillic?

Avatar
WebSpilka

Community Member, 89 Posts

15 November 2012 at 2:04pm

Edited: 15/11/2012 2:05pm

To solve the problem with Cyrillic can fix php file as described above.
Or you can edit the framework/javascript/GridField.js

+escape(searchField.attr('name'))+'='+escape(searchField.val()),

fix on

+encodeURIComponent(searchField.attr('name'))+'='+encodeURIComponent(searchField.val()), 

Avatar
Sean

Forum Moderator, 922 Posts

15 November 2012 at 4:57pm

Thanks for this, I've done a pull request to get it fixed in the next version of 3.0:

https://github.com/silverstripe/sapphire/pull/953

Sean