Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

We've moved the forum!

Please use forum.silverstripe.org for any new questions (announcement).
The forum archive will stick around, but will be read only.

You can also use our Slack channel or StackOverflow to ask for help.
Check out our community overview for more options to contribute.

Data Model Questions /

Moderators: martimiz, Sean, Ed, biapar, Willr, Ingo, swaiba

Error in JSONDataFormatter convertDataObjectToJSONObject() ?


Go to End


4 Posts   1331 Views

Avatar
snel

Community Member, 12 Posts

2 October 2014 at 9:34am

I'm building some custom request handlers (inspired by the RestfulServer module). The default response-formatter intentionally is set to JSON, that's why the JSONDataFormatter's function "convertDataObjectToJSONObject()" is fired. Now, when the service returns an object with a varchar property from the database, that contains an ampersand '&' within the value string, the convertDataObjectToJSONObject() function converts this to a &.

I've debugged and analized the code and just until the line 59 of JSONDataFormatter (Silverstripe Version 3.1) where it says:

55		foreach($this->getFieldsForObj($obj) as $fieldName => $fieldType) {
56			// Field filtering
57			if($fields && !in_array($fieldName, $fields)) continue;
58
59			$fieldValue = $obj->obj($fieldName)->forTemplate();
60			$serobj->$fieldName = $fieldValue;
61		}

The forTemplate() function finally utilizes the Convert::raw2xml() function to convert the database-value of the Varchar field to an XML string. This is where the & is converted to an &

I think this behaviour is wrong, because JSON mus only contain unicode characters: When reading the definition of json on json.org I assume that a string must not be converted using htmlentities() or urlencode but must contain only unicode-characters. That's why I think the function's implementation is wrong - there shouln'd be a conversion to XML using HTML-entities. Either the Unicode-Values returnedby the database should be passed or a conversion to utf8 should be done

What do you think?

Avatar
Schippie

Community Member, 38 Posts

4 October 2014 at 10:02pm

Edited: 04/10/2014 10:11pm

I have ran into the same issue and what i decided to do was simply change the line to:

        foreach($this->getFieldsForObj($obj) as $fieldName => $fieldType)
        {
            // Field filtering
            if($fields && !in_array($fieldName, $fields)) continue;

            $fieldValue = TSRestfulUtils::castDBField($obj->obj($fieldName)->getValue(), $obj->obj($fieldName), $fieldType);
            $serobj->$fieldName = $fieldValue;
        }

Where the castDBField does:

/**
     * Function that casts a Database field instance to a specified output this is useful when dealing with JSON data in which you want integers to be represented as integers.
     * @param int|null|string|float|boolean $value
     * @param DBField|null $class
     * @param String|null $fieldType
     * @return float|null|string
     */
    public static function castDBField($value, DBField $class = null, $fieldType = null)
    {
        if(!isset($value)) return null;
        switch(true)
        {
            case ($fieldType === 'ForeignKey'):
                return (int)$value;
                break;
            case ($class instanceof Boolean):
                return (boolean)$value;
                break;
            case ($class instanceof Date):
                return (string)$value;
                break;
            case ($class instanceof Decimal):
                return (float)$value;
                break;
            case ($class instanceof Float):
                return (float)$value;
                break;
            case ($class instanceof Int):
                return (int)$value;
                break;
            case ($class instanceof Money):
                return (float)$value;
                break;
            case ($class instanceof StringField):
                return (string)$value;
                break;
            case ($class instanceof Time):
                return (string)$value;
                break;
            case ($class instanceof Year):
                return (string)$value;
                break;
        }
        return null;
    }

The reasoning for this is that the has_one relations generate a <has_one>ID field is casted to a TEXT type for some reason and is shown as a string while it should be an int...
I am not sure if this would fix your utf8 but you could easily modify it to do so. I do find it weird that they use an xml thing for json but ah well..

Avatar
snel

Community Member, 12 Posts

6 October 2014 at 4:08am

Thank you for the hint. I eventually came to the following solution, that seems to working:

$fieldValue = $obj->obj($fieldName)->JS();

The function JS() is defined within the class DBField and is calling Convert::raw2js(). I think, we should do a pull request on that.

Avatar
Schippie

Community Member, 38 Posts

6 October 2014 at 8:13pm

Edited: 06/10/2014 9:59pm

While this in general works (i used it first to) it has one major drawback. Foreign keys like the ones auto generated from has one relations <has_one>ID are not converted to an INT for some unexplainable reason. Since the DBField type is Text (think this is some sort of default fallback if no type is found). It also apparently applies to the ID table column which is again seen as a TEXT field.