Skip to main content

This site requires you to update your browser. Your browsing experience maybe affected by not having the most up to date version.

We've moved the forum!

Please use forum.silverstripe.org for any new questions (announcement).
The forum archive will stick around, but will be read only.

You can also use our Slack channel or StackOverflow to ask for help.
Check out our community overview for more options to contribute.

General Questions /

General questions about getting started with SilverStripe that don't fit in any of the categories above.

Moderators: martimiz, Sean, Ed, biapar, Willr, Ingo, swaiba

Import XML data


Go to End


2 Posts   3194 Views

Avatar
MarijnKampf

Community Member, 176 Posts

8 June 2010 at 7:55pm

I'm working on a website that will require importing XML data on a regular basis from the admin section. It should be able to update existing entries / create new ones and delete old ones.

I was thinking of something based on the CSVBulkUpload module or possibly the RestFulService. Before I start coding however, I'd like to check whether there is any standard SilverStripe functionality available that I've missed and would do the job.

Also any tips to which of the two services may be best suitable are appreciated.

Avatar
MarijnKampf

Community Member, 176 Posts

10 June 2010 at 11:36pm

Minus a couple of remaining issues here is my solution. I based it on CsvBulkLoader.php and added a section to the ModelAdmin.

I've used the XmlLoader class from: http://www.stress-free.co.nz/integrating_google_site_search_into_silverstripe and added XmlLoader.php to mysite/code/admin/XmlLoader.php

mysite/code/admin/XmlLoader.php is an abstract class that you can extend for the classes you want to import.

<?php
/**
 * Uses the XmlLoader.php to process XML input.
 *
 * @author Marijn Kampf, www.exadium.com. (<myfirstname>@exadium.com)
 *
 * Notes: ColumnMap is ignored implement abstract function nodeToColumns($node) instead.
 *
 */

require('XmlLoader.php');

abstract class XmlBulkLoader extends CsvBulkLoader {
	/**
	 * Use cURL (Default: false).
	 *
	 * @var boolean
	 */
	public $useCurl = false;

	/**
	 * Url parameters (Default: empty array);
	 *
	 * @var array
	 */
	public $parameters = array();

	/**
	 * Identifies if the has a header row.
	 * @var boolean
	 */
	public $hasHeaderRow = true;

	/**
	 * Identifies nodes to attepmt to import
	 * @var SimpleXMLElement::xpath
	 */
	public $nodesXPath;

	/**
	 * Abstract function to convert every node found in the XML to array record
	 *
	 * @param SimpleXMLElement object $node, single XML node to process
	 * @return record array with each column as paremeter
	 *
	 * Sample implementation:
	 *
	 * 	public function nodeToColumns($node)	{
	 * 		$record = array();
	 * 		$record['Lat'] = (string)$node['lat']; // XML attribute example
	 * 		$record['Name'] = (string)$node->name; // XML element example
	 * 		return $record;
	 * 	}
	 */
	abstract protected function nodeToColumns($node);

	public function processAll($url, $preview = false) {
		$results = new BulkLoader_Result();

		$xmlLoader = new XmlLoader();
		$xml = $xmlLoader->pullXml($url, $this->parameters, $this->useCurl);

		$nodes = $xml->xpath($this->nodesXPath);

		foreach($nodes as $node) {
			$this->processRecord($node, $this->columnMap, $results, $preview);
		}

		return $results;
	}

	protected function processRecord($node, $columnMap, &$results, $preview = false) {
		$record = $this->nodeToColumns($node);
		return parent::processRecord($record, $columnMap, $results, $preview);
	}
}

Example implementation of XmlBulkLoader for

<?php
/**
 * Implementation of XmlLoader.php to process XML input specific for property.
 *
 * @author Marijn Kampf, www.exadium.com. (<myfirstname>@exadium.com)
 *
 * Notes: 
 *		implement nodeToColumns function to map XML node to record array instead of CSV's columnMap
 *		$nodesXPath is used to select nodes
 * 
 *		Read CSV bulk loader docs for background info on duplicateChecks and relationCallbacks 
 *		http://doc.silverstripe.org/csvbulkloader?s[]=csv&s[]=import
 */

class PropertyXmlBulkLoader extends XmlBulkLoader {
	public $nodesXPath = '/markers/marker';

/**
 * Convert every node found in the XML to array record
 *
 * @param SimpleXMLElement object $node, single XML node to process
 * @return record array with each column as paremeter
 */
	public function nodeToColumns($node)	{
		$record = array();
		$record['Name'] = (string)$node['propname'];
		$record['Code'] = (string)$node->propref;
		return $record;
	}

	public $duplicateChecks = array(
		'Code' => 'Code'
	);

  public $relationCallbacks = array(
		'TheArea' => array(
			'relationname' => 'TheArea',
			'callback' => 'getAreaByCode'
		)
  );

 	static function getAreaByCode(&$obj, $val, $record) {
		$SQL_val = Convert::raw2sql($val);
		$do = DataObject::get_one(
		  'Area', "Code = '{$SQL_val}'"
		);
		Debug::Show($do);
		return $do;
  }
}

To enable inclusion in admin create mysite/code/admin/PropertyAdmin.php

<?php
class PropertyAdmin extends ModelAdmin {

	public static $managed_models = array(
    'Property',
	);

	public static $model_importers = array(
		'Property' => 'PropertyXmlBulkLoader',
	);

	static $url_segment = 'properties';
	static $menu_title = 'Properties';

	public static $collection_controller_class = "PropertyAdmin_CollectionController";

	public function getModelForms() {
		$models = $this->getManagedModels();
		$forms  = new DataObjectSet();

		foreach($models as $class => $options) {
			if(is_numeric($class)) $class = $options;
			Debug::Show($class);
			$forms->push(new ArrayData(array (
				'SearchForm' => $this->$class()->SearchForm(),
				'CreateForm' => $this->$class()->CreateForm(),
				'ImportForm' => $this->$class()->ImportForm(),
				'ImportXML' => $this->$class()->ImportXMLForm(),
				'Title'     => (is_array($options) && isset($options['title'])) ? $options['title'] : singleton($class)->i18n_singular_name(),
				'ClassName' => $class,
				'Content'   => $this->$class()->getPropertyModelSidebar()
			)));
		}

		return $forms;
	}
}

class PropertyAdmin_CollectionController extends ModelAdmin_CollectionController {
	public function ImportXMLForm() {
		$modelName = $this->modelClass;

		if ($this->hasMethod('alternatePermissionCheck')) {
			if (!$this->alternatePermissionCheck()) return false;
		} else {
			if (!singleton($modelName)->canCreate(Member::currentUser())) return false;
		}

		$buttonLabel = 'Import XML from breconcottages.com';

		$actions = new FieldSet(
			$createButton = new FormAction('importXML', $buttonLabel)
		);
		$createButton->dontEscape = true;

		return new Form($this, "ImportXML", new FieldSet(), $actions);
	}

	function importXML($request) {
		$modelName = $this->modelClass;

		$xmlBulkLoader = new PropertyXmlBulkLoader($modelName);

		$result = 'Importing $modelName' . $xmlBulkLoader->processAll('http://localhost/properties.xml'); // Change to your XML file.

		return new SS_HTTPResponse(
			$result,
			200,
			$result
		);
	}

	/**
	 * Get a combination of the Search, Import and Create forms that can be inserted into a {@link ModelAdmin} sidebar.
	 *
	 * @return string
	 */
	public function getPropertyModelSidebar() {
		return $this->renderWith('PropertyModelSidebar');
	}

}

Finally to include XML import button on admin create cms/templates/Includes/PropertyModelSidebar.ss. Ideally this file should *NOT* be created in the CMS tree, but I couldn't get SilverStripe to find it in the themes/mytheme/templates/ folder.

<% if CreateForm %>
	<h3><% _t('ADDLISTING','Add') %></h3>
	$CreateForm
<% end_if %>

<h3><% _t('SEARCHLISTINGS','Search') %></h3>
$SearchForm

<% if ImportXMLForm %>
	$Class
	<h3><% _t('IMPORT_XML_TAB_HEADER', 'Import XML') %></h3>
	$ImportXMLForm
<% end_if %>

<% if ImportForm %>
	<h3><% _t('IMPORT_TAB_HEADER', 'Import') %></h3>
	$ImportForm
<% end_if %>