Export Types

Provide new ways to display and download the data.

Overview

People need data in different formats. The Core script includes a number of standard formats: CSV, Excel, HTML, XML, JSON, assorted programming language data structures and SQL. However, this list is far from exhaustive. This page explains how to add your own Export Types so you can use the Data Generator to generate the random data in whatever format you require.

Export Types appear in the main user interface and may contain whatever additional settings you may need. You can include an optional JS module that ties in with the interface, to do things like perform Export Type-specific validation on the Data Type rows selected, or really anything else you may need. On the backend, Export Types do the job of tying together all the data produced by the Data Types and generating whatever strings is needed for the entire data set. We'll step through how it all works in a bit, but let's start with dissecting an Export Type to see each part.

Anatomy of an Export Type

Now let's do a high-level view of what goes into a module: the files and folders, the JS + PHP components and how the translations / internationalization works. We'll get into the details about the code in the following sections.

Files and Folders

All Export Types are found in the /resources/plugins/exportTypes/ folder. Each Export Type has its own folder, which acts as the namespace for the JS and PHP code. What I mean is that the exact string you choose for the folder (like XML or CSV) has to be used in your JS module creation and PHP class definition. I'll explain all that below.

An Export Type has the following required files. Let's assume the folder name is MyNewExportFormat.

• /resources/plugins/exportType/MyNewExportFormat.js: this file can actually be called whatever you want, but for consistency and for keeping reading the Web Inspector / Firebug net panel, I'd name them like this. You can have as many JS files as you want, but one is almost certainly enough.
• /resources/plugins/exportType/MyNewExportFormat.class.php: this contains your ExportType_MyNewExportFormat class, which handles all necessary server-side code: the actual markup/export string generation and any markup you want available in the generator webpage. More info about all that below.
• /resources/plugins/exportType/lang/en.php: A PHP file containing a single array (hash) that lists all strings used in your module.

You can also include any custom CSS files you want. See the PHP class definition below for more information.

JavaScript

The JS module for your Export Type does the following:

• Registers itself with the Manager JS component, to allow it to publish and subscribe to messages; i.e. to interact with the Core script and detect when certain user interface events happen.
• Save and load whatever Export Type settings are offered by your Export Type (if any).
• Perform whatever validation is required to ensure the user fills in the Export Type settings properly.

PHP

The PHP class for your Export Type handles the following functionality:

• Creates whatever HTML and fields should be included in the Export Type tab.
• Pieces together the various return values from the Data Types and creates the final output string.
• Handles the different export types: In-page, New window/tab, and Prompt to download and specifies which of them the Export Type supports.
• Specifies which CodeMirror mode(s) that it needs in the UI to syntax highlight the generated data.
• Specifies the download filenames for the Prompt to download option.

Language Files

All text strings that appear in your module should be pulled from a language file. It's very simple. Just create a file called en.php in your /resources/plugins/exportTypes/[export type folder]/lang/ folder. That file should contain a single $L hash, like so:

<?php 

$L = array();
$L["EXPORT_TYPE_NAME"] = "XML";
$L["label1"] = "Label 1";
$L["label2"] = "Label 2";

// ...

Once you do that, the Data Generator automatically makes that information accessible to your PHP and JS code. I'll explain how that works in the following sections.

The PHP Class

All plugins - Data Types, Export Types and Country plugins have to extend a base, abstract class defined by the core code. Hopefully you know what this means, but if not - time for some Googling! Simply put, abstract classes are a mechanism to help ensure that the class being defined has a proper footprint and contains all the functionality that's expected and required.

For Export Types, take a look at this file: /resources/classes/ExportTypePlugin.class.php. That's the class you'll need to extend.

Example: CSV Export Type

Now let's look at an actual implementation. If you want to see the complete list of available variables and methods, check out the source code of the Export Type abstract class (/resources/classes/ExportTypePlugin.abstract.class.php). It's well documented.

This is the PHP class for the CSV class. It's a simple Export Type that outputs the randomly generated data in CSV format. It provides the user with the option to choose the line ending char (Windows / Mac, Unix) and the delimiter char. Pretty straightforward.

<?php

/**
 * @package ExportTypes
 */

class CSV extends ExportTypePlugin {
	protected $isEnabled = true;
	protected $exportTypeName = "CSV";
	protected $jsModules = array("CSV.js");
	protected $contentTypeHeader = "application/csv";
	public $L = array();


	function generate($generator) {
		$exportTarget = $generator->getExportTarget();
		$postData     = $generator->getPostData();
		$data         = $generator->generateExportData();

		$csvDelimiter = ($postData["etCSV_delimiter"] == '\t') ? "\t" : $postData["etCSV_delimiter"];
		$csvLineEndings = $postData["etCSV_lineEndings"];

		switch ($csvLineEndings) {
			case "Windows":
				$newline = "\r\n";
				break;
			case "Unix":
				$newline = "\n";
				break;
			case "Mac":
			default:
				$newline = "\r";
				break;
		}

		$content = "";
		if ($data["isFirstBatch"]) {
			$content .= implode($csvDelimiter, $data["colData"]);
		}
		foreach ($data["rowData"] as $row) {
			$content .= $newline . implode($csvDelimiter, $row);
		}

		return array(
			"success" => true,
			"content" => $content
		);
	}

	/**
	 * Used for constructing the filename of the filename when downloading.
	 * @see ExportTypePlugin::getDownloadFilename()
	 * @param Generator $generator
	 * @return string
	 */
	function getDownloadFilename($generator) {
		$time = date("M-j-Y");
		return "data{$time}.csv";
	}

	function getAdditionalSettingsHTML() {
		$LANG = Core::$language->getCurrentLanguageStrings();

		$html =<<< END
<table cellspacing="0" cellpadding="0" width="100%">
<tr>
	<td width="50%">
		<table cellspacing="2" cellpadding="0" width="100%">
		<tr>
			<td width="160">{$this->L["delimiter_chars"]}</td>
			<td>
				<input type="text" size="2" name="etCSV_delimiter" id="etCSV_delimiter" value="|" />
			</td>
		</tr>
		</table>
	</td>
	<td width="50%">
		<table cellspacing="0" cellpadding="0" width="100%">
		<tr>
			<td width="160">{$this->L["eol_char"]}</td>
			<td>
				<select name="etCSV_lineEndings" id="etCSV_lineEndings">
					<option value="Windows">Windows</option>
					<option value="Unix">Unix</option>
					<option value="Mac">Mac</option>
				</select>
			</td>
		</tr>
		</table>
	</td>
</tr>
</table>
END;

		return $html;
	}
}

Let's look at each line in turn.

• class CSV extends ExportTypePlugin: our class definition. All Export Type class names must be the same as the folder.
• $isEnabled: this var explicitly enables/disables the module. In case you're tinkering around with a new Export Type, sometimes you may not want it to show up in the UI - so you'd just set this to false.
• $exportTypeName: this is the human-readable name of your module. It can be in whatever language you want, but we prefer English as the default language string. The value you enter in this variable is automatically overridden if the current selected language has the following key in the language file array: $L["EXPORT_TYPE_NAME"] = "New Name"; This provides a simple mechanism to provide alternative translations of your Export Type names.
• $jsModules: an array containing whatever JS files you want to include. Note: these must be in AMD format, for compatibility with requireJS.
• $contentTypeHeader: this is used for the Prompt to download option. It lets your Export Type send additional headers that let the user's operating system know how to handle the data format.
• $L: this variable is automatically populated with the appropriate language strings when your class is instantiated.

Now onto the methods.

• generate: this is the main generation function of your class. It does the job of actually generating the final output data. One important thing to know about Export Type generation is that is happens in different contexts. For the in-page export type, the Export Type has to generate it in chunks - each chunk being (say) 100 rows at a time. That information is sent back by the Core script via an Ajax call. As such, if the Export Type supports that export type, the generate function needs to be able to handle both scenarios: generating the final result piece-meal, or in one big chunk. You can see how it works in the code above. It checks the $data["isFirstBatch"] boolean to figure out whether to create the first row or not. Similarly, there's a $data["isLastBatch"] key available as well (not used here).
• getDownloadFilename: returns the filename for the downloadable content, used in the Prompt to Download export format.
• getAdditionalSettingsHTML: this method returns whatever HTML you want to appear in the Export Type tab. This is handy if you want to allow for a little fine-tuning of exactly what your Export Type generates. Look at the other Export Types to get an idea of how this method can be used.

Class Variable List

Alright! Here's the full list of class vars that have special meaning.

Class Method List

generate()

getDownloadFilename()

__construct()

getAdditionalSettingsHTML()

Non-overridable Methods

The following methods are defined on the Export Plugin abstract class, which you can use when developing your Export Type.

The JS Module

Each Data Type may choose to have an optional JS component: a javascript module that performs certain functionality like saving/loading the data type data, running client-side validation on the user inputs (if required) and triggering whatever additional JS code is necessary.

Optional or required?

The JS module is optional. The Core script handles saving and loading the Column Title and Data Type for all Data Types, so if you don't need anything in the Example or Options columns, you don't need to include a JS module.

Explaining how the JS module works can be a little abstract, so let's start with an example.

Example: CSV Export Type

The following is the JS module for the Alphanumeric Data Type. Give it a look over, then we'll pull it apart and explain each bit below.

/*global $:false*/
define([	
	"manager",
	"constants",
	"lang",
	"generator"
], function(manager, C, L, generator) {

	"use strict";

	/**
	 * @name CSV
	 * @see ExportType
	 * @description Client-side code for the CSV Export Type.
	 * @namespace
	 */

	var MODULE_ID = "export-type-CSV";
	var LANG = L.exportTypePlugins.CSV;


	var _loadSettings = function(settings) {
		$("#etCSV_delimiter").val(settings.delimiter);
		$("#etCSV_lineEndings").val(settings.eol);
	};

	var _saveSettings = function() {
		return {
			"delimiter": $("#etCSV_delimiter").val(),
			"eol":       $("#etCSV_lineEndings").val()
		};
	};

	var _resetSettings = function() {
		$("#etCSV_delimiter").val("|");
		$("#etCSV_lineEndings").val("Windows");
	};

	var _validate = function() {
		var delimiterField = $("#etCSV_delimiter");
		var errors = [];

		// note we don't trim it. I figure whitespace could, technically be used as a delimiter
		if (delimiterField.val() === "") {
			errors.push({
				els: delimiterField,
				error: LANG.validation_no_delimiter
			});
		}

		return errors;
	};

	manager.registerExportType(MODULE_ID, {
		validate: _validate,
		loadSettings: _loadSettings,
		saveSettings: _saveSettings,
		resetSettings: _resetSettings
	});
});

Now let's go line by line.

• /*global $:false*/ this first line is for jshint/jslint. In my local environment, I use jshint with strict mode to catch problems. This line just tells the interpreter to ignore the jQuery dollar sign global.

• define([
  	"manager",
  	"constants",
  	"lang",
  	"generator"
  ], function(manager, C, L, generator) {
  	//...
  });
  

  The outer code that wraps the entire JS module is called within requireJS's define function. This ensures the code is defined as an AMD (Asynchronous Module Definition) for consumption by other code. The important thing to understand here is the parameters. The first array params define string labels to other modules: they all map to specific JS files - you can find the mapping in /resources/scripts/requireConfig.js. Each of those discrete modules is in turn passed to the Data Type module via functions in the anonymous section param to define(). Whatever public API those modules reveal are now accessible via the four params: manager, constants, lang, generator.

  When defining your own Export Type module JS file, you'll want to include all four of those params. They all contain useful functionality and data that you'll need.

• "use strict"; - do it! JS strict mode is never a bad idea. :D
• Here we're going to skip ahead to the very end of the code, to these lines:

  	manager.registerExportType(MODULE_ID, {
  		validate: _validate,
  		loadSettings: _loadSettings,
  		saveSettings: _saveSettings,
  		resetSettings: _resetSettings
  	});
  

  This chunk of code is required for your Export Type JS Module. It registers your Export Type with the Core. That allows it to listen to published events, publish its own events for other code to listen to, tie into the validation functionality and so on. It's pretty straightforward. The manager.registerExportType() function takes two parameters: the unique MODULE_ID constant, (see below) and an object containing certain required and optional functions, whose property names have special values. Again, more on that below. Now let's go back to the top of the code again.

• 	/**
  	 * @name CSV
  	 * @see ExportType
  	 * @description Client-side code for the CSV Export Type.
  	 * @namespace
  	 */
  
  	var MODULE_ID = "export-type-CSV";
  	var LANG = L.exportTypePlugins.CSV;
  

  • The comment is of a particular format for being understood by JSDoc. For more information on that, see the JS Doc project.
  • The MODULE_ID variable is special. It must always be of the form data-type-[FOLDER NAME]. That acts a unique identifier within the client-side code so the Manager can keep track of who's who.
  • As with the PHP code, the language strings for your Data Type are automatically accessible: you don't have to do any extra work to get access to them. The L function param fed to your Export Type contains all language strings in the system - in whatever language is currently selected. To locate the strings for your own module, just reference it by your Export Type folder name, again: L.exportTypePlugins.[FOLDER NAME]
• The following lines all define special functions. Rather than explain the implementation details of each of these for the CSV type, we'll discuss these in a more abstract sense in the next section.

Registration Functions

As explained above, the second parameter of the manager.registerExportType() function is an object containing various predefined functions. This explains what are the properties for that object and what they're used for. Note: all properties are optional, but you'll almost certainly need one or more.

Pub/Sub & Event List

As mentioned elsewhere, the client-side code revolves around the idea of publish/subscribe - or pub/sub. Different parts of the script can publish arbitrary events with arbitrary information associated with them, and any module can choose to listen out for particular events and run code when they occur. This is a very elegant pattern: it allow us to keep our modules loosely coupled and reduce the likelihood of introducing dependencies that can break things.

The core script publishes the following script for certain events that occur in the lifetime of the page. They're all found in /resources/scripts/constants.php (returned as JS). You can refer to them in your code via the C parameter, mapping to the constants module. The names are pretty descriptive so I won't bother explaining them any further.

• C.EVENT.RESULT_TYPE.CHANGE
• C.EVENT.COUNTRIES.CHANGE
• C.EVENT.DATA_TABLE.ONLOAD_READY
• C.EVENT.DATA_TABLE.ROW.CHECK_TO_DELETE
• C.EVENT.DATA_TABLE.ROW.UNCHECK_TO_DELETE
• C.EVENT.DATA_TABLE.ROW.DELETE
• C.EVENT.DATA_TABLE.ROW.TYPE_CHANGE
• C.EVENT.DATA_TABLE.ROW.EXAMPLE_CHANGE
• C.EVENT.DATA_TABLE.ROW.ADD
• C.EVENT.DATA_TABLE.ROW.RE_SORT
• C.EVENT.DATA_TABLE.ROW.HELP_DIALOG_OPEN
• C.EVENT.DATA_TABLE.ROW.HELP_DIALOG_CLOSE
• C.EVENT.DATA_TABLE.CLEAR
• C.EVENT.GENERATE
• C.EVENT.IO.SAVE
• C.EVENT.IO.LOAD
• C.EVENT.TAB.CHANGE
• C.EVENT.MODULE.REGISTER
• C.EVENT.MODULE.UNREGISTER

How to subscribe to an event

Generally you'll want to set up your subscriptions in your module's init() function. Here's how it works:

...

var _init = function() {
	var subscriptions = {};
	subscriptions[C.EVENT.COUNTRIES.CHANGE] = _onChangeCountries;
	manager.subscribe(subscriptions);
};

var _onChangeCountries = function(msg) {
	console.log(msg);
};

...

manager.registerDataType(MODULE_ID, {
	init: _init
});

...

That would subscribe to the C.EVENT.COUNTRIES.CHANGE event (which is where the user adds/removes a country from the Country List section in the UI) and attaches a callback function - _onChangeCountries(). The manager.subscribe() function can be called at any time in any of your functions, so you can subscribe to events on the fly.

Available Resources

There are several client-side code libraries already available in the page that can be used in your Data Type:

• jQuery ($)
• jQuery UI
• MomentJS- date/time formatting script
• Chosen - dropdown enhancement

You can always include additional libraries should you wish, but do try to namespace them.

Adding your Export Type

When you add a new Export Type, just creating the new files and folders won't get it to show up in the UI. First, you'll need to follow the steps below to make sure your PHP class and (optionally) JS Module have been created properly, and afterwards you'll need to refresh the UI.

To update the list of available Export Types in the UI, go to the second Settings tab. There, click the Reset Plugins button. A dialog will appears which resets all the available plugins (don't worry, this won't cause any problems with saved content or anything like that). After refreshing the page, you should see your Export Type appear as a tab at the bottom of the page.

How to Contribute

If you feel that your Export Type could be of use to other people, send it my way! I'd love to take a look at it, and maybe even include it in the core script for others to download. Read the How to Contribute page.