Sdmx Data Parser

The Data Parser is capable of inspecting data enabling systems to determine the data format, header information and dataset details. The Data Parser is also capable of obtaining a DataReader for data allowing systems to read data without having to know what the underlying format is. It can also create a DataWriter based on a requested output data format.

The Data Parser also offers the capabilities for creating a SDMX Schema for data validation, other data validation capabilites are offered which are in general much faster then using Schema. Data transformation capabilities are also provided allowing a system to tranform complete or subsets of datasets into any SDMX / SDMX-EDI formats in any version. SdmxSource takes advatage of Streaming technology, so there is no size limit for data processing.

Sample 1 – Get Data format/version

The following sample code will obtain data information from a data source. The Interface we use is the DataInformationManager

String url = "http://www.sdmxsource.org/samples/data/somedata.xml";
ReadableDataLocation dataLocation = new ReadableDataLocation(url);
//Get the data type (in this case GENERIC_2_0)
DATA_TYPE dataType = dataInformationManager.getDataType(dataLocation);

//Base Data format is GENERIC
BASE_DATA_FORMAT dataFormat = dataType.getBaseDataFormat();

//SDMX Version is 2.0
SDMX_SCHEMA version = dataType.getSchemaVersion();

Sample 2 – Get a Data Reader

The following sample code will get a Data Reader from a source of data. The Interface we use is the DataReaderManager

//1. Get a ReadableDataLocation for our data file
String dataUrl = "http://www.sdmxsource.org/samples/data/somedata.xml";
ReadableDataLocation dataLocation = new ReadableDataLocation(dataUrl);

//2. Get a ReadableDataLocation for our structure file
String structureUrl = "http://www.sdmxsource.org/samples/data/somedata.xml";
ReadableDataLocation structureLocation = new ReadableDataLocation(structureUrl);

//3. Create an inmemory bean retrieval mananger so our DataReader can get the relevant
//Data Structure for each dataset it is reading
SdmxBeanRetrievalManager beanRetrieval = new InMemoryRetrievalManager(structureLocation);

//Gets a DataReaderEngine. DataReaderEngine  is an interface, we don't need to know what the
//implementation is, but in this case it is the GenericDataReaderEngine
//The DataReaderManager in this application will only return SDMX or EDI DataReaders, but it is possible
//to inject a new implementation that can undertand other data formats, with NO change to the application
DataReaderEngine dataReader = dataReaderManager.obtainAppropriateReader(dataLocation, beanRetrieval);

Sample 3 – Data Transformation

The following sample code will transform data from Compact 1.0 to Generic 2.1 to SDMX-EDI and then to Compact 2.0

DataWriterEngine dwe;
DataReaderEngine dre;

dataReaderWriterTransform.copyToWriter()