The Data Parser is capable of inspecting data enabling systems to determine the data format, header information and dataset details. The Data Parser is also capable of obtaining a DataReader for data allowing systems to read data without having to know what the underlying format is. It can also create a DataWriter based on a requested output data format.
The Data Parser also offers the capabilities for creating a SDMX Schema for data validation, other data validation capabilites are offered which are in general much faster then using Schema. Data transformation capabilities are also provided allowing a system to tranform complete or subsets of datasets into any SDMX / SDMX-EDI formats in any version. SdmxSource takes advatage of Streaming technology, so there is no size limit for data processing.
The following sample code will obtain data information from a data source. The Interface we use is the DataInformationManager
String url = "http://www.sdmxsource.org/samples/data/somedata.xml"; ReadableDataLocation dataLocation = new ReadableDataLocation(url); //Get the data type (in this case GENERIC_2_0) DATA_TYPE dataType = dataInformationManager.getDataType(dataLocation); //Base Data format is GENERIC BASE_DATA_FORMAT dataFormat = dataType.getBaseDataFormat(); //SDMX Version is 2.0 SDMX_SCHEMA version = dataType.getSchemaVersion();
The following sample code will get a Data Reader from a source of data. The Interface we use is the DataReaderManager
//1. Get a ReadableDataLocation for our data file String dataUrl = "http://www.sdmxsource.org/samples/data/somedata.xml"; ReadableDataLocation dataLocation = new ReadableDataLocation(dataUrl); //2. Get a ReadableDataLocation for our structure file String structureUrl = "http://www.sdmxsource.org/samples/data/somedata.xml"; ReadableDataLocation structureLocation = new ReadableDataLocation(structureUrl); //3. Create an inmemory bean retrieval mananger so our DataReader can get the relevant //Data Structure for each dataset it is reading SdmxBeanRetrievalManager beanRetrieval = new InMemoryRetrievalManager(structureLocation); //Gets a DataReaderEngine. DataReaderEngine is an interface, we don't need to know what the //implementation is, but in this case it is the GenericDataReaderEngine //The DataReaderManager in this application will only return SDMX or EDI DataReaders, but it is possible //to inject a new implementation that can undertand other data formats, with NO change to the application DataReaderEngine dataReader = dataReaderManager.obtainAppropriateReader(dataLocation, beanRetrieval);
The following sample code will transform data from Compact 1.0 to Generic 2.1 to SDMX-EDI and then to Compact 2.0
DataWriterEngine dwe; DataReaderEngine dre; dataReaderWriterTransform.copyToWriter()