solutions > data validation software and methods

Data Validation Software and Methods

Research and development of statistical methodology and software tools and systems for validation and editing of statistical data including outlier detection, data cleansing and advanced information models for formal validation rules.

• IST/RacWeb: Research and development of an advanced web-based platform (SOA and web services) which will improve risk assessment in customs declarations by enhancing the identification of risk profiles through the utilisation of data mining techniques (DG Information Society and Media, 6th Framework Programme for R&TD, 2006-2008)
• IST/iWebCare: An advanced web service based information system for fraud detection in public health care, based on advanced statistical and data mining methods. (DG Information Society, 6th Framework Programme for R&TD, 2006 - 2008)
• IST/Inspector: Research and development of a generic, distributed system for validation of large statistical data sets, consisting of a repository of validation rules and a validation engine, and based on a novel declarative approach for the storage of rules. (Eurostat, 5th framework program for Research, 2001-2003)
• New Technology for the Control of Data Quality: Methodological research work in the field of statistical data validation, including a new scheme for the classification of data validation rules as well as a data model for their formal declaration and storage. (Eurostat, 1999-2000)
• Mirror Outliers Detection Software: A software tool for the detection and classification of mirror outliers in intra-EU trade statistics stored in Eurostat's COMEXT database; methodological recommendations for improved outlier detection. (Eurostat, 2005-2006)

Also refer to projects "XML for Foreign Trade Statistics", "GENEDI" and "Harmonisation of validation for Foreign Trade statistics" (see section on "XML and EDI Technology for Statistical Data"), as well as to project "NSSG Integrated Information System" (see section on "Statistical Data Warehouses and Information Systems") for other implementations of statistical data validation.