Green Deal Data Observatory

Automated data observatory

Yes!Delft AI+Blockchain Validation Lab

Climate change and environmental degradation are an existential threat to Europe and the world. To overcome these challenges, the European Union created the European Green Deal strategic plan. It aims to make the EU’s economy sustainable by turning climate and environmental challenges into opportunities, and making the transition just and inclusive for all.

Our automated observatory want to support evidence-based policy making, and KPIs for business and policy-makers alike with truly usable open science and open governmental data. We want to make sure that environmental, socio-economic, political data is brought to easy-to-import, easy to compare, tidy formats. We use data that has been scientifically validated, and we aim to validate all our open-source processing code in scientific peer-review.

Our collaboration is open for individuals, citizens scientists, research institutes, NGOS, companies.

Download our description and open call for collaboration.

Get involved in services: our ongoing projects, team of contributors, open-source libraries and use our data for publications. See some use cases.

Follow news about us.

Contact us .

Services

Automated Data Services

[Professional Data Processing](/data/open-gov/#is-there-value-left-in-open-data)Uncut diamonds need to be polished. Data is only potential information, raw and unprocessed. — Professional Data Processing
Uncut diamonds need to be polished. Data is only potential information, raw and unprocessed.

[Data-as-Service](/services/data-as-service/)Reusable, easy-to-import, interoperable, always fresh data in tidy formats with a modern API. — Data-as-Service

Reusable, easy-to-import, interoperable, always fresh data in tidy formats with a modern API.

Metadata

Adding metadata exponentially increases the value of data. Did somebody already adjust old data to conform to constantly changing geographic boundaries? What are some practical ways of combining satellite sensory data with my organization’s records? And do I have the right to do so? Metadata logs the history of data, providing instructions on how to reuse it, also setting the terms of use. We automate this labor-intensive process applying the FAIR data concept.

Data Curation

We create high value key business and policy evaluation indicators. Scientific proofs require the combination of correctly matching, formatting, and verifying controlled pieces of data. Our data comes from verified and legal sources, with information about use rights and a complete history. You can always take a look at the processing code, too. We do not deal in blood diamonds.

Data Processing

We create high value key business and policy evaluation indicators. Scientific proofs require the combination of correctly matching, formatting, and verifying controlled pieces of data. Our data comes from verified and legal sources, with information about use rights and a complete history. You can always take a look at the processing code, too. We do not deal in blood diamonds.

Data-as-Service

We provide our clients with simple datasets, databases, harmonized survey data, and various other rich data applications; we provide them with continuous access to high-quality, re-processed, re-usable public sector and scientific data.

Recent Posts

stacodelists: use standard, language-independent variable codes to help international data interoperability and machine reuse in R

A new building block of our Green Deal Data Observatory went through code peer review and was released yesterday. The statcodelists R package aim to promote the reuse and exchange of statistical information and related metadata with making the internationally standardized SDMX code lists available for the R user.

Daniel Antal

Last updated on Jul 1, 2022 3 min read

stacodelists: use standard, language-independent variable codes to help international data interoperability and machine reuse in R

A reprezentative sample of n=100793 from 5 years on the most serious global problem. Get the tidy dataset from our repository or API. Subject Numbrer Datasets Environmental Protection 2/2 Government Budget Allocations for R&D in Environment Environment’s Share in Total Government Budget Allocations for R&D Climate change 1/1 Most Serious Global Problem: Climate Change (Percentage of European Individuals) For Climate change subject, as recognized by the Library of Congress Subject Headings, with variations Changes, Climatic, Changes in climate, Climate change, Climate change science, Climate changes, Climate variations, Climatic change, Climatic changes–Environmental aspects, Climatic fluctuations, Climatic variations, Global climate changes, Global climatic changes, we find 265 datasets, but after close inspection, only 25 are tabular data in csv or xlsx files.

Daniel Antal

Last updated on Jan 3, 2022 4 min read

100,000 Opinions on the Most Pressing Global Problem

Imagine if you could compare data easily from surveys taken about climate change from all European countries, maybe even from other continents, from different years? If you could work with a sample of not only n=1000, n=4000, or n=10,000 but n=100,000? What type of granularity it would give you about the perception of climate change or supported policy measures? That is exactly what our survey harmonization software allows for you to do.

Daniel Antal

Last updated on Nov 26, 2021 5 min read

100,000 Opinions on the Most Pressing Global Problem

How We Add Value to Public Data With Imputation and Forecasting?

Public data sources are often plagued with missng values. Naively you may think that you can ignore them, but think twice: in most cases, missing data in a table is not missing information, but rather malformatted information which will destroy your beautiful visualization or stop your application from working. In this example we show how we increase the usable subset of a public dataset by 66.7%, rendering useful what would otherwise have been a deal-breaker in panel regressions or machine learning applications.

Daniel Antal

Last updated on Dec 10, 2021 7 min read

How We Add Value to Public Data With Imputation and Forecasting?

How We Add Value to Public Data With Better Curation And Documentation?

Many people ask if we can really add value to free data that can be downloaded from the Internet by anybody. We do not only work with easy-to-download data, but we know that free, public data usually requires a lot of work to become really valuable. To start with, it is not always easy to find.

Daniel Antal

Last updated on Nov 10, 2021 5 min read

See all posts

Data

API to our data, our sources, how we process and validate it, how we increase its quality.

Survey Harmonization

Ex post survey harmonization is in a way a passive form of pooling research funding because you can utilize information from surveying that were made on somebody else’s expense. The aim of ex ante survey harmonization is to maximize the value from future retrospective harmonization; in a way, it is an active form of pooling research funding, because you benefit from money spent on related open governmental and open science survey programs.

Last updated on Jul 8, 2021

Data API

Get data from the Green Deal Data Observatory via our API

Daniel Antal, Botond Vitos

Last updated on Jul 8, 2021

Metadata

Uncut diamonds need to be cut, polished, and you have to make sure that they come from a legal source.

Daniel Antal

Last updated on Jul 8, 2021

Open Data

Many countries in the world allow access to a vast array of information, such as documents under freedom of information requests, statistics, datasets. In the European Union, most taxpayer financed data in government administration, transport, or meteorology, for example, can be usually re-used. More and more scientific output is expected to be reviewable and reproducible, which implies open access.

Last updated on Jul 8, 2021

Ongoing projects

See what we are doing, why we are doing it, and how you can contribute.

EU Datathon 2021

Reprex, a Dutch start-up enterprise formed to utilize open source software and open data, is looking for partners in an agile, open collaboration to win at least one of the three EU Datathon Prizes.

Make Coal History

Mapping coal mining communities

Mapping Sub-national Boundaries

Think globally, act locally - create indicators, analyze data below national level.

Contributors of the Green Deal Data Observatory

Join our open collaboration team as a data curator, developer or business developer! More about contributing: Automated Observatory Contributors’ Handbook.

developers

Andrés García Molina, PhD

Data Scientist & Ethnomusicologist

Botond Vitos

Data scientists and developer

Daniel Antal

Data Scientist & Founder of the Digital Music Observatory

Kasia Kulma

Contributor, data science and software engineering

Leo Lahti

rOpenGov coordinator

Pyry Kantanen

R package testing and data curation.

data curators

Istvan Simon

Data curator, climate risk management

István Bart

Climate policy data curator

Karel Volkaert

Climate policy and mitigation data curator

Stephan Okhuijsen

Data visualization and dissemination

service development team

Annette Wong

Contributor, digital strategist and product marketer

Maaike Lalleman

Coordinator, policy use cases

Paula Cooper

Open-source survey harmonization

Robin Nagy

Mentor, Contributor, Business Development

Suzan Sidal

Business Case Development & Service Design

institutional partners

Datagraver

Data dissemination partner

rOpenGov

rOpenGov network

join us

New Curators

Future curator

New Developers

Future co-developer

Observatory Business Associate

Future Team Member

Milestones

Green Deal Open Collaboration

Reprex and collaborators

Mar 2021 – Present the Hague, Netherlands

International Open Data Day 2021:

We organized the only event in the Netherlands
Decided to start an open collaboration on music and climate change.

Trustworthy AI PPP

NLAIC + EU AI Alliance

Feb 2021 – Present the Netherlands

Joined the Dutch & the European AI PPPs.

Use Cases

Data, Maps Produced & Their Uses

Green Deal Data Observatory, Daniel Antal

March 2021

Identifying Roadblocks to Net Zero Legislation

Our first use case is about identifying potential political roadblocks for climate policies. We are combining survey data about attitudes to climate change policies with socio-economic coal mining and and voting data. We examine the relationship between voter attitudes and economic dependency on coal mining.

Open-Source Software

Our peer-reviewed, open source statistical software packages

We believe that transparency is the key to the highest data quality. We use only open source software. We open up the critical elements of our software for peer-review.

We use open-source software, there is no vendor lock-in.
Our data products go through many, automated (unit) tests, replacing countless error-prone human validation working hours.
The critical elements of our code go through external validation and peer-review by computational statisticians and data scientists.

Quickly discover relevant content by filtering our software releases..

Daniel Antal

Last updated on Jul 7, 2021

retroharmonize R package for survey harmonization

The goal of retroharmonize is to facilitate retrospective (ex-post) harmonization of data, particularly survey data, in a reproducible manner.

Daniel Antal

Last updated on Jul 7, 2021

regions R package to create sub-national statistical indicators

regions currently takes care of 20,000 sub-divisional boundary changes in Europe since 1999.

Daniel Antal

Last updated on Jul 7, 2021

iotables R package for working with symmetric input-output tables

The goal of iotables is to make allow a programmatic acces to the symmetric input-output tables of Eurostat. It creates multipliers, calculates direct, indirect and induced effects from European SIOT tables.

Open-Source Software

Our peer-reviewed, open source statistical software packages

Quickly discover relevant content by filtering our software releases..

Recent & Upcoming Talks

Conversations with users, contributors and volunteers.

Reprex Open Data Day 2021

Open Data Day 2021 focusing on environmental, sustainability and public spending data mapping.

Mar 6, 2021 3:30 PM — 5:00 PM Online

Contact

Recent Publications

Quickly discover relevant content by filtering publications.

Daniel Antal, Rita Kiss (2009). Market size of the re-usable public sector information in Hungary. Market size of the re-usable public sector information in Hungary.

Follow Connect

Green Deal Data Observatory

Automated data observatory

Services

Recent Posts

Data

Ongoing projects

Contributors of the Green Deal Data Observatory

developers

Data Scientist & Ethnomusicologist

Data scientists and developer

Data Scientist & Founder of the Digital Music Observatory

Contributor, data science and software engineering

rOpenGov coordinator

R package testing and data curation.

data curators

Data curator, climate risk management

Climate policy data curator

Climate policy and mitigation data curator

Data visualization and dissemination

service development team

Contributor, digital strategist and product marketer

Coordinator, policy use cases

Open-source survey harmonization

Mentor, Contributor, Business Development

Business Case Development & Service Design

institutional partners

Data dissemination partner

rOpenGov network

join us

Future curator

Future co-developer

Future Team Member

Milestones

Use Cases

Open-Source Software

Open-Source Software

Popular Topics

Recent & Upcoming Talks

Contact

Recent Publications