CCI to host major data workshop: E-ARK

The event, “Is a Data Warehouse a Data Archive and Why Does It Matter?” will be co-hosted by the E-ARK project (coordinated in Portsmouth), the Digital Preservation Coalition (DPC) and the UK Data Archive.

They are delighted to invite you to a workshop on the preservation of relational databases. The event is free but you must register here.

Relational database management systems are one of the essential building blocks of information technology. Ubiquitous but often obscured behind layers of scripting, processing, forms and queries, they are arguably the most important invention of the Twentieth Century. It’s hard to think of a software application or service that does not have some fundamental dependency on database technologies.

Wikimedia Conference Berlin 2009 - MediaWiki Developer MeetingSo it’s surprising that the digital preservation community seems to have spent so little time explicitly considering their preservation. More accurately it’s surprising that there is less awareness and integration between the digital preservation community and the various tools and approaches used in commercial settings to manage the long term accessibility of records in database systems known as ‘data warehousing’. There’s no question that databases present a complex challenge to preservation. They can be difficult to document and difficult to understand even when they are documented. The complex interdependencies of data, query and scripting make migration problematic and highly specialised. Database migration is often seen as a purely technical operation, upgrading one legacy system with another soon-to-be-legacy replacement.

Relational databases and to some extent data warehousing approaches, which favour structure and homogeneity, are sometimes contrasted with ‘big data’ approaches that tend to favour heterogeneity and de-normalisation. It could be suggested that a concern with relational databases is outmoded and that the preservation community could simply adopt big data approaches. But the contrast can be overstated, especially when preservation issues are discussed. In practice ‘big data’ tools seem to offer improved workflows that complement rather than replace existing data warehouse tools. And even if ‘big data’ tools are the solution for access they still need to integrate with fundamental preservation processes and standards.

databaseBetter technical guidance and organisational know-how are needed if the digital preservation community is to offer confident and consistent solutions to long-term access for relational databases.

This two-part workshop, made possible by the E-ARK project and sponsored by the DPC, will review the state of the art in the preservation of databases and explore emerging themes around the preservation of ‘big data’.



The workshop will be split over two days:

Day one will:

  • Start out by clarifying where databases, data warehouses and big data complement / overlap each other
  • Review the state of the art in the preservation of databases
  • Present case studies of current tools and practices around the preservation of relational databases
  • Introduce commercial approaches to ‘data warehousing’ and explore the relationship with preservation
  • Introduce big data approaches for database preservation

Day two will:

  • Review the state of the art in the use of ‘big data’ and its implications for preservation
  • Examine and debate the use cases for archived databases / big data
  • Identify recommendations for further research and guidance in the preservation of ‘big data’

Interested parties are welcome to attend either or both days.

Leave your thought