Dashboard
Open source tools
The Dashboard was designed and developed leveraging a set of open source software libraries, frameworks and applications. This tool is a part of the Arthub Flanders platform, designed by the Flemish Art Collection, in close cooperation with PACKED vzw. The Arthub Flanders platform consists of a set of tailor made tools with the primary goal of delivering collection metadata in an optimally reusable digital form. The entire toolset is made available to the GLAM community under an open source license. Cultural organisations whose digital strategy outlines opening up their collections are invited to include and adapt these tools in their own digital projects.
Page contents
Technology
The Dashboard is an application which analyzes collection metadata from the platform based on a specially developed measurement methodology and digitally visualizes this in a user-friendly way. The platform is built as an architecture of 'loosely coupled' components which automatically exchange and transform collection metadata and present this to people or machines through interfaces (API's).
The Dashboard
The Dashboard analyzes the aggregated collection metadata from the Datahub and provides a report for each data provider or source about the state of the provided data in function of its digital usability.
The basis for the application is the PHP-based Symfony Framework. Each day, the Dashboard automatically fetches the collection metadata from the Datahub through the OAI-PMH API and stores the relevant data in an underlying MongoDB database.
https://github.com/thedatahub/datahub
The Datahub
The Datahub is a web-based metadata aggregator. This applicaton aggregates and stores collection metadata from one or more providers or sources, and publishes the aggregated metadata via web-based API's.
The Datahub implements two API types. The application features aREST API which serves metadata in the JSON format. Metadata is also made available via an OAI-PMH API.
The Datahub front end is, just like the Dashboard, built on the Symfony Framework. Aggregated metadata is here persistently stored in a MongoDB database as well.
https://github.com/thedatahub/Datahub-Dashboard
Arthub Flanders
Arthub Flanders is a public discovery interface. It is the visual part of the platform and provides direct, user friendly access to the collection metadata.
Arthub Flanders is an instance of Project Blacklight tailored towards searching and browsing through cultural heritage objects. Blacklight is an open source Solr user interface discovery platform. Blacklight uses the Apache Solr search engine to search full text and/or metadata and has a highly configurable Ruby on Rails front-end. Blacklight is made public under an Apache 2.0 license.
https://github.com/VlaamseKunstcollectie/Arthub-Frontend
Catmandu
Catmandu is a data processing toolkit that allows access and conversion of datasets from one format or datamodel into another.
The Catmandu framework is Perl based and features an ecosystem of pluggable modules which cater to a wide array of formats and use cases.
The Datahub Factory
The Datahub Factory is a tailor made command line tool which makes it easier to configure and monitor complex ETL (Extract Transform Load) pipelines. This toolkit is Perl based and leverages Catmandu modules under the hood.
This command line tool is used to create ETL pipelines between collection registration systems, the Datahub and Arthub Flanders. The Datahub Factory is used to automatically transfer and transform collection metadata between these systems.
https://github.com/thedatahub/Datahub-Factory
https://github.com/thedatahub/Datahub-Factory-Pipelines
https://github.com/thedatahub/Datahub-Factory-Arthub
Open formats
Collection metadata is automatically exchanged between the various systems which are operational in the platform. Before collection metadata is exchanged, it is transformed to a single unified format: LIDO XML
LIDO is an XML harvesting schema. The schema is intended for delivering metadata, for use in a variety of online services, from an organization’s online collections database to portals of aggregated resources, as well as exposing, sharing and connecting data on the web.
Metadata harvested from a collection registration system, is transformed to LIDO XML by the Datahub Factory. The transformed metadata is then aggregated by The Datahub. Records are transformed via scripts which are written in Catmandu's Fix programming language.
A transformation script was created per institution. Each script defines a mapping between record fields provided by the collection registration system and the fields of the LIDO datamodel. The scripts also contains logic that determines when values are to be taken over into the LIDO record.
The scripts used by the Arthub Flanders platform are published on Github.
https://github.com/VlaamseKunstcollectie/Datahub-Fixes
Community of developers
All components are made available on Github, a popular platform for sharing and collaborating on software. We invite third party developers to contribute to the components. There are several ways to contribute.
- Bug reports, suggestions, questions and feedback can be submitted on a per component basis as separate issues. These issues can be submitted to the Github project issue tracker associated with the relevant component.
- Contributors are free to fork the components, make changes such as fixing bugs and adding new features, and contributing these changes back to the projects as Github pull requests.
In order to contribute to the projects, you are required to register an account on Github.
Licensing
Unless otherwise specified by the original authors, all software is made available as free software; you can redistribute it and/or modify it under the terms of the GNU General Public License (GPL) v3 license.
