Data access and distribution is an ongoing problem in science, affecting many research fields, from genomic information to microscopic images of rocks. Issues such as differing database schema and file formats, inability to customize or enforce laboratory terms of use, infrastructure failure and financial limitations have reduced public access to scientific data. Centralized solutions have been funded by government agencies in an attempt to expand access, but often, valuable resources cannot be published in repositories without being part of a peer-reviewed publication. Previously, we proposed to answer the demand for public access to raw scientific data using Open Index Protocol, a specification to publish metadata into a public blockchain-based ledger and host files in peer-to-peer file systems. With this method, there are 30TB of cryo-electron tomography datasets publicly available today. Now, we generalized this idea to let researchers publish any kind of scientific datasets using a distributed public ledger as a common index between interoperable databases. Here we describe a customizable gateway capable of exploring these distributed databases and publishing new records. The basic gateway design is built to be intuitively operable by academic and non-academic public alike, expanding the reach of the data distribution. As Open Index Protocol becomes a popular choice for data distribution by laboratories, focus on the user experience of the interface for data consumption will be key to achieve its full impact on society.
In the demo part of this presentation, we will demonstrate how to build a distributed database to share scientific data using Open Index Protocol, a specification to publish metadata on FLO blockchain and use a peer-to-peer file system for file storage and distribution. To begin, we will launch an instance of DDX and publish the metadata schema to the blockchain. Next, we will publish a few datasets to the database using the schema. Then, we will configure the explorer template and customize it to create a static webpage capable of exploring, searching & downloading the datasets published. A remote colleague will run another instance of DDX, configured to be compatible with the database we just created, and will use it to publish some records. We will be able to visualize their records in our own instance of DDX. Finally, we will show how to deploy and build a static website that serves as a gateway to visualize the records in the newly created database. In this brief demonstration, we will show the flexibility and power of this distributed resource for increasing access to raw datasets. Our main goal is to make it easy for researchers to participate in the effort to host and share their own data.
