Citations

These sources were referenced directly to compile benchmarks and supplemental information about metadata quality frameworks.

• Bruce & Hillmann (2004). The Continuum of Metadata Quality: Defining, Expressing, Exploiting. https://www.ecommons.cornell.edu/handle/1813/7895

• Bruce & Hillmann (2013). Metadata Quality in a Linked Data Context. https://blog.law.cornell.edu/voxpop/2013/01/24/metadata-quality-in-a-linked-data-context/

• FAIR data principles: https://www.go-fair.org/fair-principles/

• Graham, P. S. (1990). Quality in cataloguing: Making distinctions. Journal of Academic Librarianship, 16(4), 213-218.

• McDonald (2011). 15 Ways to Think About Data Quality (Just for a Start). https://lists.w3.org/Archives/Public/public-lod/2011Apr/0140.html

• Moen, Stuart, McClure (1998). Assessing metadata quality: findings and methodological considerations from an evaluation of the US Government Information Locator Service (GILS). https://doi.org/10.1109/ADL.1998.670425

• Stvilia, Gasser, et al. (2007). A framework for information quality assessment. https://doi.org/10.1002/asi.20652

Other Resources

This (non-comprehensive) list of references includes a wide array of literature and other resources that may be helpful for organizations that are thinking about benchmarking projects, such as papers and articles related to metadata quality work and benchmarking processes within and outside the library sphere. We have also tried to include links to resources that could support specific goals that organizations may have for metadata quality or user interactions more generally.

Sources Related to Benchmarking

Library Resources

• Papers:

  • Ochoa, X., & Duval, E. (2009). Automatic evaluation of metadata quality in digital repositories. International Journal on Digital Libraries, 10(2), 67-91. doi: 10.1007/s00799-009-0054-4

  • Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002). Data quality assessment. Communications of the ACM, 45(4), 211-218.

• Websites:

  • Metadata Assessment Framework and Guidance: https://dlfmetadataassessment.github.io/projects/framework/

    Good starting point for assessment with general information specifically related to metadata based on the Bruce/Hillmann quality metrics.

  • “Quality” tiers based on user functionality for Europeana: https://pro.europeana.eu/post/publishing-framework

  • What is Metadata Assessment [blog], https://www.diglib.org/what-is-metadata-assessment/

  • Case Studies: Metadata Assessment [blog], https://www.diglib.org/case-studies-metadata-assessment/

  • Zotero library of resources about metadata assessment: https://www.zotero.org/groups/488224/metadata_assessment

Non-Library Resources

• “IT Benchmarking Explained: How to Assess Your IT Efforts”: https://www.bmc.com/blogs/it-benchmarking-metrics/

  More specific overview of benchmark processes applied to software and IT practice with examples and references to more information; this is helpful because software concepts are more closely aligned to digital library software/metadata.

• What is the Purpose of Benchmarking?: https://www.sixsigmadaily.com/what-is-the-purpose-of-benchmarking/

• Understanding the Purpose of Benchmarking: https://www.isixsigma.com/benchmarking/understanding-purpose-and-use-benchmarking/

Sources Related to Digital Libraries

Although the scope of the metadata benchmarks is focused on descriptive metadata, there are other aspects of design, infrastructure, etc. that may affect how users can interact with items in digital collections (regardless of metadata quality). Here are some other resources that may not be directly related to metadata quality, but could affect institutional or public user experiences.

Sources Related to Infrastructure

• Levels of Born-Digital Access: https://osf.io/hqmy4

• Recording & resources related to ADA Title II (website accessibility requirements): https://drive.google.com/drive/folders/1R0D5Y3F8jtvnS7Q1lKm1fAnRFKE-Trld

• Kussmann, C., Alliance, D. S., Graham, W., Atkins, W., & Reich, A. (2019, April 25). Levels of Digital Preservation Version 2.0. https://doi.org/10.17605/OSF.IO/QGZ98.

Sources Related to Reparative Metadata

• Inclusive Metadata Toolkit: https://osf.io/yf96h

• Zotero library of resources about equity & inclusion: https://www.zotero.org/groups/2394367/dlf_cei_resources/library

• Towards protocols for describing racially offensive language in UK public archives: https://link.springer.com/article/10.1007/s10502-019-09314-y

Sources Related to UX/UI

• DLF User Studies in Digital Libraries: https://docs.google.com/document/d/1AVlT3QlmhbcUmoRtZ0hpvEhpbD9_Eld5bHsa-JNHX24/edit?tab=t.0

• Surveying the Landscape: Use and Usability Assessment of Digital Libraries [White Paper]: https://osf.io/uc8b3

• Digital Libraries User Personas Best Practices: https://osf.io/wpb8q/

• Digital Content Reuse Assessment Framework Toolkit (D-CRAFT): https://reuse.diglib.org/

Sources Related to Non-Descriptive Metadata

• PREMIS dictionary for preservation metadata: https://www.loc.gov/standards/premis/

• Dublin Core (DCMI) overview of metadata types and resources: https://www.dublincore.org/resources/metadata-basics/

• List of metadata standards and resources: https://nkos.dublincore.org/Metadata-standards.html