From its beginnings as a descriptive framework for structure determination experiments, CIF has grown to be a powerful framework for many aspects of crystallographic practice. It describes or is actively used in raw data collection (such as diffraction images), data reduction (e.g. as structure factors or Rietveld profiles), structure solution and refinement, and the publication, database curation and visualisation of crystal and molecular structures. Although not the only format in common use in the structural sciences, it is central to structure determination of small-unit-cell and macromolecular structures, and its ontologic approach informs the best practice of information and data handling throughout crystallographic researches.

Brian McMahon is the Research and Development Officer at the International Union of Crystallography's offices in Chester, UK, and a former IUCr Representative to CODATA, the ICSU Committee on Scientific Data. He is Coordinating Secretary of COMCIFS and a Co-editor of International Tables for Crystallography Volume G: Definition and exchange of crystallographic data.

A jewel in the crown of structural biology, the Protein Data Bank is an unparalleled archive of protein and nucleic acid structures. Its operation depends on information-rich relational databases, the schemas for which are efficiently expressed in the macromolecular CIF (mmCIF) formalism. Discrete entities that may be stored in the databases are characterised in CIF-format data dictionaries, which express ontologies widely used in areas as diverse as structural genomics, NMR, cryo-electron microscopy and protein production.

John Westbrook is a Project Team Leader of the RCSB Protein Data Bank, and is based at Rutgers University. He has played key roles in creating and maintaining the PDB database schema, in developing many of the software tools that underpin PDB operation, and in developing formal ontologies with other structural biology communities. He is a member of COMCIFS.

Powder diffraction may well be the impoverished cousin of single crystal diffraction, but for many materials and measurements there is no other choice; structural analyses have been performed from powders for materials ranging from simple salts to proteins. Many other types of material characterizations can be performed with powder diffraction. Excellent instruments are available that measure powder diffraction in different ways, depending on the needs of the study and the type of illumination source. Successful modeling requires that data be kept in a form close to the original measurement, which creates complexity for a powder CIF dictionary (pdCIF).  It has been nearly 15 years since the initial pdCIF dictionary was completed and the successes and failures of pdCIF will be considered.

Brian Toby has had extensive professional experience in the chemical industry, academic and in the government sectors, where the latter included both synchrotron and research reactor facilities. His research interests are in understanding how the arrangements of atoms in solids determine how the material functions, chemically or physically, and for the development and teaching of techniques for those studies. To do this, he works on software and instrument development as well as conducts measurements and analyzes the results. He has collaborated with researchers in academia, industry and government  to produce 120 papers that have been cited nearly 6000 times. He is a former member of COMCIFS and was the leader of the powder CIF (pdCIF) dictionary development effort.

Crystal structure determination has become a high-throughput activity, and even at the level of the departmental laboratory, automation is increasingly important in managing the experiment and, crucially, the data collected from the experiment and its subsequent processing, analysis and dissemination. The UK National Crystallography Service is a medium-scale facility which needs to address issues of data management, accountability and dissemination, on top of its efforts to achieve best experimental practice. In providing a service to chemists as well as crystallographers, it has amassed considerable experience in cross-discipline ontology building, data publication via repository platforms, and integration with laboratory management systems.

Simon J. Coles is Head of the UK National Crystallography Service and a member of staff of the Department of Chemistry at Southampton University.

The IUCr Executive Committee has charged a Working Group with an assessment of the potential benefits of depositing raw experimental data sets (with initial emphasis on X-ray diffraction images), and the cost, technical and structural ramifications of doing so. There are a number of potential locations for depositing raw images that allow their reuse in validation, re-refinements, reanalysis for new science, education and software development – for example, in discipline-specific data centres, in large-scale instrument facilities, or in institutional repositories. These are not necessarily exclusive (for example, a central data centre might archive only data sets associated with published structures), and initiatives such as the Australian TARDIS demonstrate approaches to federating separate repository platforms. Crucial to interoperability between such federated archives will be well-defined metadata and procedural standards.

John R. Helliwell trained in physics and molecular biophysics and is now Emeritus Professor of Structural Chemistry at the University of Manchester. He is former Editor-in-Chief of the journals of the International Union of Crystallography and Past President of the European Crystallographic Association. His research involves crystallography methods developments applied to structural chemistry and biology. He is currently IUCr representative to CODATA (the ICSU Committee for Scientific Data) and ICSTI (the International Council for Scientific and Technical Information), and chairs the IUCr Diffraction Data Deposition Working Group. He is also a member of the CODATA/VAMAS Working Group on the description of nanomaterials.

The Rutherford Appleton Laboratory is a UK National Facility that handles a considerable amount of experimental research, and operates many instruments that generate large volumes of data such as the ISIS Pulsed Neutron Source, Central Laser Facility, and Diamond Light Source. It has developed a Core Scientific Metadata Model (CSMD) for the management of the data resources of the facilities in a uniform way. This talk discusses the emerging opportunities opened up by the availability of systematically managed data in a large laboratory setting and the technical barriers of making extensive use of them in real world open data infrastructure developments. The latest developments of CSMD will be presented to highlight the current direction we are undertaking in the context of the PaNData-ODI project, a collaborative European project involving thirteen major world class research laboratories that operate one or more neutron or photon sources in Europe.

Erica Yang is a senior computer scientist at the STFC Rutherford Appleton Laboratory, where she works in the Scientific Computing Department (SCD). She is also the national labs services liaison officer, responsible for developing a sustainable long term data services research and development strategy and roadmap for STFC's national laboratories. She has worked with the UK National Crystallography Service and the University of Cambridge to develop cross-organisation data infrastructure technologies for the UK structural science communities. She also manages and directs projects involving STFC's facilities and large scale data and HPC infrastructures. In EU FP7 project "PaNData-ODI", she works closely with international facilities (e.g. ESRF, ILL, DESY) to define and develop a fully integrated and cross-facility data management roadmap and services for EU photon and neutron communities.

A specific aspect of interoperability within large scientific facilities is the management of different data formats associated with different fields. NeXus, HDF5 and CIF are scientific data formats that  overlap in some areas (e.g. in the capture and management of X-ray diffraction images), and there is considerable benefit to be gained by harmonising their content and working towards maximum interoperability. Format conversion at the syntactic level can be a straightforward process, but truly interoperable software  applications require functional mappings between different representation standards. An initial approach to this is being attempted by  constructing a DDL2 dictionary, acting as a concordance between the existing NeXus and imgCIF formats. Full interconvertibility between NeXus and imgCIF relies on methods evaluation, and may be possible using the new dictionary definition language DDLm and an evaluation engine (such as dREL).

Herbert J. Bernstein is Professor of Mathematics and Computer Science at Dowling College, Oakdale, NY. He is a member of COMCIFS, Chair of the imgCIF dictionary working group, and lead developer of CIFtbx, a Fortran library for handling CIF data. He is also a member of the NeXus International Advisory Committee (NIAC).

Although this meeting focuses on the technical aspects of sound data management, because we believe the benefits of this are self-evident to scientists who are used to taking the greatest care in analysing and using data, we should be aware also of the general research policy framework in which so much modern research is located. At the level of national and international science policy, funding bodies increasingly require evidence for best practice in data management; likewise, open-access mandates for publications may become a reality. This presentation contextualises the practice of our science within the evolving policy framework for publicly funded research.

Simon Hodson is Executive Director of CODATA http://www.codata.org, an organisation whose mission is to strengthen international science for the benefit of society by promoting improved scientific and technical data management and use.  He also sits on the Board of Directors of the Dryad data repository http://datadryad.org, a not-for-profit initiative to make the data underlying scientific publications discoverable, freely reusable, and citable.  From 2009 to 2013, as Programme Manager, he led two successive phases of Jisc's innovative Managing Research Data programme http://researchdata.jiscinvolve.org/wp/.

Structural journals published by the IUCr have an efficient workflow built around the CIF standard. For small-unit-cell structures, authors submit their articles in this format, building on files created directly by their structure solution/refinement software. The processed experimental data from which the structure has been determined (structure factors, Rietveld profiles) are also uploaded in this format, allowing stringent technical peer review of the quality of the structural modelling. All published structures are accompanied by the underlying data that would permit their redetermination, and can be explored in the online publication through three-dimensional visualisation and analysis tools.

Michael A. Hoyland is a Systems Developer at the International Union of Crystallography, Chester, UK. He maintains the checkCIF web service for validating small-unit-cell structures, and is lead developer of the submission and review system for authors of IUCr journals.

When small structures are submitted for publication in IUCr journals, they are analysed as part of the peer review process by the software suite checkCIF. The goal of this analysis is to provide an objective assessment of the quality and reliability of the published structure, with reference both to the available experimental data and the level of consistency with known chemistry.

For this to work well, validation protocols have to keep up with advances in structure determination methodologies. Consequently, it is important that CIF tools and definitions are both practical and extended and revised regularly. The purpose and output of validation also have to be understood easily by users, reviewers and journal editors, some of whom may not be expert crystallographers.

Anthony Linden is a Research Group Leader at the X-ray Crystallography Facility, Institute of Organic Chemistry, University of Zurich, and Section Editor of Acta Crystallographica Section C.

For biological macromolecules, where structures and experimental data have traditionally been deposited in a central archive (the Protein Data Bank), integrating structural data with derivative publications is more complex. IUCr journals have developed an online publication tool that can extract deposited macromolecular data from the archive, and prompt the author for the additional information required for the fullest characterisation of a macromolecular structure determination. Again, the goal is to maximise the integrity of the scientific discussion.

Manfred Weiss works at the Institute for Soft Matter and Functional Materials of the Helmholtz Zentrum Berlin, and is a Section Editor of Acta Crystallographica Section F. He has been a Co-editor of Acta Crystallographica Section D since 2002, is a member of the IUCr Commission on Crystallographic Teaching and a Consultant for the IUCr Commission on Biological Macromolecules.

Increasingly, the structural biology community is aware that the quality of structures deposited in the Protein Data Bank (PDB) needs critical assessment using informative, well-defined and community-accepted validation methods. To obtain recommendations on validation methods and criteria to be applied to newly deposited as well as existing structures in the PDB, the Worldwide Protein Data Bank (wwPDB) has convened several Validation Task Forces (VTFs). The validation methods and criteria suggested by the wwPDB X-ray VTF will be used to validate all X-ray structures deposited in the PDB as well as all crystal structures already in the archive. The talk will highlight the importance of validation and describe the implementation of the recommendations of the X-ray VTF. Salient features of the validation reports generated by the new wwPDB X-ray validation pipeline will also be discussed. It is hoped that all journals that publish biomacromolecular structure papers will make submission of these reports mandatory whenever they receive a manuscript describing a new structure.

Sameer Velankar is a team leader at the EBI, responsible for content and integration of the Protein Data Bank in Europe (PDBe) resource. He earned his PhD in Structural Biology from the Indian Institute of Science in Bangalore, India in 1997, working on protein crystallographic studies of thymidylate synthase and triose phosphate isomerase. He then joined Dale Wigley's group in Oxford for post-doctoral research on the elucidation of the mechanism of DNA helicase. He has worked with and contributed to all parts of the PDBe team's operations, from annotation of newly deposited structures to the development of advanced PDBe services.

The Cambridge Structural Database of organic and metal-organic structures is one example of the curated archives of structural data that have been crucially important crystallographic and chemical resources for many decades. Many structural analyses based on the holdings in such databases rely on the quality and reliability of the individual structures. Historically, CCDC's Scientific Editors have worked hard to validate the individual depositions, a task greatly helped by the adoption of CIF as a community standard format. CCDC now applies probabilistic computational tools to determine the 'chemistry' represented by a CIF alongside expert human analysis.

Colin Groom is the Executive Director  of the Cambridge Crystallographic Data Centre. After a career in academia in the UK and in New Zealand, Dr Groom joined Pfizer where he established the protein crystallography group in the UK. He subsequently held various computational and informatics roles in the UK and US. Following this he joined Celltech/UCB, leading investigational chemistry and computer-assisted drug design groups.

The development of CIF has allowed IUCr journals to capture the structured semantic content of research articles to a very high degree. Hyperlinking between components of the scholarly article (the text, structural data model, experimental data) provides a sound technical basis for mining, validating and reusing scientific data with the help of high-volume robotic tools.

Peter Murray-Rust was until 2012 Reader in Molecular Informatics at the Unilever Centre for Molecular Informatics, University of Cambridge. He is co-developer with Henry Rzepa of Chemical Markup Language (CML) and a Consultant to COMCIFS.

When CIF was adopted as an information exchange standard by the IUCr in 1991, it was, if anything, ahead of the then state of the art. Free-format, extensible, low-overhead - it offered an easy route to implementation by scientific software, much still written in Fortran. By externalising the semantics of its tags to external dictionaries, it allowed ontology development to be decoupled from format, and thus had an important role to play in interoperability between purely crystallographic applications and the wider worlds of structural biology, chemical informatics and laboratory data management, as illustrated elsewhere in this symposium. The latest enhancements to CIF, with the introduction of an even more powerful dictionary definition language that support methods definitions and multiple data models, provide unrivalled potential for taking computer ontologies into completely new territories.

Nick Spadaccini is one of the main authors of the STAR File format, the STAR methods dictionary development language (DDLm) and the dictionary methods evaluator software dREL.