The data collection, handling, and management plan plays an important role within a research project. The plan provides a roadmap documenting the flow of data through the sequential phases of collection, storage, cleaning, reduction, analysis, and finally to archiving. Further, the management plan documents the relationships between all of the software tools and programs necessary to guide the data through this research life cycle. The data handling and management plan needs to be developed before a research project begins. The plan, however, can evolve as the researcher learns more about the data, and as new avenues of data exploration are revealed.
"The process of organizing, storing, retrieving The process of organizing, storing, retrieving and maintaining the data you collect and maintaining the data you collect
Having a data storage, management, and Having a data storage, management, and retrieval system is essential for every monitoring retrieval system is essential for every monitoring program (Volunteer WQ Monitoring factsheet) program (Volunteer WQ Monitoring factsheet)"
Data Management Software
Many DBMSs are available for personal computers. Options include:
- Spreadsheet (e.g., Excel, SPSS datasheet)
- Commercial database program (e.g., Oracle, Access)
- Specialty data entry program (e.g., SPSS Data Entry Builder, EpiData)
Spreadsheet are to be avoided for all but the smallest data systems since they are unreliable and easily corruped (e.g., easy to type over, lose track of records, duplicate data, mis-enter data, and so on. ). Commercially available database programs are expensive, tend to be large and slow, and often lack controlled data-entry facilities. Specialty data entry programs are ideal for data entry and storage. We use EpiData for this purpose because it is fast, reliable, allows for controlled data-entry, and is open-source. Use of EpiData is introduced in the accompanying lab.
- An important link between effective monitoring An important link between effective monitoring efforts and informed data analysis for assessing efforts and informed data analysis for assessing water quality is DATA MANAGEMENT water quality is DATA MANAGEMENT
- An important link between doing water quality An important link between doing water quality restoration activities and determining their restoration activities and determining their effectiveness is monitoring and more DATA effectiveness is monitoring and more DATA MANAGEMENT MANAGEMENT
Purpose of Data Management
Proper data handling and management is crucial to the success and reproducibility of a statistical analysis. Selection of the appropriate tools and efficient use of these tools can save the researcher numerous hours, and allow other researchers to leverage the products of their work. In addition, as the size of databases in transportation continue to grow, it is becoming increasingly important to invest resources into the management of these data. There are a number of ancillary steps that need to be performed both before and after statistical analysis of data. For example, a database composed of different data streams needs to be matched and integrated into a single database for analysis. In addition, in some cases data must be transformed into the preferred electronic format for a variety of statistical packages. Sometimes, data obtained from “the field” must be cleaned and debugged for input and measurement errors, and reformatted. The following sections discuss considerations for developing an overall data collection, handling, and management plan, and tools necessary for successful implementation of that plan.
The Future of Data Management
Organizations are discovering what digital startups and disruptors already know: Data is a valuable asset for identifying trends, making decisions, and taking action before competitors. The new position of data in the value chain is leading organizations to actively seek better ways to derive value from data.