After collection, most data requires some degree of cleaning or reformatting before it can be analyzed or used to create visualizations. Surveys of data scientists show that data cleaning can take anywhere from 15% (Kaggle 2018) to 80% (Crowdflower 2016) of their time working with data. Smaller (<5,000 records or so) data sets can be cleaned in a spreadsheet program like Excel; larger data sets require the processing power of a programming tool like R, Python, or SPSS. Useful data cleaning resources include:
