The School of Data Handbook advises us that:
While there are many different types of data, almost all processing can be expressed as a set of incremental stages. The most common stages include data acquisition, extraction, cleaning, transformation, integration, analysis and presentation. Of course, with many smaller projects, not each of these stages may be necessary.
Each of these stages is covered by multiple tools and methods, and every “data wrangler” will have their own favorites, furnished by experience, expertise, the day’s technology fashions, or just plain preference. While Information Technology moves at a rapid pace, software tools being quickly replaced as the industry moves on, the steps and techniques relevant to Data Literacy have remained constant.
This website makes some suggestions of tools which may be useful for each stage of the process of data wrangling, based on the School of Data pipeline pictured below. It is a community curated site which you can edit via GitHub, or just send us any suggestions or change requests.
Image by School of Data Switzerland CC BY-SA 4.0
More tips for your toolbox can be found in these places:
- School of Data CH forum
- The School of Data Handbook
- Prego! Tools for data journalism
- Wikidata: Tools
In our workshops and Data Expeditions, we cover tools and techniques like this - and introduce some of the principles by which the School of Data has helped thousands of people around the world to become data literate: to go beyond making decisions from charts and factoids, to discover and delight in worlds of information as we go beyond surface level.
Please contact us in case of any questions. Sign up to our public forum or newsletter to be notified of upcoming Expeditions and other Events. Invite your friends / colleagues / family too, they will thank you!
“I want to suggest that to write to your best abilities, it behooves you to construct your own toolbox and then build up enough muscle so you can carry it with you. Then, instead of looking at a hard job and getting discouraged, you will perhaps seize the correct tool and get immediately to work.” ― Stephen King, On Writing: A Memoir of the Craft
• Recommended by: Oleg Lavrovsky