Essential CLI Tools for Data Analysts

While graphical tools like Excel, Power BI, and Jupyter Notebooks are standard in data analysis, the command-line interface (CLI) remains one of the most powerful and fastest environments for exploring, filtering, and cleaning data.

Here are the top CLI tools every data analyst should integrate into their daily workflow.

1. csvkit: The Swiss Army Knife for CSVs

csvkit is a suite of command-line tools for converting to and working with CSV files. It brings database-like queries and inspection tools directly to your terminal.

1
# Preview column names and data types of a CSV
2
csvcut -n sales_data.csv
3

4
# Filter rows where region is 'West' and output specific columns
5
csvgrep -c "Region" -m "West" sales_data.csv | csvcut -c "Date","Customer","Amount"

With csvsql, you can even run SQL queries directly on your CSV files without importing them to a database first:

1
csvsql --query "SELECT Category, SUM(Amount) FROM sales_data GROUP BY Category" sales_data.csv

2. jq: Lightweight and Flexible JSON Processor

If you work with web APIs or NoSQL data sources, you’ll constantly encounter JSON. jq is the ultimate tool to slice, filter, map, and transform JSON datasets.

1
# Extract value of a specific key
2
curl -s https://api.github.com/repos/jayabelajar/muyoy | jq '.stargazers_count'
3

4
# Map and filter array elements
5
cat users.json | jq '[.[] | select(.active == true) | {name: .name, email: .email}]'

3. xsv: High-Performance CSV Handler Written in Rust

When dealing with large CSV files (millions of rows) where Python’s Pandas or Excel might struggle or slow down, xsv is incredibly fast. Written in Rust, it can index, slice, and partition large files in milliseconds.

1
# Quickly view descriptive statistics of a CSV file
2
xsv stats sales_large.csv
3

4
# Slice lines 10,000 to 10,500
5
xsv slice -i 10000-10500 sales_large.csv > sample.csv

4. ripgrep (rg): Fast Text Pattern Searcher

ripgrep is a line-oriented search tool that recursively searches your current directory for a regex pattern. It is invaluable for locating specific IDs, variables, or functions across raw log files and data exports.

1
# Search for a transaction ID across all log files in a directory
2
rg "TXN-90218" logs/

By mastering these simple terminal tools, you can automate minor cleaning and parsing tasks, saving significant time before feeding clean data into your scripting environments or BI tools.