Mastering grep: A Comprehensive Guide to Searching Directories

Introduction

grep is a powerful tool for searching for strings within files in Linux and Unix environments.

It is especially useful because it can search across multiple files in a directory, making it widely used for tasks like log analysis and checking configuration files.

This article explains the basics of directory searching with grep through to practical usage techniques, covering the points that beginners often find tricky.

Reference: GNU grep

Basic syntax for searching a specific string in a directory with grep

Create file

cat << 'EOF' > input.txt
sample text
error message
success log
EOF

Create file

mkdir -p logs

Create file

cp input.txt logs/app.log

Command to run

grep -r "error" logs

Output

logs/app.log:error message

How it works

Item	Description
grep	Text search command
-r	Recursively searches through a directory
"error"	String to search for
logs/	Target directory to search
Search flow	grep reads files within the directory in order and outputs matching lines

Explanation

Using grep -r lets you search for a specific string across all files under a specified directory.
Matching lines are displayed in the format "filename:content", so you can quickly pinpoint where the match occurred.

How to recursively search through a directory with grep

Create file

cat << 'EOF' > input.txt
sample text
error message
success
target keyword
EOF

Create file

mkdir -p logs/app

Create file

cp input.txt logs/app/app.log

Command to run

grep -r "target" logs

Output

logs/app/app.log:target keyword

Command to run

grep -rn "target" logs

Output

logs/app/app.log:4:target keyword

Command to run

grep -r --include="*.log" "target" logs

Output

logs/app/app.log:target keyword

How it works

Item	Description
grep	Command for searching text
-r	Recursively searches through a directory
-n	Displays the line number of each match
--include="*.log"	Only searches files with the specified extension
logs	Target directory to search
"target"	String to search for

Explanation

Using grep -r lets you recursively search all subdirectories and files under a specified directory.
This is a method that is frequently used for log investigation and configuration file searches.

Restricting grep to search only a specific directory

Create file

mkdir -p logs config

Create file

cat << 'EOF' > logs/app.log
INFO Start application
ERROR Database connection failed
INFO Retry connection
EOF

Create file

cat << 'EOF' > config/app.conf
database=mysql
port=3306
EOF

Command to run

grep 'ERROR' logs/app.log

Output

ERROR Database connection failed

Command to run

grep -r 'ERROR' logs

Output

logs/app.log:ERROR Database connection failed

Command to run

grep --include='*.log' -r 'ERROR' logs

Output

logs/app.log:ERROR Database connection failed

Command to run

grep -r 'database' config

Output

config/app.conf:database=mysql

Command to run

grep -r 'port' config

Output

config/app.conf:port=3306

How it works

Item	Description
logs/app.log	Used as the log file
config/app.conf	Used as the configuration file
grep 'ERROR' logs/app.log	Searches for a string within a single file
grep -r 'ERROR' logs	Recursively searches through the logs directory
--include='*.log'	Limits the search to only .log files
grep -r 'database' config	Searches the configuration data under the config directory

Explanation

grep can take a directory as a search target, not just a single file.
Using -r lets you search recursively through a directory, and --include lets you narrow down the target files.

Displaying file names when searching a directory with grep

Create file

cat << 'EOF' > input.txt
sample text
error message
success
EOF

Create file

mkdir -p logs

Create file

cat << 'EOF' > logs/app.log
application start
error message
application end
EOF

Create file

cat << 'EOF' > logs/system.log
system start
success
system end
EOF

Command to run

grep -r "error" logs

Output

logs/app.log:error message

Command to run

grep -rn "error" logs

Output

logs/app.log:2:error message

Command to run

grep -rH "success" logs

Output

logs/system.log:success

How it works

Item	Description
grep	Command for searching text
-r	Recursively searches through a directory
-H	Always displays the file name
-n	Displays the line number of each match
"error"	Search keyword
logs	Target directory to search

Explanation

Using grep -r lets you recursively search through every file in a directory.

Since the result shows both the matching content and the file name, you can quickly check which file contains the target string.

Displaying matching line numbers within a directory with grep

Create file

cat << 'EOF' > input.txt
apple
banana
orange
apple pie
grape
EOF

Create file

mkdir -p sample

Create file

cp input.txt sample/file1.txt

Create file

cat << 'EOF' > sample/file2.txt
melon
apple juice
peach
EOF

Create file

cat << 'EOF' > sample/file3.txt
strawberry
banana split
apple tart
EOF

Command to run

grep -rn "apple" sample

Output

sample/file2.txt:2:apple juice
sample/file3.txt:3:apple tart
sample/file1.txt:1:apple
sample/file1.txt:4:apple pie

Command to run

grep -rnH "banana" sample

Output

sample/file3.txt:2:banana split
sample/file1.txt:2:banana

How it works

Option	Description
grep	Command for searching a string
-r	Recursively searches through a directory
-n	Displays the line number of each match
-H	Displays the file name
"apple"	String to search for
sample	Target directory to search

Explanation

Using grep -rn lets you recursively search through all files in a directory and check the line numbers of matches.
This combination of options is frequently used for log analysis and configuration file investigation.

Ignoring case when searching a directory with grep

Create file

cat << 'EOF' > input.txt
Hello World
HELLO WORLD
hello world
Error Message
ERROR MESSAGE
error message
EOF

Create file

mkdir -p logs

Create file

cp input.txt logs/app.log

Command to run

grep -ri "hello" logs

Output

logs/app.log:Hello World
logs/app.log:HELLO WORLD
logs/app.log:hello world

Command to run

grep -ri "error" logs

Output

logs/app.log:Error Message
logs/app.log:ERROR MESSAGE
logs/app.log:error message

How it works

Item	Description
grep	Command for searching text
-r	Recursively searches through a directory
-i	Searches without distinguishing between uppercase and lowercase
grep -ri "hello" logs	Searches the files in the logs directory for "hello" while ignoring case
grep -ri "error" logs	Searches the files in the logs directory for "error" while ignoring case

Explanation

Using grep -ri lets you recursively search across all files under a directory while ignoring case differences.
This combination of options is commonly used for log investigation and configuration file searches.

Searching for an exact word match within a directory with grep

Create file

cat << 'EOF' > input.txt
sample text
error
warning
error
EOF

Create file

mkdir -p logs

Create file

cp input.txt logs/app.log

Create file

printf "error
success
" > logs/system.log

Command to run

grep -Rwx "error" logs

Output

logs/app.log:error
logs/app.log:error
logs/system.log:error

How it works

Item	Description
grep	Command that performs text search
-R	Recursively searches through a directory
-w	Searches for an exact word match
-x	Only displays results where the entire line matches the search term
"error"	Word to search for
logs	Target directory to search

Explanation

Using grep -Rwx lets you recursively search files under a specified directory and extract only exact word matches.
This is convenient when you want to precisely find a specific keyword in configuration files or log files.

Searching a directory with regular expressions using grep

Create file

cat << 'EOF' > input.txt
error: connection failed
info: startup complete
warning: disk usage 80%
error: timeout occurred
EOF

Create file

mkdir logs

Create file

cp input.txt logs/app.log

Command to run

grep -Er 'error|warning' logs

Output

logs/app.log:error: connection failed
logs/app.log:warning: disk usage 80%
logs/app.log:error: timeout occurred

How it works

Item	Description
grep	Text search command
-E	Uses extended regular expressions
-r	Recursively searches through a directory
'error|warning'	Regular expression that matches either "error" or "warning"
logs	Target directory to search

Explanation

Using grep -Er lets you run regular expression searches across multiple files under a directory.
This is a method that's commonly used for log analysis and bulk configuration file investigation.

Searching for multiple keywords within a directory with grep

Create file

cat << 'EOF' > input.txt
logs/app.log:INFO Application started
logs/app.log:ERROR Database connection failed
logs/web.log:WARN High response time detected
logs/web.log:ERROR Timeout occurred
config/settings.conf:database.host=localhost
config/settings.conf:database.port=5432
EOF

Command to run

grep -E "ERROR|database" input.txt

Output

logs/app.log:ERROR Database connection failed
logs/web.log:ERROR Timeout occurred
config/settings.conf:database.host=localhost
config/settings.conf:database.port=5432

Command to run

grep -r -E "ERROR|database" .

Output

./input.txt:logs/app.log:ERROR Database connection failed
./input.txt:logs/web.log:ERROR Timeout occurred
./input.txt:config/settings.conf:database.host=localhost
./input.txt:config/settings.conf:database.port=5432

How it works

Item	Description
grep	Command for searching text
-r	Recursively searches through a directory
-E	Uses extended regular expressions
ERROR|database	Specifies multiple keywords with an OR condition
.	Specifies the current directory as the search target

Explanation

Using grep -E lets you perform an OR search for multiple keywords with |.

Combining it with the -r option lets you search all the files within a directory at once.

Restricting grep to only specific file extensions within a directory

Create file

cat << 'EOF' > input.txt
src/main.py:print("hello")
src/app.py:def main():
src/config.yaml:version: 1.0
logs/app.log:ERROR occurred
README.md:sample document
EOF

Command to run

grep '.py:' input.txt

Output

src/main.py:print("hello")
src/app.py:def main():

Command to run

grep '.py:' input.txt | grep 'main'

Output

src/main.py:print("hello")
src/app.py:def main():

Command to run

grep -r --include="*.py" "main" .

Output

no output

How it works

Item	Description
grep	Command for searching the specified string
-r	Recursively searches through a directory
--include="*.py"	Only targets .py files for the search
"main"	Search keyword
.	Searches under the current directory

Explanation

When searching a directory with grep, use -r and restrict the target extension with --include.
This helps exclude unnecessary files, improving search speed and making results easier to read.

How to exclude specific files when searching a directory with grep

Create file

cat << 'EOF' > input.txt
sample text
error message
target string
EOF

Create file

mkdir logs

Create file

cat << 'EOF' > logs/app.log
application start
error message
EOF

Create file

cat << 'EOF' > logs/debug.log
debug mode
target string
EOF

Create file

cat << 'EOF' > logs/exclude.log
error message
excluded file
EOF

Command to run

grep -rn --exclude="exclude.log" "error message" logs

Output

logs/app.log:2:error message

Command to run

grep -rn --exclude="*.log" "target string" logs

Output

no output

Command to run

grep -rn --exclude-dir=".git" --exclude="exclude.log" "error message" .

Output

./input.txt:2:error message
./logs/app.log:2:error message

How it works

Option	How it works
-r	Recursively searches through a directory
-n	Displays the line number of each match
--exclude="exclude.log"	Excludes the specified file name from the search target
--exclude="*.log"	Excludes multiple files using a wildcard
--exclude-dir=".git"	Excludes everything under the specified directory from the search target

Explanation

When searching a directory with grep, you can use --exclude to leave out unnecessary files from the search.
Excluding log files or backup files improves search speed and makes the results easier to read.

How to exclude a specific directory when searching with grep

Create file

cat << 'EOF' > input.txt
ERROR Database connection failed
INFO Application started
ERROR External library error
print("hello")
EOF

Create file

mkdir -p logs vendor src

Create file

sed -n '1,2p' input.txt > logs/app.log

Create file

sed -n '3p' input.txt > vendor/lib.log

Create file

sed -n '4p' input.txt > src/main.py

Command to run

grep -r --exclude-dir=vendor "ERROR" .

Output

./logs/app.log:ERROR Database connection failed

Command to run

grep -r --exclude-dir={vendor,.git} "ERROR" .

Output

./logs/app.log:ERROR Database connection failed

How it works

Item	Description
grep -r	Recursively searches through the specified directory
--exclude-dir=vendor	Excludes the vendor directory from the search target
--exclude-dir={vendor,.git}	Excludes multiple directories from the search target
"ERROR"	String to search for
.	Searches under the current directory

Explanation

Using grep's --exclude-dir option lets you exclude specific directories while performing a recursive search.

Excluding things like library installation paths or .git directories prevents unnecessary search results.

Getting only the matching file names within a directory with grep

Create file

cat << 'EOF' > input.txt
error
warning
info
EOF

Create file

mkdir -p logs/app logs/system

Create file

cat << 'EOF' > logs/app/app.log
info start
error database connection failed
warning memory usage high
EOF

Create file

cat << 'EOF' > logs/system/system.log
info boot completed
error disk full
EOF

Create file

cat << 'EOF' > logs/system/debug.log
debug mode enabled
EOF

Command to run

grep -rl "error" logs

Output

logs/app/app.log
logs/system/system.log

Command to run

grep -rl "warning" logs

Output

logs/app/app.log

Command to run

grep -ril "ERROR" logs

Output

logs/app/app.log
logs/system/system.log

How it works

Option	Description
grep	Text search command
-r	Recursively searches through a directory
-l	Displays only the file name instead of matching lines
-i	Does not distinguish between uppercase and lowercase
"error"	Search keyword
logs	Target directory to search

Explanation

Using grep -rl lets you recursively search through a directory and obtain only the names of the files containing a match.
This is often used for identifying log files and searching configuration files.

How to show surrounding lines for search results within a directory with grep

Create file

cat << 'EOF' > input.txt
INFO: application start
INFO: loading config
ERROR: database connection failed
DETAIL: host=db01
DETAIL: timeout=30
INFO: retry start
INFO: retry success
WARN: high memory usage
ERROR: disk space low
DETAIL: mount=/data
INFO: cleanup completed
EOF

Command to run

grep -A 2 "ERROR" input.txt

Output

ERROR: database connection failed
DETAIL: host=db01
DETAIL: timeout=30
--
ERROR: disk space low
DETAIL: mount=/data
INFO: cleanup completed

Command to run

grep -B 2 "ERROR" input.txt

Output

INFO: application start
INFO: loading config
ERROR: database connection failed
--
INFO: retry success
WARN: high memory usage
ERROR: disk space low

Command to run

grep -C 2 "ERROR" input.txt

Output

INFO: application start
INFO: loading config
ERROR: database connection failed
DETAIL: host=db01
DETAIL: timeout=30
--
INFO: retry success
WARN: high memory usage
ERROR: disk space low
DETAIL: mount=/data
INFO: cleanup completed

Command to run

grep -r -C 1 "ERROR" .

Output

./input.txt-INFO: loading config
./input.txt:ERROR: database connection failed
./input.txt-DETAIL: host=db01
--
./input.txt-WARN: high memory usage
./input.txt:ERROR: disk space low
./input.txt-DETAIL: mount=/data

How it works

Option	Description
-A N	Displays N lines after the matching line
-B N	Displays N lines before the matching line
-C N	Displays N lines before and after the matching line
-r	Recursively searches through a directory
--	Separator between multiple search result blocks

Explanation

When searching within a directory with grep, use -r. If you also want to check the lines surrounding each result, combining it with -A, -B, or -C lets you efficiently review the context around when an error occurred.

Efficiently searching a directory with grep and find

Create file

cat << 'EOF' > input.txt
apple
banana
orange
error: file not found
warning: retry
EOF

Command to run

grep "error" input.txt

Output

error: file not found

Command to run

grep -n "warning" input.txt

Output

5:warning: retry

Command to run

find . -name "input.txt"

Output

./input.txt

Command to run

find . -type f | grep "input"

Output

./input.txt

How it works

Command	How it works	Purpose
grep "error" input.txt	Searches within a file for the specified string	Content search
grep -n "warning" input.txt	Displays matching lines along with their line numbers	Pinpointing matches
find . -name "input.txt"	Searches for a file name under a directory	File discovery
find . -type f | grep "input"	Narrows down find results using grep	Conditional search

Explanation

grep searches file contents, while find searches for files and directories themselves.
Combining the two lets you efficiently locate the information you need within a directory.

Techniques for speeding up directory searches with grep

Create file

cat << 'EOF' > input.txt
ERROR: Database connection failed
INFO: Application started
ERROR: Invalid user input
database.host=localhost
database.port=5432
EOF

Create file

mkdir -p project/config project/src

Create file

printf "ERROR: Database connection failed
INFO: Application started
" > project/app.log

Create file

printf "ERROR: Invalid user input
" > project/error.log

Create file

printf "database.host=localhost
database.port=5432
" > project/config/settings.conf

Create file

touch project/src/main.py project/src/test.py

Command to run

grep -r "ERROR" project

Output

project/error.log:ERROR: Invalid user input
project/app.log:ERROR: Database connection failed

Command to run

grep -r --include="*.log" "ERROR" project

Output

project/error.log:ERROR: Invalid user input
project/app.log:ERROR: Database connection failed

Command to run

grep -r --exclude-dir=src "database" project

Output

project/config/settings.conf:database.host=localhost
project/config/settings.conf:database.port=5432

How it works

Technique	Example command	How it works	Effect
Recursive search	grep -r "ERROR" project	Automatically traverses through a directory to search	Reduces manual searching
Restricting target files	grep -r --include="*.log" "ERROR" project	Only targets .log files for the search	Speeds things up by excluding unnecessary files
Excluding a directory	grep -r --exclude-dir=src "database" project	Excludes the specified directory from the search target	Shortens search time when there are many files

Explanation

Using grep -r lets you search an entire directory all at once.
You can further improve search speed by combining it with --include or --exclude-dir to leave out unnecessary files and directories.

Summary of grep and directory search key points

grep is a go-to command that streamlines searching for strings within a directory.

Once you understand the basic syntax, using recursive search, exclusion settings, and regular expression search as needed will greatly improve your efficiency.

It's recommended to start with basic searches and gradually work your way up to more advanced features.