Summary of Symbols Requiring Escaping in sed: A Comprehensive Guide to Points Where Beginners Get Stuck

Introduction

While sed is a powerful command that can streamline text processing, it often fails to work as expected due to "escaping." Handling symbols, in particular, is the first major hurdle for beginners.

In this article, we will explain the concept of escaping in sed while validating actual symbols. By understanding not just the "how" but the "why" behind it, you will gain the ability to apply these techniques to various scenarios.

Reference: GNU sed

Understanding the Mechanism of the sed Command

We will use the following command for this explanation:

sed 's/\//hello/' input.txt

The breakdown of this command's behavior is as follows:

Element	Content
s	Substitute command
/	Delimiter
\/	Searching for "/" (escaped)
hello	Replacement string

Since / is used as a delimiter in sed, it cannot be treated as a search target as is. Therefore, it is escaped using a backslash .

State Before Execution

First, let's create a file:

cat << 'EOF' > input.txt
.,*[]()!"#$%-=~^|@`{;+:*}<>?_&\/'
EOF

The content of input.txt is:

.,*[]()!"#$%-=~^|@`{;+:*}<>?_&\/'

State After Execution

Run the command:

sed 's/\//hello/' input.txt

The output result will be:

.,*[]()!"#$%-=~^|@`{;+:*}<>?_&\hello'

You can confirm that / has been replaced by hello.

Execution image

Symbols That Require Escaping in Search Patterns

In sed search patterns, certain symbols have special meanings as regular expressions. To treat them as literal characters, they must be escaped.

Symbol	Command Example	Meaning
.	sed 's/\./X/'	Any single character
*	sed 's/\*/X/'	Match 0 or more of the preceding character
$	sed 's/\$/X/'	End of line
^	sed 's/\^/X/'	Start of line
[	sed 's/\[/X/'	Start of character class
]	sed 's/\]/X/'	End of character class
\	sed 's/\\/X/'	Escape character
/	sed 's/\//X/'	Delimiter

Because these carry "special meanings," using them without an escape will result in unintended behavior.

Why Use Single Quotes (')?

Consider the following command:

sed 's/hello/$test/'

By wrapping the command in single quotes, $test is treated as a literal string.

On the other hand, if you use double quotes:

sed "s/hello/$test/"

This is because the shell performs variable expansion on $test. In other words, the value might change unexpectedly.

Therefore, it is generally safer to use single quotes in sed.

How to Specify a Single Quote (')

To handle a single quote itself, write it like this:

sed 's/'\''/hello/' input.txt

This can be broken down as:

• ' → Ends the string
• \' → Escaped single quote
• ' → Restarts the string

The key is to "close the string and then escape the character."

Differences Between BRE and ERE

By default, sed uses BRE (Basic Regular Expressions).

記号	BRE	ERE
()	Requires escape \(\)	Use as is
{}	Requires escape \{\}	Use as is
+	Requires escape \+	Use as is
?	Requires escape \?	Use as is

To use ERE (Extended Regular Expressions), add the -E option.

Behavior Comparison:

echo "(a)" | sed 's/(a)/X/'
echo "(a)" | sed -E 's/(a)/X/'

Symbols That Require Escaping in the Replacement String

First, create a file:

cat << 'EOF' > input.txt
X
EOF

In the replacement part, you must be careful with the following symbols:

Symbol	Command Example
&	sed 's/X/\&/'
\	sed 's/X/\\/'
/	sed 's/X/\//'

Example:

sed 's/X/\&/' input.txt

→ The & symbol means "the matched string," so an escape is required to treat it literally.

Changing the Delimiter

In sed, you can use characters other than / as delimiters.

sed 's|/|hello|' input.txt

When dealing with paths that contain many / characters, changing the delimiter improves readability. This is often recommended in professional practice.

For more details, please see our article "How to freely change the delimiter in sed | Writing methods other than / and practical techniques," which explains this in detail.

Quick Reference (Reverse Lookup)

Here are some common operations:

Example 1: Replace /

cat << 'EOF' > input.txt
a/b/c
EOF

sed 's/\//-/g' input.txt

Example 2: Replace $ at the end of a line

cat << 'EOF' > input.txt
test$
EOF

sed 's/\$/END/' input.txt

Example 3: Replace a dot

cat << 'EOF' > input.txt
a.b.c
EOF

sed 's/\./-/g' input.txt

Common Failure Examples

1. Forgetting to escape:

sed 's/./X/'

→ This targets every character.

2. Using double quotes:

sed "s/hello/$test/"

→ Triggers variable expansion.

3. Delimiter collision:

sed 's//home/user//tmp/'

This results in an error.

Summary: Mastering sed and Escaping Safely

The key to sed is understanding "which symbols have special meanings." Escaping is not just a rule for symbols, but a way to handle regular expression syntax correctly.

Keep these points in mind for professional use:

• Be careful with regex symbols in search patterns.
• Watch out for & and \ in replacement strings.
• Standardize on using single quotes.
• Delimiters can be changed flexibly.

By grasping these concepts, you will gain confidence in handling sed and escaping!