makadeade
home about archive (feed)

Multiline editing with sed

Published: October 27th 2026 (week 44 of 2026) in programming

sed(1) is notorious for not allowing patterns containing the newline character, but there are ways to work around this limitation.

This post is inspired by that one time when I needed to edit more than a hundred files to insert a line between two other lines, which looks like the perfect job for a pipeline of find(1), xargs(1), and sed(1). The problem is, sed(1) is line-oriented, and does not allow multiline patterns.

Naturally, I went on the internet to see if there is a solution to this conundrum, and I found one stackoverflow. Actually, there were several suggested answers.

Arcane sorcery

The first answer was a small sed program utilising loops, branches, loading lines into the pattern space, and all that. The simplified version looks like this:

/foo$/ { N; s/foo\nbaz/foo\nbar\nbaz/ }

Looks fine, but there are probably edge-cases that can bite you in the ass.

It works by matching the first line of you desired pattern and, if that worked, executing a block of commands. Inside the block you load the second line into the pattern space using N (which loads the immediately following line into the pattern space), and then executing the substitution you want.

Simple enough, but I am not even going to entertain the notion of engaging in scripting sed(1) unless absolutely necessary. If this was the only way, I would have just written a short Python script to do the job, but, fortunately, that was not the case.

An abomination

The second answer was simply "Use perl instead of sed":

perl -pie 's/foo\nbaz/foo\nbar\nbaz/' file.txt
Technically legal, but fuck Perl

As the caption says, the police will not come haul you to jail if you use the above line, but fuck Perl. It is an abomination, a terribly-designed programming language, created by someone who was more concerned about being cute than correct. I once had the displeasure of working on a system written in Perl, and I learned to hate this language with every ounce of my being, body and soul, because of how atrocious it is. It seems to me as if Perl was designed to deceive and trick programmers using it, setting pitfalls everywhere, and putting loaded footguns wherever a pitfall would not fit. There is no cooperation between you and Perl, there is only struggle and strife. You have to become paranoid and put the language in the heaviest of shackles to achieve anything substantial, but why would you do it to yourself? Just use a different language.

Anyway.

A nice trick

The third answer was to use a tr(1) trick: translate newlines to some other character, run you sed(1) command, then translate back to newlines.

cat file.txt |
tr '\n' '\r' |
sed -e 's/foo\nbaz/foo\nbar\nbaz/' |
tr '\r' '\n'
The double tr(1) trick

While nice in theory, it is less useful in practice, because it does not allow modifying files in place. You would have to cook up a more complex pipeline with file renames if you need to modify more than a couple files.

The double tr(1) trick is definitely something to keep up your sleeve if you do not have access to GNU version of sed(1).

A reasonable thing

The fourth answer is the one I liked the most, and the one that solved my problem. If you have access to GNU sed(1) (and, in 2026, why would you not) you can just use the -z switch to change the line separator to NUL. With this one simple trick you can just use a simple, easily xargs(1)-able command:

sed -z -i 's/foo\nbaz/foo\nbar\nbaz/' file.txt
Simple and elegant

As in most cases, the GNU version of a classic Unix tool comes with some extra quality-of-life features.

Separation by nul

The possibility of switching from newline-separated to nul-separated records is not unique to sed(1). Equivalent flags exist for other programs too eg, find(1) has -print0 to switch from newlines to nuls on output, and xargs(1) has -0 to accept nul-terminated input. This is the most famous pair, but I suspect there exist other programs with similar mechanisms. It just makes sense.

nul vs null

Isn't nul a typo? Kind of, yes, but not really.

If you look at ascii(7) you will learn that:

Oct   Dec   Hex   Char
───────────────────────────────────────────
000   0     00    NUL '\0' (null character)
The name of the beast

So the name of the character encoded by a byte composed only of zero bits ie, a null character, is NUL.

Tags

Earlier under programming: Using Typst to render equations on this blog

Earlier under linux: Speed test using iperf

Previous: The Death of Cleaopatra (1890)