Skip to content
LumenaryAcademy
Practice 70 min

Regular expressions

Perl's great strength in depth, from matching and captures to substitution, global matching, lookaround, and knowing when not to use a regex.

Code tested · 4AI-draftedAwaiting review by a computer scientist
MotivationH

Patterns as part of the language

A regular expression (regex, or pattern) describes a set of strings: "four digits, a dash, two digits", "a word that starts with a capital letter", "anything between angle brackets". With one you can ask whether a string contains such a thing, pull out the parts you care about, or replace them.

Most languages offer regular expressions through a library: you build a pattern object, call a method, and inspect a result object. Perl made them part of the syntax, with their own operators: =~ to apply a pattern, m// to match, s/// to substitute. That is a large part of why Perl became the language for text. A check, an extraction or a rewrite is one short line, so you use them freely, the way you use + or if.

Perl also pushed regular expressions far beyond their original form. Non-greedy quantifiers, lookahead and lookbehind, named captures, and the /x modifier for commented patterns all became widespread through Perl. In 1997 Philip Hazel wrote PCRE, Perl Compatible Regular Expressions, a C library that copied Perl's syntax; it, and the many engines that imitate it, is why the patterns in JavaScript, Python, PHP, Java and grep -P look so much like Perl's. Learn them well here and you have learned them almost everywhere.

This lesson goes deep, because this is the part of Perl you will use most. Take it in two sittings if you like: matching and captures first, then substitution, global matching and the advanced features.

DefinitionsH

The pattern language

Definition— Binding a pattern

STRING =~ /PATTERN/ tests whether PATTERN matches somewhere in STRING and returns true or false (in scalar context). !~ is the negation. Without =~, a pattern is matched against $_. The m of m/.../ is optional when you use slashes; with m you may choose other delimiters, such as m{...} or m|...|, which saves escaping when the pattern contains slashes.

Inside a pattern, most characters match themselves. These are the special ones you need first:

PatternMatches
.any character except a newline
\d, \w, \sa digit, a "word" character (letter, digit or underscore), a whitespace character
\D, \W, \Sthe opposite of each
[aeiou], [A-Za-z0-9], [^,]one character from a set or range; ^ first means "not in the set"
^, $the start and end of the string (or of a line, with /m)
\A, \zthe very start and very end of the string, whatever the modifiers
\ba word boundary: between a \w and a \W character
*, +, ?the previous item 0 or more times, 1 or more times, 0 or 1 time
{3}, {2,5}, {2,}exactly 3, between 2 and 5, at least 2 times
*?, +?, ??, {2,5}?the same, but non-greedy: as few as possible
a|beither a or b
( ... )a group that also captures what it matched into $1, $2, …
(?: ... )a group that does not capture
(?<name> ... )a named capture, available as $+{name}

To match a special character literally, put a backslash before it: \. is a full stop, \$ a dollar sign, \( a parenthesis.

Modifiers go after the closing delimiter and change how the whole pattern behaves:

ModifierEffect
/iignore case
/gglobal: find every match (see below)
/xextended: spaces and # comments inside the pattern are ignored, so you can lay it out readably
/mmulti-line: ^ and $ match at every line start and end inside the string
/ssingle-line: . also matches a newline
/rwith s///, return the changed string instead of changing the original (Perl 5.14)
/ewith s///, treat the replacement as Perl code
Worked examplesH

Matching and capturing

perl
use strict;
use warnings;
use feature qw(say);

my @lines = (
    '2026-03-14 ERROR disk /dev/sda1 is 97% full',
    '2026-03-14 info  backup finished in 42s',
    '2026-03-15 Error login failed for user amara',
    'not a log line',
);

for my $line (@lines) {
    if ($line =~ /error/i) {                       # /i: ignore case
        say "has an error: $line";
    }
}

# Captures: parentheses remember what they matched.
for my $line (@lines) {
    if ($line =~ /^(\d{4})-(\d\d)-(\d\d)\s+(\w+)/) {
        say "year $1, month $2, day $3, level ", lc $4;
    } else {
        say "no date in: $line";
    }
}

# In list context a match returns the captures.
my ($percent) = $lines[0] =~ /(\d+)%/;
say "disk usage: $percent";

# Named captures and /x for a readable pattern.
my $entry = qr{
    ^ (?<date>  \d{4}-\d\d-\d\d )   # the date
    \s+ (?<level> [A-Za-z]+ )        # a word: the level
    \s+ (?<text>  .* ) $             # the rest of the line
}x;
if ($lines[1] =~ $entry) {
    say "date=$+{date} level=$+{level} text='$+{text}'";
}

# Greedy versus non-greedy.
my $html = '<b>bold</b> and <i>italic</i>';
my ($greedy) = $html =~ /<(.+)>/;
my ($lazy)   = $html =~ /<(.+?)>/;
say "greedy: $greedy";
say "lazy:   $lazy";

It printed:

text
has an error: 2026-03-14 ERROR disk /dev/sda1 is 97% full
has an error: 2026-03-15 Error login failed for user amara
year 2026, month 03, day 14, level error
year 2026, month 03, day 14, level info
year 2026, month 03, day 15, level error
no date in: not a log line
disk usage: 97
date=2026-03-14 level=info text='backup finished in 42s'
greedy: b>bold</b> and <i>italic</i
lazy:   b

Captures, two ways

After a successful match, $1 holds what the first pair of parentheses matched, $2 the second, and so on, counting opening parentheses from the left. That is the scalar-context style, used inside an if.

In list context, a match returns the list of captures, so my ($percent) = $line =~ /(\d+)%/; both matches and extracts in one line. If the match fails, the list is empty and $percent is undef. This form is usually clearer, because the variable gets a real name immediately.

Named captures (Perl 5.10) go one step further: (?<date> ...) stores the match in the special hash %+ as $+{date}. Combined with /x, which lets you spread a pattern over several lines with comments, a complicated pattern becomes readable. qr{...}x compiles the pattern once into a value you can store in a variable and reuse; more on that below.

Greedy and non-greedy

Quantifiers are greedy: .+ grabs as much as it can and gives back only what it must for the rest of the pattern to match. In <(.+)>, the .+ runs to the end of the string and backs off just to the last >, so it captures almost everything. Adding ? makes the quantifier non-greedy (or lazy): .+? takes as little as possible, so it stops at the first >. A negated class is often even better, and faster: <([^>]+)> says exactly "characters that are not >".

Misconception

"$1 is cleared when a match fails." It is not. $1 and friends keep the values from the last successful match. Code like $line =~ /id=(\d+)/; save($1); silently saves a stale value from an earlier line when the current line has no id. Always use captures inside the if that tested the match, or use the list form, which gives undef on failure.

Worked examplesH

Substitution and transliteration

s/PATTERN/REPLACEMENT/ finds the first match and replaces it; with /g, every match. The replacement is like a double-quoted string, so it can use $1, $2 and the case-changing escapes \U (upper case until \E), \L (lower case), \u and \l (next character only).

perl
use strict;
use warnings;
use feature qw(say);

my $text = "colour, favour   and   honour";
(my $us = $text) =~ s/our\b/or/g;        # copy, then substitute on the copy
$us =~ s/\s+/ /g;                        # squeeze runs of spaces
say "US spelling: $us";

my $shout = $text =~ s/(\w+)/\U$1/gr;    # /r returns the new string, leaves $text alone
say "unchanged:   $text";
say "upper case:  $shout";

# /e: the replacement is Perl code.
my $prices = "tea 2.50, rice 1.20, fish 7.00";
(my $doubled = $prices) =~ s/(\d+\.\d+)/sprintf "%.2f", $1 * 2/ge;
say "doubled: $doubled";

# Swap with captures; rearrange dates from DD/MM/YYYY to YYYY-MM-DD.
my $dates = "due 05/11/2026, paid 28/10/2026";
$dates =~ s{(\d\d)/(\d\d)/(\d{4})}{$3-$2-$1}g;
say $dates;

# tr/// works on characters, not patterns: translate or count.
my $dna = "GATTACAGATTACA";
(my $rna = $dna) =~ tr/T/U/;
my $gc = ($dna =~ tr/GC//);              # counts, changes nothing
say "RNA: $rna, G or C: $gc of ", length $dna;
(my $complement = reverse $dna) =~ tr/ACGT/TGCA/;
say "reverse complement: $complement";

It printed:

text
US spelling: color, favor and honor
unchanged:   colour, favour   and   honour
upper case:  COLOUR, FAVOUR   AND   HONOUR
doubled: tea 5.00, rice 2.40, fish 14.00
due 2026-11-05, paid 2026-10-28
RNA: GAUUACAGAUUACA, G or C: 4 of 14
reverse complement: TGTAATCTGTAATC

Points to notice:

  • s/// changes the string it is bound to and returns the number of substitutions, not the new string. To keep the original, either copy first with the idiom (my $copy = $original) =~ s/.../.../; or use /r, which leaves the original alone and returns the result.
  • With /e, the replacement is code, run once per match. sprintf "%.2f", $1 * 2 reformats each price. That is how you do arithmetic inside a substitution.
  • Choosing braces as delimiters, s{...}{...}, means the slashes in dates need no escaping.
  • tr/// (also spelt y///) is not a regex at all. It maps characters one-for-one: tr/ACGT/TGCA/ swaps each base for its complement in a single pass. It returns how many characters it matched, so tr/GC// with an empty replacement simply counts G and C. Bioinformatics code uses exactly these two lines all the time.
Worked examplesH

Every match: /g, pos, \G and split

The /g modifier means "all matches", and what it does depends on context: in list context, it returns every match at once; in scalar context, each evaluation finds the next match, remembering where it stopped. That position is available, and settable, as pos($string), and \G in a pattern anchors to it.

perl
use strict;
use warnings;
use feature qw(say);

my $tweet = "Loving #perl and #regex today! #Perl #til";

my @tags = $tweet =~ /#(\w+)/g;          # list context: every capture
say "tags: @tags";

my %count;
$count{lc $1}++ while $tweet =~ /#(\w+)/g;   # scalar context: one match per pass
say join ', ', map { "$_=$count{$_}" } sort keys %count;

my $n = () = $tweet =~ /#/g;             # count matches
say "hash signs: $n";

# pos() and \G: walk a string token by token.
my $expr = "12+345-6";
pos($expr) = 0;
my @tokens;
while (pos($expr) < length $expr) {
    if    ($expr =~ /\G(\d+)/gc) { push @tokens, "NUM($1)" }
    elsif ($expr =~ /\G([-+])/gc) { push @tokens, "OP($1)" }
    else  { die "unexpected character at ", pos($expr) }
}
say "tokens: @tokens";

# split takes a pattern; a capture in it keeps the separators.
say join '|', split /\s*;\s*/, "a; b ;c;  d";
say join '|', split /([-+])/, $expr;
say join '|', split //, "abc";

# Build patterns from parts with qr//, and quote user text with \Q...\E.
my $word   = qr/[A-Za-z]+/;
my $number = qr/\d+(?:\.\d+)?/;
my $pair   = qr/($word)=($number)/;
my $config = "width=80 height=24.5 name=box";
while ($config =~ /$pair/g) { say "  $1 is $2" }

my $search = "1+1";                      # user input with a metacharacter
say "plain: ",  ("1+1=2" =~ /$search/   ? "match" : "no match");
say "quoted: ", ("1+1=2" =~ /\Q$search\E/ ? "match" : "no match");

It printed:

text
tags: perl regex Perl til
perl=2, regex=1, til=1
hash signs: 4
tokens: NUM(12) OP(+) NUM(345) OP(-) NUM(6)
a|b|c|d
12|+|345|-|6
a|b|c
  width is 80
  height is 24.5
plain: no match
quoted: match

What each part shows

  • List-context /g returns all the captures: the quickest way to extract every hashtag, number or email address.
  • Scalar-context /g in a while loop visits matches one at a time, which lets you do more work per match and uses no extra memory on huge strings.
  • my $n = () = ... is the "countof" idiom: assigning to an empty list puts the match in list context, and a list assignment in scalar context gives the number of elements on its right.
  • A tokenizer with \G and /gc. \G says "match exactly where the last match ended", so each pattern must start precisely at the current position. The /c modifier keeps pos unchanged when a match fails, so the elsif can try the next kind of token at the same place. This is the classic way to write a small lexer in Perl.
  • split /PATTERN/, STRING cuts a string wherever the pattern matches. Capturing parentheses in the pattern keep the separators in the output, and an empty pattern splits into characters. Remember the special case from lesson 4: split ' ' splits on runs of whitespace and ignores leading spaces.
  • qr// compiles a pattern into a value. Interpolating one qr into another, as $pair does, lets you build big patterns from small named, tested pieces.
  • \Q ... \E escapes every metacharacter in the interpolated text (the function form is quotemeta). The plain search failed because + in 1+1 was read as "one or more 1s". Whenever text from a user, a file or a database goes into a pattern and is meant literally, quote it. Forgetting this is a real security bug: a malicious pattern can make your program hang (see below).
Worked examplesH

Lookaround, anchors and boundaries

Lookahead (?= ...) and lookbehind (?<= ...) check what follows or precedes the current position without consuming it; (?! ...) and (?<! ...) are their negations. They let a pattern say "here, where a digit is behind me and groups of three digits are ahead".

perl
use strict;
use warnings;
use feature qw(say);

# Lookahead and lookbehind test what is around a position without consuming it.
sub commify {
    my ($n) = @_;
    1 while $n =~ s/^(\d+)(\d{3})/$1,$2/;     # classic loop: move one comma at a time
    return $n;
}
sub commify2 {
    my ($n) = @_;
    $n =~ s/(?<=\d)(?=(?:\d{3})+$)/,/g;       # one pass with lookarounds
    return $n;
}
say commify(1234567), ' ', commify2(1234567), ' ', commify2(999), ' ', commify2(1000);

# A password rule with lookaheads: at least 10 characters, a digit and a letter.
for my $pw (qw(sunflower42x short1 onlyletterslong 2026-tea-time)) {
    my $ok = $pw =~ /^(?=.*\d)(?=.*[A-Za-z]).{10,}$/;
    printf "%-16s %s\n", $pw, $ok ? 'accepted' : 'rejected';
}

# Anchors: validate the whole string, not just part of it.
for my $code (qw(AB-1234 ab-1234 XAB-12345 AB-12)) {
    my $loose  = $code =~ /[A-Z]{2}-\d{4}/   ? 'yes' : 'no';
    my $strict = $code =~ /\A[A-Z]{2}-\d{4}\z/ ? 'yes' : 'no';
    say "$code: loose $loose, strict $strict";
}

# Word boundaries and alternation.
my $sentence = "The cat scattered the cats' catalogue.";
my @whole = $sentence =~ /\b(cat)\b/g;
my @any   = $sentence =~ /(cat)/g;
say "whole word 'cat': ", scalar @whole, ", anywhere: ", scalar @any;

It printed:

text
1,234,567 1,234,567 999 1,000
sunflower42x     accepted
short1           rejected
onlyletterslong  rejected
2026-tea-time    accepted
AB-1234: loose yes, strict yes
ab-1234: loose no, strict no
XAB-12345: loose yes, strict no
AB-12: loose no, strict no
whole word 'cat': 1, anywhere: 4

commify2 inserts a comma at every position that has a digit behind it and a multiple of three digits ahead of it up to the end. Nothing is consumed, so the replacement inserts a comma without deleting anything. The password rule stacks two lookaheads at the start, each scanning ahead independently, so the order of the requirements in the string does not matter.

The anchor test is the most practically important part. XAB-12345 contains a valid-looking code, so the loose pattern accepts it. For validation, anchor both ends with \A and \z. Prefer them to ^ and $ there: $ also matches just before a final newline, so "AB-1234\n" would pass a ^...$ check.

Common misconceptionsH

When not to use a regular expression

Regular expressions are a sharp tool, and experienced Perl programmers are as quick to put them down as to pick them up.

Misconception

"I can parse HTML, JSON or CSV with a regex." For a quick look at well-behaved data, sometimes. In general, no: these formats nest, quote and escape in ways a regex cannot follow reliably (a comma inside a quoted CSV field, a > inside an HTML attribute). Use a parser: JSON::PP (core) for JSON, Text::CSV from CPAN for CSV, and an HTML parser such as HTML::TreeBuilder or Mojo::DOM for HTML.

Misconception

"Regexes are always fast." Most are. But Perl's engine is a backtracking engine: when part of a pattern fails, it goes back and tries other ways to divide up the text. A pattern with nested quantifiers, such as ^(\w+\s?)+$, can need exponentially many tries on a string that almost matches. This is called catastrophic backtracking, and it has taken real websites offline when an attacker sent a carefully chosen input. Defend with anchored, specific patterns ([^,]* instead of .*), avoid quantifiers nested inside quantifiers over the same characters, and never interpolate untrusted text without \Q.

Some other habits:

  • Use /x for anything longer than a line, with comments. Your future self will thank you.
  • Name the pieces with qr// and test them separately.
  • Use index, substr and eq for fixed strings. index($s, 'needle') >= 0 is clearer and faster than a regex with no special characters.
  • Mind Unicode. By default \w and \d can match letters and digits from any script when a string holds Unicode characters, so \d might match an Arabic-Indic digit. If you mean ASCII digits, write [0-9] or add the /a modifier. Reading text correctly as Unicode is part of the next lesson.
Exercises

Exercises

1. Regular expressions

Warm-upComputational

What does this program print?

use strict;
use warnings;
use feature qw(say);
my $s = "Order 66 shipped to Lot 7";
my @n = $s =~ /(\d+)/g;
say "@n";

2. Regular expressions

CoreComputational

What does this program print?

use strict;
use warnings;
use feature qw(say);
my $t = "a-b-c-d";
my $k = ($t =~ s/-/+/g);
say "$k $t";

3. Regular expressions

CoreComputational

What does this program print?

use strict;
use warnings;
use feature qw(say);
my $p = "path/to/the/file.txt";
my ($long)  = $p =~ m{^(.*)/};
my ($short) = $p =~ m{^(.*?)/};
say "$long $short";

4. Regular expressions

Warm-upComputational

What number does this program print?

use strict;
use warnings;
use feature qw(say);
my $w = "Mississippi";
my $n = ($w =~ tr/s//);
say $n;

5. Regular expressions

CoreHuman

A form field must contain exactly a five-digit postcode and nothing else. Which pattern should the program use to accept it?

Choose one

6. Regular expressions

StretchComputational

What does this program print?

use strict;
use warnings;
use feature qw(say);
say join '|', split /(,)/, "a,b,,c";

7. Regular expressions

StretchComputational

What does this program print? (Ignore any trailing space.)

use strict;
use warnings;
use feature qw(say);
my @lines = ('id=7 ok', 'no id here', 'id=9 ok');
for my $line (@lines) {
    $line =~ /id=(\d+)/;
    print "$1 ";
}
print "\n";

8. Regular expressions

CoreComputational

What does this program print?

use strict;
use warnings;
use feature qw(say);
my $csv = "  Lagos ;Accra;  Dakar  ";
my @cities = map { s/^\s+|\s+$//gr } split /;/, $csv;
say join ',', @cities;

Further reading and resources

  • perlretutThe official regular expressions tutorial, from the first match to advanced features.
  • perlreThe complete reference for Perl's pattern syntax and modifiers.
  • perlrequickA one-page quick start to Perl regular expressions.
  • perlop: Regexp Quote-Like OperatorsThe exact behaviour of m//, s///, qr// and tr///, including every modifier.
  • PCRE2The Perl Compatible Regular Expressions library, which brought Perl's pattern syntax to many other programs and languages.