Master Python Regular Expressions: Syntax, Flags, and Practical Usage

1. Regular Expression Syntax

1.1 Characters and Character Classes Special characters such as . ^ $ ? + * { } [ ] ( ) | must be escaped to be used literally. A character class is defined inside [] and matches any single character within it. Ranges can be specified, e.g., [a-zA-Z0-9]. A leading ^ inside the brackets negates the class, e.g., [^0-9]. Inside a class most characters lose their special meaning; ^ at the first position means negation, - denotes a range unless it appears first.

Shorthand character classes include . (any character except newline, or any character with the DOTALL flag), \d (Unicode digit, ASCII digit with the ASCII flag), \D (non‑digit), \s (Unicode whitespace, ASCII whitespace with the ASCII flag), \S (non‑whitespace), \w (Unicode word character, ASCII word character with the ASCII flag), and \W (non‑word character).

1.2 Quantifiers ? matches 0 or 1 of the preceding token. * matches 0 or more. + matches 1 or more. {m} matches exactly m times. {m,} matches at least m times. {,n} matches at most n times. {m,n} matches between m and n times. All quantifiers are greedy by default; appending ? makes them non‑greedy.

1.3 Groups and Capturing Parentheses ( ) group sub‑patterns and capture the matched text. Adding ?: after the opening parenthesis creates a non‑capturing group. Captured groups can be referenced later by number or by name (using (?P<name>...) and (?P=name)).

1.4 Assertions and Anchors Assertions do not consume characters but impose constraints. Common assertions include: \b (word boundary), \B (non‑word boundary), \A (start of string), ^ (start of line with MULTILINE), \Z (end of string), $ (end of line with MULTILINE), (?=…) (positive look‑ahead), (?!…) (negative look‑ahead), (?<=…) (positive look‑behind), (?<!…) (negative look‑behind).

1.5 Conditional Matching The syntax (?(id)yes|no) matches the yes pattern if the group with the given id has matched; otherwise it matches the no pattern.

1.6 Regular Expression Flags Flags can be passed to re.compile() using the bitwise OR operator, e.g., re.compile(r"#[da-f]{6}", re.IGNORECASE|re.MULTILINE), or embedded directly in the pattern with (?flags), e.g., (?ms). Common flags: re.ASCII (ASCII mode), re.IGNORECASE (case‑insensitive), re.MULTILINE (^ and $ match line boundaries), re.DOTALL (dot matches newline), re.VERBOSE (allows whitespace and comments in the pattern).

2. Python re Module

2.1 Core Functions Matching (boolean), searching, extracting, replacing, and splitting strings using regular expressions.

2.2 Using the re Module Two approaches: compile a pattern with re.compile() to obtain a regex object for repeated use, or use module‑level functions that accept the pattern string directly for one‑off operations.

2.3 Common Regex Object Methods findall() returns all matches as a list. finditer() returns an iterator of match objects. search() returns the first match object or None. match() checks for a match at the start of the string. sub() replaces matches with a replacement string or function. subn() returns a tuple (new_string, number_of_substitutions). split() splits the string by the pattern, including captured groups in the result if present.

2.4 Match Object Attributes and Methods Methods such as group(), groupdict(), groups(), start(), end(), span(), re(), and string() provide access to captured data and match metadata.

2.5 Summary Python’s re module does not return explicit true/false for matches; instead, match or search returning None indicates no match. Use search / match for single matches, finditer for multiple matches, sub / subn for replacements, and split for dividing strings, remembering that captured groups affect the split output.