regex.bastardsbook.comAn introduction to regular expressions : The Bastards Book of Regular Expressions

-- The Bastards Book of Regular Expressions A free guide to finding patterns in text by Dan Nguyen . Download it at Leanpub . The windows of the Standard Hotel, in Chelsea Tweet Everday text patterns About a year ago, I started writing an online book for people who wanted to learn Ruby (the programming language) and how to use it for practical tasks. The longest chapter in that book so far is about regular expressions . And it begins with this advice: If you learn nothing else from this book, learn regular expressions. Since regular expressions are plenty useful on their own without knowing Ruby (or any other kind of programming or software), I’ve decided to write a whole separate introductory-level book on them. It’s called the Bastards Book of Regular Expressions ) and you can download it as an e-book here – for free – at Leanpub, where I am self-publishing it . (If you don’t want to go through the (easy) Leanpub signup process, you can download the PDF here , though Leanpub will have the most up-to-date version in multiple formats.) Note: This book is a work in progress and still in the pre-release stage. Most of the main content is finished but I’m still adding content/code and still need to edit it. Leanpub refreshes the downloadable book draft as I revise it. You can follow me at @dancow and/or @bastardsbook for updates. What are regular expressions? The easiest way to describe them is: “Find-and-Replace on steroids”. A longer description: Regular expressions – regexes for short – are just a way to describe patterns in text , either to find or to replace. Their general usefulness is easier told through demonstration, so here’s an imaginary scenario that shows some of the basic usage. A block of text Note: The following is just a quick walkthrough showing how regular expressions can be used in real life, how it doesn't require learning totally new software or buying a new computer. It's not meant to be an explanation of the syntax; that's what the book is for. The workday was just about to end until you checked your email and saw that damn priority email from your boss. You open it up and it’s an attached text file. “Someone just sent us data that we need to look for, can you get this done by tomorrow?” You open up the file and it’s a massive glob of text. Maybe it was data at some point, on someone else’s computer, before it got exported into this mess: JAMES FALLON 40,000 ALEX O’HARE 90 SAL THOMAS 300.25 ALVIN BARRY 121.99 KALEY SIMS 67,000 AL SMITH 52 SHAWN RALSON 300 MALLORY JOHNSON 800 ALBERT SACKS 300,500.99 SALLY POST 400 FALSTAFF YORK 0 ALLISON STEVENS 97,000 THOMAS BALLARD 10,000 CAL SIMMONS 56,800 RALPH MOUSE 24.54 ALEXANDER SINGER 9,300 ALFRED STAMPER 1,780.00 DAN SALSBURY 2,000.99 It looks like a list of first names, last names, and amounts of money, except that someone forgot to hit “Return” at the right places. (This is just an excerpt. Imagine it being a thousand times longer.) Well, at least you don’t have to do anything with it, your boss tells you. You just need to look for entries related to a particular client. “The client’s name begins with ‘Al,’” your boss says. “Al…Al, what?” “I’m not sure. Could be ‘Alfred’, or ‘Albert’. Or maybe even ‘Allie’.” You sigh. Well, you know how to find words using Microsoft Word. You hit Ctrl-F and type AL into Microsoft Word: Well, that was not much help. A search for AL ends up highlighting AL whether it appears in ALVIN , SALLY , or SALSBURY , nevermind if it appears in the first or last name. A better Find At this point, there’s not much you can do but accept that you’ll be clicking Find Next way more times than you want to. If only there was a way to describe the kind of thing you’re looking for, rather than just the literal term. This is what regular expressions allow us to do. Keep reading to see how regex can be used in this scenario. You can also follow along interactively by going to Rubular.com and pasting in the regex and text. ( Note: it’s worth noting that Microsoft Word allows some regex usage . However, they use a syntax extremely different from what you’ll find in virtually all the other programs and contexts that use regex. If you want to learn two different flavors of regex, knock yourself out. But for now, it’s just enough to understand the concepts and potential ) AL, just AL, please We’re looking for an AL but we don’t need SAL . So, what we’re actually looking for is: AL , when it’s the beginning of a name. In Microsoft Word, you can be a little clever and change your search to look for a space character and then the word AL . That would limit your matches to where AL occurs at the beginning of a word. In other words, we don’t literally need just AL , but a pattern containing AL at the beginning of a word . In regular expression syntax, we can express that a little more specifically: this is how you find AL when it occurs at the beginning of the word (which is usually, but not always, after a space character): \bAL And here’s what that highlights: JAMES FALLON 40,000 AL EX O’HARE 90 SAL THOMAS 300.25 AL VIN BARRY 121.99 KALEY SIMS 67,000 AL SMITH 52 SHAWN RALSON 300 MALLORY JOHNSON 800 AL BERT SACKS 300,500.99 SALLY POST 400 FALSTAFF YORK 0 AL LISON STEVENS 97,000 THOMAS BALLARD 10,000 CAL SIMMONS 56,800 RALPH MOUSE 24.54 AL EXANDER SINGER 9,300 AL FRED STAMPER 1,780.00 DAN SALSBURY 2,000.99 Nice! Adding an extra two characters – \b – saves you a bit of time already. The important thing here is that using regex is no different than doing any other kind of Find operation. This is how easy it using Rubular.com : And this is how easy it is in a regex-supporting text editor : (The Bastards Book of Regular Expressions has a chapter on choosing from the several free and powerful text editors out there and how to use them.) First name “AL”-something, last name S…something Time to narrow the search even more. You ask your boss: “Do you have anything more specific than ‘Al’? Like a last name?” She thinks for a few seconds. “Yes, I remember now. The client’s last name begins with the letter ‘S’.” Now, without regexes, that bit of information is actually almost no help at all. You don’t know if the client’s name is “Alvin”, “Allison”, or just plain “Al”. With the standard Find function, there’s no way to specify this particular pattern: The letters AL Followed by a bunch of other letters (or not, in the case of someone named just “Al”) A space character A word that begins with S But here’s how we describe that pattern with regexes: \bAL\w* S And this is what it highlights: JAMES FALLON 40,000 ALEX O’HARE 90 SAL THOMAS 300.25 ALVIN BARRY 121.99 KALEY SIMS 67,000 AL SMITH 52 SHAWN RALSON 300 MALLORY JOHNSON 800 ALBERT SACKS 300,500.99 SALLY POST 400 FALSTAFF YORK 0 ALLISON STEVENS 97,000 THOMAS BALLARD 10,000 CAL SIMMONS 56,800 RALPH MOUSE 24.54 ALEXANDER SINGER 9,300 ALFRED STAMPER 1,780.00 DAN SALSBURY 2,000.99 That pattern reduced the number of finds by a lot . Middle initials Let’s make this scenario harder. Let’s say the text includes maybe a middle initial – or maybe not – for each first-name, last-name pair. Now our previous regex won’t work. It won’t match any of the AL S instances we found before, and it will make unintended matches, such as ALEX S O'HARE : JAMES C. FALLON 40,000 ALEX S . O’HARE 90 SAL A. THOMAS 300.25 ALVIN BARRY 121.99 KALEY SIMS 67,000 AL J. SMITH 52 SHAWN K. RALSON 300 MALLORY A. JOHNSON 800 ALBERT O. SACKS 300,500.99 SALLY R. POST 400 FALSTAFF E. YORK 0 ALLISON O. STEVENS 97,000 THOMAS BALLARD 10,000 CAL R. SIMMONS 56,800 RALPH S. MOUSE 24.54 ALEXANDER SINGER 9,300 ALFRED Z. STAMPER 1,780.00 DAN R. SALSBURY 2,000.99 But we just have to make a more specific regex. In plain English, the pattern we want is: A word that begins with “AL”, followed by a space, followed by an optional middle initial with a period, and then another word that begins with an S : \bAL\w*( \w\.)? S\w+ And that works: JAMES C. FALLON 40,000 ALEX S. O’HARE 90 SAL A. THOMAS 300.25 A...