Introduction to Regular Expressions (RegEx) in AutoHotkey

Wikipedia describes a Regular Expression as "a sequence of characters that forms a search pattern, mainly for use in pattern matching with strings, or string matching, i.e. 'find and replace'-like operations." I would describe RegEx as a data mining machine. RegEx is like a train rolling down a track of computer characters looking for patterns which match a specific set of given parameters. If it finds characters which match the pattern set, it grabs them and puts them on board the train.

As the RegEx train runs down the line, it continues picking up characters—as long as they fit the written instruction set. Some groups of characters may be saved for later reuse (backreferences). At times RegEx may look back at previous characters for validation or forward to coming data for confirmation (backward and forward assertions—see Chapter Twelve of A Beginner's Guide to Using Regular Expressions in AutoHotkey). While a particular RegEx may be forgiving in what it will accept on board, if the pattern does not completely match the given set of criteria, the entire group (including all previously collected characters) is kicked off the train and RegEx continues rolling along looking for the another possible set of matching characters. This continues until it either hits the ends of the line or finds a complete solution to its data schedule. Then RegEx stops. The RegEx data mining machine can be started up again by placing it in a Loop which restarts the same search from a point just beyond its current solution.

This data mining train is the image I visualize when working with a RegEx. The key to understanding RegEx is knowing what the conductor is trying to do when it interprets the special symbols in a RegEx set of instructions to bring the right character passengers on board. It took me a while to comprehend that the primary purpose of the AutoHotkey RegEx functions are data extractions, RegExMatch(), or data correction, RegExReplace() as discussed in Chapter Three of A Beginner's Guide to Using Regular Expressions in AutoHotkey.

Practical Uses for RegEx in AutoHotkey

Maybe the most important question is, "If Regular Expressions can be so confusing, why bother?" Often when doing simple text searches or replacements it's quicker and easier to use functions built into a scripting language. RegEx may be adding needless complication. However, a RegEx might do with one expression what takes several lines of code when using those other functions. It may take slightly longer to complete (a few more microseconds), but the added flexibility could make the seemingly impossible a reality. RegEx has more power and flexibility than a standard search and/or replace.

For example, IP addresses are many and varied—although they all conform to the same pattern. Each IP consists of four numbers (one to three digits long and between zero and 255) separated by a dot. With the proper RegEx, the engine can search through a document pulling out only the IP addresses. Then those extracted addresses can be used to find where the IP is located. AutoHotkey with RegEx creates a Web IP lookup app finding extracted IP address locations throughout the world. (This example and many more of the following examples using RegEx in AutoHotkey are discussed in the e-book A Beginner's Guide to Using Regular Expressions in AutoHotkey.)

Another use for a RegEx may be to find duplicate words in a document. This can be done with other functions, but it would take a few lines of code with conditionals (If-Then), whereas only one RegEx line is needed. How about swapping the first and last words in selected text?

Maybe you want to strip all of the HTML code out of a Web page leaving only the text? Or, possibly you need to extract a list of all of the Web links found in a Web page? Regular Expressions are the best way to ensure that a properly formatted, valid e-mail address is entered into a data field.

Pulling the numbers out of alphanumeric data is relatively simple with RegEx. Or, maybe a key symbol (escape character) needs to be inserted in front of (or behind) each in a group of special characters.

If it's a pattern you need to locate (and possibly manipulate) in your haystack of data, then RegEx may be your best bet for finding that needle. This may be all the incentive you need to explore the mysteries of Regular Expressions.

The Mechanics of RegEx in AutoHotkey

Critical to using a RegEx is understanding how it works. RegEx is a system for finding matches within strings (text) which may be file names, variables, or the contents of a file. (Such a search string is called the "Haystack" in the online documentation for the AutoHotkey Regex functions, while the search expression is called the "NeedleRegEx"—as in "needle in a haystack.").

Knowing what a RegEx will actually do depends on how well you understand what it is trying to do. A RegEx starts at the beginning of a string and looks at each character one by one until if finds a match for the entire expression. If it finds a match (NeedleRegEx) it stops looking, otherwise it continues until it reaches the end of the input string (Haystack). In AutoHotkey the function used for matching a RegEx is RegExMatch() which returns the numeric location of the first character of first occurrence of a match. (A numeric location is found by counting the number of characters from the beginning of the Haystack to the first character in the NeedleRegEx.)

For example, in its simplest form the NeedleRegEx might be a lowercase a (or any other letter, number, or character). The RegEx engine will search the Haystack looking for an a. If found, it stops and returns the location of the letter:

FoundPos := RegExMatch(Haystack, "a")

To make the RegEx slightly more complicated we add another character to our RegEx:

FoundPos := RegExMatch(Haystack, "ab")

What? That FoundPos coincides with the "ab" in "absent", not the first "ab" in "Abby." This brings us to an important concept—RegEx is case sensitive. If you want to find a capital letter, it better be capitalized in the RegEx. The word "Abby" in the haystack is skipped as a match because the "A" is uppercase while the needle is "a" lowercase.

Note: There is an option to make the RegEx case insensitive, but that will be left for another chapter. That's the problem with RegEx. There are so many possibilities and options that it's easy to get confused.

As RegEx moves through the Haystack it stops at each letter "a", then checks for a letter "b" immediately following it, but none are found until reaching the word "absent" starting at position 22. Having found a complete match, RegExMatch() stops.

This is the essence of how RegEx works. If more characters are added to the expression'sNeedleRegEx, then more is required to find a match. However, in the problem of validating numbers (for example in the Calorie Count app originally discussed in the book AutoHotkey Applications) the digits can be any numbers, but no letters.

Using Ranges in RegEx Matches

The simple way to match any number in the RegEx is to give a range of options. This is done by enclosing all the optional characters within square brackets […]. For example, placing all the vowels within square brackets makes each one a possible match:

FoundPos := RegExMatch(Haystack, "c[aeiou]t")

In our situation we want to use the numeric digits [0123456789]. (The order of the digits inside of the square brackets doesn't matter.) If we wanted to match two digits in a row then [0123456789][0123456789] would do the job. The problem is that we don't know how many digits in a row we need to match. It could be one, two, three or more—at least theoretically. At those times when you don't know how many characters will occur in a row, rather than repeating the range for each matching character, adding the plus + sign after the range (or character) will do the job:

FoundPos := RegExMatch(Haystack, "[0123456789]+")

Tip: Ranges of numbers or letters can be shortened by using a hyphen. For example, [0-9] is the same as [0123456789]. [A-Z] is the same as all capital letters while [a-z] is all lowercase letters. All letters and digits can be represented by [a-zA-Z0-9]. To shorten the expression for the numeric digit range even more use \d in place of [0-9]. Our shortened function becomes:

FoundPos := RegExMatch(Haystack, "\d+")

A Beginner's Guide to Using Regular Expressions in AutoHotkey: Exploring the Mysteries of RegEx

For More Information

If you're interested in testing AutoHotkey to see if it might be right for you, then go to "Installing AutoHotkey and Writing Your First Script." This page shows you how to get up and running with AutoHotkey, plus it offers links to other articles on how to use AutoHotkey.

If you want more information in either the Amazon Kindle format, EPUB format for use on the iPad and other types of tablet computers (or on your PC), or PDF for printing on notebook size paper, then check out the following e-books by Jack Dunning:

See how to get this e-book FREE, AutoHotkey Tricks You Ought to Do with Windows!

Also available at Amazon.com for the Kindle and Kindle software.

To get more detailed information about AutoHotkey and see a List of AutoHotkey commands visit the AutoHotkey Web site.

Some More AutoHotkey Uses