Making Friends with Regular Expressions

Overview, Terminology, and Grammar

Regular Expressions are Simple and Beautiful

• Regular expressions are simple, not complex.

  • The "regular" in regular expressions means "well-behaved" or "easy to handle."
  • Example: Normal distribution in the field of statistics.

• However, it is often said in the community that "regular expressions are difficult" or "I don't understand them at all."

  • This article is here to tell you that's not the case.
  • I want to convey that regular expressions are a great abstraction for handling strings programmatically.

Minimum Terminology Check

• Organizing terms
  • Regular Expression Matching
    • Checking whether a given string matches a regular expression.
    • Think of it like an "if" statement in programming.
  • Meta characters
    • Characters that have a special meaning in regular expressions.
    • You can just look them up as needed.
  • Literals
    • Characters that do not have a special meaning in regular expressions.
  • Quantifiers
    • Operators that determine the quantity of other regular expressions.
    • You can just look them up as needed.

Regular Expression Grammar (Typical Examples)

• Concatenation

  • “ab”
  • The string "ab"

• Alternation (OR)

  • “A|B”
  • The string "A" or "B"
  • Same as the "|" in an "if" statement.

• Repetition

  • Quantifiers like “[AB]*”
  • Corresponds to a "for" loop in programming.
  • Like "characters"*n in Python 3
  • Like the repeat("characters", n) function in PHP

• Grouping

  • (ab)
  • Characters can be grouped together, such as (ab){2}|(ac){3}

• Character class

  • By putting a range of characters in [], you can represent any single character belonging to that range.

• Key Points

  • You can generally read and write regular expressions by keeping the above in mind and looking up the rest as you go.
    • Combine small components to create complex ones.
    • Example of a complex pattern
    • Can you read this regular expression for extracting HTML tags?

Let's Try Writing a Regular Expression

• Pick the targets you want to match

  スターバックスコーヒー
  ドトールコーヒー
  タリーズコーヒー
  サンマルクカフェ
  プロント
  コメダ珈琲店
  シャノアール
  カフェ・ド・クリエ
  シアトルズベストコーヒー
  

• Think about the common parts

  • The strings consist of "Japanese Katakana", "珈琲" (Coffee), and "・"
  • Japanese Katakana is "U+30A0-U+30FF", which is represented as "[\u30A0-\u30FF]" in regular expressions
    • [range] represents any single character within the range
    • (Let's set aside the discussion about extended Katakana for now)
  • Since "珈琲" is never reversed to "琲珈", we treat it as a single unit: "(珈琲)"
  • Since the word "コーヒー" (Coffee) often appears at the end, we use "(コーヒー)?$"

• Clarify the targets

  • Patterns of Katakana + コーヒー
  • Patterns containing 珈琲 or カフェ (Cafe)
    • I assume "コーヒー" doesn't appear at the beginning, so I'll omit that case
  • Other patterns like Chat Noir (シャノアール) or Pronto (プロント)

• Write it

  • Once you get used to it, you can write it in 5 minutes
  • In my personal opinion, it's important to write it yourself

  |||(シャノアール)|(プロント)
  [\u30A0-\u30FF]+(コーヒー)$|||(シャノアール)|(プロント)
  [\u30A0-\u30FF]+(コーヒー)$|.+(珈琲).*|.*(カフェ).*|(シャノアール)|(プロント) // Since I don't see "珈琲〇〇" very often...
  

• Test

  • Check using https://www-creators.com/tool/regex-checker

  • Multiple lines or 'm' flag

  • Input

  スターバックスコーヒー
  ドトールコーヒー
  タリーズコーヒー
  サンマルクカフェ
  プロント
  コメダ珈琲店
  シャノアール
  カフェ・ド・クリエ
  シアトルズベストコーヒー
  コーヒー
  コーヒーhogehoge
  コーラ
  珈琲
  珈琲hogehoge
  

  • Output

  1st match
  スターバックスコーヒー
  2nd match
  ドトールコーヒー
  3rd match
  タリーズコーヒー
  4th match
  サンマルクカフェ
  5th match
  プロント
  6th match
  コメダ珈琲店
  7th match
  シャノアール
  8th match
  カフェ・ド・クリエ
  9th match
  シアトルズベストコーヒー
  

Why Do People Find Regular Expressions Difficult?

• Not leaving information about the regular expression in the source code

  • Unless you leave information about the string represented by the regular expression in function names or comments, it becomes a hassle to re-read it.
  • Just as you leave comments when writing PHP or JavaScript, you should leave them for regular expressions too.

• Getting stuck on parts that are not the regular expression itself

  • Not knowing the concept of character encoding.
  • Regular expressions and character codes are inseparable, so it's necessary to learn them separately.

• Rugged appearance

  • Simple $\ne$ Easy.
  • Because unnecessary elements are stripped away, the appearance doesn't look like high-level languages such as PHP or JavaScript.

• Getting stuck by copy-pasting regular expressions without reading them

  • Read a book.
  • To reiterate, simple $\ne$ easy.

• Not writing regular expression tests effectively

  • Not testing the regular expression in isolation.
    • Testing it along with other processes.
  • Not writing negative test cases.
    • If you only write positive cases, you'll run into trouble later with strings you didn't account for.
    • Writing negative cases allows you to define the target strings more strictly.

• Writing logic into the regular expression

Exercise

• I've put together a practice problem for you.

  • Try solving it as a review of what we've discussed so far.

• Write a regular expression that matches the following strings:

  • However, please use the expression "(SQL)".

    MySQL
    PostgreSQL
    BigQuery
    SQLite
    

  • Also, ensure that it does not match the following strings:

    SQ
    QL
    SQL
    SQLエスキューエル
    エスキューエルSQL
    1SQL
    SQL1
    

  • Finally, describe any strings that might cause problems with the regular expression you created.

References

• Ryoma Shinya, Yusuke Suzuki, Ken Takada. "Introduction to Regular Expression Technology: Latest Engine Implementation and Theoretical Background (WEB+DB PRESS plus)". 2015. Gijutsu-Hyouron-sha.