{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "# Advanced Regular Expressions\n",
    "\n",
    "- https://github.com/rexdwyer/Splitsville/blob/master/Splitsville.ipynb\n",
    "- http://www.python-course.eu/python3_re.php\n",
    "- http://www.python-course.eu/python3_re_advanced.php\n",
    "- https://github.com/tartley/python-regex-cheatsheet/blob/master/cheatsheet.rst"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Flags for re.compile(), etc. Combine with '|':\n",
    "\n",
    "```\n",
    "re.I == re.IGNORECASE   Ignore case\n",
    "re.L == re.LOCALE       Make \\w, \\b, and \\s locale dependent\n",
    "re.M == re.MULTILINE    Multiline\n",
    "re.S == re.DOTALL       Dot matches all (including newline)\n",
    "re.U == re.UNICODE      Make \\w, \\b, \\d, and \\s unicode dependent\n",
    "re.X == re.VERBOSE      Verbose (unescaped whitespace in pattern\n",
    "                        is ignored, and '#' marks comment lines)\n",
    "```\n",
    "\n",
    "Module level functions:\n",
    "\n",
    "```\n",
    "compile(pattern[, flags]) -> RegexObject\n",
    "match(pattern, string[, flags]) -> MatchObject\n",
    "search(pattern, string[, flags]) -> MatchObject\n",
    "findall(pattern, string[, flags]) -> list of strings\n",
    "finditer(pattern, string[, flags]) -> iter of MatchObjects\n",
    "split(pattern, string[, maxsplit, flags]) -> list of strings\n",
    "sub(pattern, repl, string[, count, flags]) -> string\n",
    "subn(pattern, repl, string[, count, flags]) -> (string, int)\n",
    "escape(string) -> string\n",
    "purge() # the re cache\n",
    "```\n",
    "\n",
    "RegexObjects (returned from compile()):\n",
    "\n",
    "```\n",
    ".match(string[, pos, endpos]) -> MatchObject\n",
    ".search(string[, pos, endpos]) -> MatchObject\n",
    ".findall(string[, pos, endpos]) -> list of strings\n",
    ".finditer(string[, pos, endpos]) -> iter of MatchObjects\n",
    ".split(string[, maxsplit]) -> list of strings\n",
    ".sub(repl, string[, count]) -> string\n",
    ".subn(repl, string[, count]) -> (string, int)\n",
    ".flags      # int, Passed to compile()\n",
    ".groups     # int, Number of capturing groups\n",
    ".groupindex # {}, Maps group names to ints\n",
    ".pattern    # string, Passed to compile()\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# https://pypi.python.org/pypi/regex\n",
    "# http://www.rexegg.com/regex-python.html\n",
    "# https://github.com/rexdwyer/Splitsville/blob/master/Splitsville.ipynb\n",
    "\n",
    "import regex as re\n",
    "t = 'The quick brown fox born on 1/23/2013 jumped over the lazy dog born on 10/6/10.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## match and search\n",
    "\n",
    "- `match`: checks for a match of re_str merely at the **beginning** of the string.\n",
    "- `search`: checks a string s for an occurrence of a **substring**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<regex.Match object; span=(0, 3), match='The'>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.match(r'The', t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Normally if we were going to reuse a pattern, we'd compile it and use the match method of the resulting pattern like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<regex.Match object; span=(0, 3), match='The'>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p = re.compile(r'The')\n",
    "p.match(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(re.match(r'the',t))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**search** looks at the whole string. It finds the **the** before **lazy dog**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<regex.Match object; span=(50, 53), match='the'>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'the',t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ignore case**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<regex.Match object; span=(0, 3), match='The'>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'the', t, re.IGNORECASE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<regex.Match object; span=(0, 3), match='The'>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "re.search(r'(?i)the', t) # or use inline modifiers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Closer Look at the Match Objects"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "232454, Date: February 12, 2011\n",
      "('232454', 'February 12, 2011')\n",
      "(17, 48)\n"
     ]
    }
   ],
   "source": [
    "m = re.search(r'([0-9]+).*: (.*)', \"Customer number: 232454, Date: February 12, 2011\")\n",
    "\n",
    "print(m.group()) # whole string\n",
    "print(m.groups()) # list of groups\n",
    "print(m.span())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  findall and finditer \n",
    "\n",
    "---\n",
    "\n",
    "`findall()` lists every match to the pattern, but doesn't give the position. `I` is short for `IGNORECASE`\n",
    "."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "t=\"A fat cat doesn't eat oat but a rat eats bats.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "m = re.findall(\"[force]at\", t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['fat', 'cat', 'eat', 'oat', 'rat', 'eat']\n"
     ]
    }
   ],
   "source": [
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['232454', '12', '2011']\n"
     ]
    }
   ],
   "source": [
    "m = re.findall(r'([0-9]+)', \"Customer number: 232454, Date: February 12, 2011\")\n",
    "print(m)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2231aa\n",
      "('2231', 'aa')\n",
      "4652aa\n",
      "('4652', 'aa')\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "regex = r\"([0-9]+)(aa)\"\n",
    "test_str = \"2231aazxczxc21zxc4652aa\"\n",
    "\n",
    "matches = re.finditer(regex, test_str)\n",
    "\n",
    "# print(matches.matches(0))\n",
    "\n",
    "# for matchNum, match in enumerate(matches):\n",
    "#     matchNum = matchNum + 1\n",
    "    \n",
    "#     print (\"Match {matchNum} was found at {start}-{end}: {match}\".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))\n",
    "    \n",
    "#     for groupNum in range(0, len(match.groups())):\n",
    "#         groupNum = groupNum + 1\n",
    "        \n",
    "#         print (\"Group {groupNum} found at {start}-{end}: {group}\".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))\n",
    "        \n",
    "        \n",
    "        \n",
    "for m in matches:\n",
    "    print(m.group())\n",
    "    print(m.groups())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compiling Regular Expressions\n",
    "\n",
    "---\n",
    "\n",
    "`re.compile(pattern[, flags])`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Search and Replace with sub\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "no I said no I will no.\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "s = \"yes I said yes I will Yes.\"\n",
    "res = re.sub(\"[yY]es\",\"no\", s)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}