{"id":995,"date":"2011-09-15T18:04:26","date_gmt":"2011-09-16T01:04:26","guid":{"rendered":"http:\/\/domemtech.com\/?p=995"},"modified":"2011-09-16T03:50:58","modified_gmt":"2011-09-16T10:50:58","slug":"antlr-php-target","status":"publish","type":"post","link":"http:\/\/165.227.223.229\/index.php\/2011\/09\/15\/antlr-php-target\/","title":{"rendered":"Antlr PHP target"},"content":{"rendered":"

My wife recently started a WordPress blog<\/a> on regulatory affairs of the drug and medical device industries.  When she started it, she decide to add posts from time to time of newly approved drugs and generics, based on information from the FDA website.  So, after a few times proof-reading her drafts, I realized that this was a good example of an aggregator<\/a> problem, something that I've been wanting to do for my own blog.  The aggregator would read the Drugs@FDA<\/a> and MedlinePlus<\/a> websites, extract approved pharmaceuticals and generics, then create a draft post for each new drug.  Sounds like an easy project, right?<\/p>\n

<\/p>\n

As with any software, the devil was in the details.  After studying what was involved in creating posts using an aggregator, I realized that the solution required three basic steps: fetching web pages; extracting the information from those pages; and making a post in WordPress.  I was able to figure out how to do steps (1) and (3) pretty quickly, but step (2) was a little harder.  How do I extract information from a web page?  Since all the web pages are in HTML, what I needed was a parser for HTML.  The parser would create a parse tree<\/a> (or abstract syntax tree<\/a>) representing the document.  I could then write code to walk the tree, and extract the information I was looking for.  In this case, I wanted to extract particular columns out of a table.  <\/p>\n

Parsing<\/a> is not easy for any language, let alone HTML.  While there are a few tools available to parse HTML (e.g., http:\/\/simplehtmldom.sourceforge.net\/<\/a> , http:\/\/tidy.sourceforge.net\/<\/a> ), I didn't want to be smart<\/a>: I wanted to write my own because I was a compiler engineer long ago!<\/p>\n

So, I decided to use Antlr<\/a>, a parser generator.  Writing a grammar for HTML wouldn't be easy because HTML is not a context-free grammar<\/a>.  It also allows unmatched elements. In addition, the target language for the parser had to be PHP because it had to work with WordPress<\/a>.  Unfortunately, the source code for the Antlr PHP runtime was disorganized<\/a>.  It was not up to date, having been last modified over a year ago; and, several developers created copies of the code, each modified in different ways.  All of these problems created a lot of confusion.<\/p>\n

As it turned out, I was luckily able to update the Antlr PHP runtime fairly quickly.  That code is Antlr 3.4\/PHP 5.3 dependent, and is available here<\/a>. Just unpack the zip file and use the enclosed antlr.jar file to generate the parser in PHP, e.g., 'java -jar antlr.jar TableLexer.g'.  I was able to use Antlr to find nested HTML tables, and HTML rows (TableLexer.g<\/a>).  But, as with all Antlr grammars, they  are target dependent because the grammar contains semantic checks written in PHP.<\/p>\n

Another example of Antlr targeted to the PHP runtime is an expression grammar<\/a>.  This web page<\/a> demonstrates how Antlr would work with this grammar rewritten for PHP.<\/p>\n

<\/table'><\/p>\n","protected":false},"excerpt":{"rendered":"

My wife recently started a WordPress blog on regulatory affairs of the drug and medical device industries.  When she started it, she decide to add posts from time to time of newly approved drugs and generics, based on information from the FDA website.  So, after a few times proof-reading her drafts, I realized that this was a good … <\/p>\n

Continue reading “Antlr PHP target”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[],"tags":[],"_links":{"self":[{"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/posts\/995"}],"collection":[{"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/comments?post=995"}],"version-history":[{"count":0,"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/posts\/995\/revisions"}],"wp:attachment":[{"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/media?parent=995"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/categories?post=995"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/165.227.223.229\/index.php\/wp-json\/wp\/v2\/tags?post=995"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}