Fred Brack   Fred Brack  
Raleigh, NC
Picture of the Cape Hatteras lighthouse

How to Build a Web Page Programmatically
by Fred Brack

If you are a webmaster who periodically creates or receives a file of data that has to be integrated into a webpage, this is one person's experience with how to do that.  Here are the assumptions:

  1. You know how to write a computer program in some language (any language).
  2. You have a web page which presents data in tabular or list format (in other words, consistent from line to line).
  3. You receive or generate yourself a file of data whose contents determine completely what will be listed on your page, other than introductory text.

In my case, I receive files of data each week in the form of an Excel spreadsheet where each row represents a single video title (for example, a movie) along with its characteristics (like genre and rating).  From this data, I build a webpage programmatically alphabetizing all the titles and adding index letters (A, B, C, etc.).  I get everything I need in the file -- I just need to extract and manipulate the data, turning it into HTML.

An overview of how this is done follows:

  1. Build your web page with a sample line of data.
  2. Include the complete header, navigation, footer, and any introductory or closing text you want.
  3. Insert "control" statements at any place that you want your program to insert data.  For example, the update date and number of titles listed, and a second location where you actually want the title listing to begin.
  4. If the data comes to you in a spreadsheet, SAVE the data in ".csv" format - Comma Separated Values.  Excel will save the data in a file where the data in each column of a row is written out followed by a comma.  If the data itself contains a comma, then the entire field will be in quotes.  Your program will extract the data from this file (since you can't process the Excel spreadsheet directly).
  5. Write a program to do any preprocessing of your CSV (or other) file data (for instance, validate it, sort it, and count valid entries); load up a template consisting of that sample page your wrote up to the point of your control statement; insert whatever is required at that pont; then continue to load up the reset of the template.
  6. The end result is a complete HTML file with your data inserted.

Here is an example of a programmatically generated web page which started out as an Excel worksheet:  iTunes Audio Described Movie Titles.  Most of the other title listings on that website are generated similarly.


Your Program:

As a former IBMer, I choose to write in a programming language you are unlikely to use:  REXX (available free as Regina Rexx).  A typical program to load data and format a web page is about 1100 lines.  In my case, a lot of that has to do with processing the genre of the movies, handling exceptions such as correcting titles, and determing changes since last time.  Your program could definitely be shorter!  The flow is something like this:

The resultant HTML file is ready for republishing to my website with a new date noted, updated title count, and a section at top for "Recent Additions."

I hope this brief summary has given you ideas on how you can programmatically create your own web pages.  Drop me an email if this is helpful or you have questions or suggestions for improvement.