Anne

I was so pleased that with the transfer to WordPress, everything read much better than on the Manilla server. When I next looked at the blog, the same text was completely compressed without any return line! Go figure! (the wordpress auto paragraph formatting had been disabled, admin put it back for me Ouf!)

Here is today’s question, how do Rails or Nitro process URLs so that a URL request can be processed without an actual file of that name on the server?

Typing a URL on the top of the page initiates a communication with the computer hosting the website: The browser opens a connection with that computer and then issues a GET command to receive a file.

telnet www.apacheweek.com 80
GET / HTTP/1.0

The receiving computer recognises the “GET” command, sends info about itself, figures out what file is wanted (in the case of “/”, it looks for a root index page), and if it finds it, sends it to the requesting computer.

Frameworks like Rails or Nitro intercept requests within a certain directory for pre-processing. The framework code has to be the one responding to the GET command issued over the http connection. How is it done?

Web applications are machine independant, so a framework probably works with the browser rather than at the machine level. How to handle URL’s must be loaded in the browser before any request comes in.

In Apache some AP_GET_ browser_fx is called upon by the browser when it detects the GET command. The browser probably stores the information parsed and initiates a default action unless… there is an alternative default setup, in which case the request information could be passed on in environment variables, or simply as the original URL string request.
http://hoohoo.ncsa.uiuc.edu/cgi/env.html

One avenue of inquiry is to look into Apache’s documentation.
Since Rails works with CGI or fcgi, CGI would be another avenue of inquiry.
I prefer to look at how it is done, and see what I can learn from it.

http://wiki.rubyonrails.com/rails/pages/UnderstandingHowRequestsAreRouted
In rails, when you deploy your application you set up Apache through its httpd.conf file. That’s hard wired and requires restarting Apache. You set up a virtual host (see Rails Agile web book on virtual host, chapt 22, page 455) and you tell it that a file in the directory corresponding to that virtual host will overide all. That file is invisible, it is called .htaccess
Any file which does not exist is redirected by this .htaccess file to dispatch.cgi by the code
RewriteRule ^(.*)$ dispatch.cgi [QSA,L]

This use of .htaccess reminds me of what I did to get erb to work
http://dekstop.de/weblog/2006/01/rhtml_on_osx_with_apache_and_erb/
“Your server is configured so that CGI scripts placed in /Library/WebServer/CGI-Executables/ map to http://127.0.0.1/cgi-bin/ …
Create the following .htaccess file somewhere below your server’s document root at /Library/WebServer/Documents/ to enable the .rhtml handler for a specific directory…”

I also found an interesting website which talks about how search engines deal with dynamic websites.
http://www.hypergurl.com/articles/dynamic.html
It explains how to use mod-rewrite to be more search engine friendly
RewriteRule ^productid([^.]+).*$ yourscript.php?id=$1 [T=application/x-httpd-php]
this code is written in htaccess file.

So over and over I find the same technical solution in apache, a virtual host with a hidden htaccess file. php, Cgi, or a framework come down the line to route the request and create dynamic web links to present.

What about other webservers? What can I learn from them?

I was able to redirect URLs with webrick using catapult. There the code is short and easy to identify.
http://www.jamesbritt.com/code/catapult/

class RequestHandler (less than sign) HTTPServlet::AbstractServlet
def process_request( request, response )
…. request.path_info.split

the request.path_info contains the URL, the process_request method can create an instance of the class requested and run the associated code to create and return a response.body and a response content type.

I was able to redirect URL requests with Nitro very easily. Unlike Rails, I did not have to generate miriads of files and folders. All I had to do was to define my class methods in a file main.rb in a folder called controller.
The server code maps the port to the controller folder.
setting :map, :default => { ‘/’ => Controller }, :doc => ‘The server map’

when you run nitro (server.rb), server/runner.rb determines which webserver to use, in the case of webrick, adapter/webrick

@webrick = WEBrick::HTTPServer.new(webrick_options)
class WebrickAdapter (less than sign) WEBrick::HTTPServlet::AbstractServlet
def handle(req, res)
…

and in rails there is a webrick server
class DispatchServlet (less than sign) WEBrick::HTTPServlet::AbstractServlet
def service(req, res) #:nodoc:

With webrick, I see basic code for which all default behaviours must be defined, but with the advantage of being able to do a basic redirect very easily, as part of one’s code as shown by the streamlined catapult.

What did I learn?
Learning how to do a redirect is not basic task. At the root of the internet behavior is a set of rules on the kind of data can be placed on the internet. HTTP data, XML data etc On top of that, any numbers of software can be written which access the internet in read or write mode. And on top of that code or frameworks are written to automate specific kinds of processes.

A more complex server requires code in its configuration file, or task specific instruction file, A lighter server like webrick expects code in a specified section of code. Frameworks require a file of the proper name in a specified directory… every where, conventions, conventions, conventions. Somehow, people out there assimilate these conventions, manipulate them, create new behaviors.

What I lalso earned from this process is the importance of finding a small program like catapult which has the desired behavior, but is so small I can see how it is done. For learning at least, small programs are more helpful. Once I know what to look for it is easier to look for similar behavior features in other more complex code.

A new webserver. I only lost half a blog page because it contained a lessthan sign which apparently tells the blog it is the end of the text! I have no idea how to find help and learn to escape characters, so for now I put text instead of the symbol and I will move on. Formating is better, with lines of code following each other line by line, instead of bunched in an unreadable paragraph.

These last couple weeks, I have made progress with Ruby and the web. It is a sweet feeling to finally get a “hello world” example to work. I spend so much of my time trying to get installs to work, that I wonder if I will move on to actually writing code! Maybe I am getting addicted to this, “install and how do you get to this to work” pattern of tension and delight in the release from tension.

“The buddha evoked the unrest, instability and uncertainty that color our lives …desire is a natural response to the reality of suffering. We feel incomplete and desire completeness; we feel unrest and desire ease; we feel insecurity and desire comfort; we feel alone and desire connection.” Mark Epstein. So I am endlessly flying on the wheel of suffering. But at least in this case, thanks to the generosity of folks on the web, there are short moments of release from suffering.

For now, I am following some of the beginner’s assignments posted by MIT professor philip Greenspun on this older course website
http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-171Fall2003/CourseHome/
Most of the implementation is left for me to discover, but in a sense, that makes it easier to learn: I don’t have to skip over perl or java code to get to the main ideas. And hopefully the text will guide me to some web application mastery. For now, I have written my first CGI code entering data in a form and sending it to a ruby code to process and generate a web page with the answer. Cool!

I have also learned one way of setting up eruby and erb on tiger. First I set up erb with the help from this weblog
http://dekstop.de/weblog/2006/01/rhtml_on_osx_with_apache_and_erb/index.html
and then I did the same with eruby, except I copied eruby to eruby.cgi.

A key part of learning, is finding an object of desire. How many come to the ruby site and say, I have read the book, now what, what problem should I try to solve? And on the other hand, how many come to 3D like I do with an overwhelming desire, I want to represent my cousin’s head, a boy jumping in the air, etc, dreams of perfection so far beyond anything the most skilled person can attain. With such richness of perception from daily life around us, how can we look at a dreary cube with the least bit of lust? I sat under a tree in bloom yesterday. Each branch was heavy with hundreds of tiny perfect white flowers sparkling in the sun by the river.

My life is stop and go. My latest: reading all about household employee’s tax. Took me two days, but I think I got it. The forms to send, the amount owed… First there is social security and medicare, kicks in at 1400$/year, then federal and state tax, kicks in at 1000$ per quarter, then there is worker’s comp, kicks in at 16 hours/week. As I am getting more help for my mother, there is more and more to do… beyond paying the person.

I got the agile rails book, so I finaly had a chance to look at it. Luckily, I had tried to install a wiki before, so I had php and mysql installed. All I had to do was reinstall mysql because I could not find the passwords. Then I found a Rails installation incantation which worked for my computer, and I was able to follow the examples — more or less. This is when I found out that unlike Ruby which folks who know a computer language can understand immediately, Rails is for web based computer experts only. Like RubyCocoa it is a giant pre-written code which is way beyond my means to fathom.

OK, so I can’t run a big powerboat, but I found Ruby, and it can still lead somewhere. After all the reason I got started with Ruby was Rails, and the reason I wanted Rails was because I wanted to know how to run our experiments off the web, without learning Java, Perl, pithon… Ruby it turns out is enough to write web applications with something called cgi.
There is even a book written on how to write experiments with cgi, “How to Conduct Behavioral Research over the Internet: A Beginner’s Guide to HTML and CGI/Perl”.

Of course learning ruby cgi may not be easy: “A lot of the focus on Ruby Web development these days is on the Rails platform (http://www.rubyonrails.com/), so it can be hard to find much for plain old CGI. There’s a very simple article (http://coolnamehere.com/geekery/ruby/web/cgi.php), the Ruby/Web chapter from “Programming Ruby 1st Edition” (http://ruby-doc.org/docs/ProgrammingRuby/html/web.html). I had trouble finding anything else when I dug around on Google.”

And then, there is the matter of the web server. Apparently, Tiger runs on Apache. But that is not the only choices: “lightTPD is the BEST webserver out there, much faster than apache and much much much more flexible/configurable/secure than tux, and its fcgi php support is simply unbeatable”. Well, but remember I am surfing the waves of the pros, and Apache is installed on my tiger, while lighttpd isn’t. But already it is suggesting that while apache ruby cgi is probably the way to go, I may get better result with lighttpd, ruby and fcgi… another install delight at the horizon.

And then there are other webservers, mongrel, webrick, which is compared to cherrypy for python and Lua!!! and also apache and lighttpd. again, I am getting too far into the wave “Zed and I worked over the weekend on smoothing out the divide between Camping (the 4k web framework) and Mongrel (the slim new Ruby web server mentioned last week.) In just a few days, Mongrel has caught the scent and is totally Campnivorous. Development gems await you.” This web server was written this week-end! On the internet, you can get lost because you are off the beaten path and there is not enough information to get going, or because you are too close to the beta crest and even the installation instructions are greek.

How strange to immerse myself in this noisy world of nerds. There is a physical web, with emails going from computer to computer in search of their destination. And there is the verbal web, with advice and information thrown into the air and connections established between folks who understand each other, while the rest falls back into nothingness. Folks like me get sucked in by mirages that shimmer out of reach, but our personal disappointments and setbacks are but a day on the beach surfing the waves, while the water carves out our future landscape!?