Possible 404 bug on rails sites with curl -I ?

[inspirix]

Not sure if this is a bug or a lack of me setting a certain config option:

ALL sites are _working_ perfectly through the browser, just noticed this odd thing when using curl -I:

PHP sites respond fine:

$ curl -I http://www.mpoweredbaseball.com
HTTP/1.1 200 OK
X-Powered-By: PHP/4.4.4
Content-type: text/html
Server: LiteSpeed
Date: Sun, 17 Sep 2006 06:22:30 GMT

Access log:
67.101.101.53 - - [16/Sep/2006:23:22:30 -0700] "HEAD / HTTP/1.1" 200 - "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"

Rails sites respond with a 404:

$ curl -I http://www.isup4sale.com
HTTP/1.1 404 Not Found
Server: LiteSpeed
Date: Sun, 17 Sep 2006 06:30:22 GMT

Access log:
67.101.101.53 - - [16/Sep/2006:23:22:06 -0700] "HEAD / HTTP/1.1" 404 - "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"

 

[mistwang]

Are you using the root directory of the Rails app to configure the Rails context? Do not use the path to the public/ folder.

Any configuration warning or error in the error log?

Is there a index file under public/ folder? Is the name of the index file added to the list of "Index files" configuration for that vhost?

 

[inspirix]

Are you using the root directory of the Rails app to configure the Rails context? Do not use the path to the public/ folder.

Yes, using the root directory, not public/

Any configuration warning or error in the error log?

None relevant. I do have this happening on 2 boxes (one freebsd, one osx), but i think this is totally seperate and will open a convo on it if I cant figure out how to get rid of it:
2006-09-17 11:53:54.001 [DEBUG] Failed to execute 'mpstat' command: No such file or directory

Is there a index file under public/ folder? Is the name of the index file added to the list of "Index files" configuration for that vhost?

If i put an index.html file in my public/ then curl -I works. However, I wouldn't normally have an index file in my rails app because I would have rails handling that.

 

[mistwang]

2006-09-17 11:53:54.001 [DEBUG] Failed to execute 'mpstat' command: No such file or directory

That's all right.

If i put an index.html file in my public/ then curl -I works. However, I wouldn't normally have an index file in my rails app because I would have rails handling that.

When there is no index file in the public folder, LSWS will forward the request to Rails application for processing, if there is a routing error, rails will output a routing error page. What is output for the same request when WEBrick or Mongrel is used?
Is .htaccess support being turned on for that vhost? Maybe the rewrite rules in public/.htaccess mess things up.

 

[inspirix]

I dont think the issue is really a big deal. My best guess is that it just has to do with how LS handles the HEAD and GET requests differently. I assume HEAD doesnt follow the error handler like GET does and instead returns the direct result of the request, which would be the 404. On the other hand, on GET, the 404 is invoked but that yields the 200 rails page, which is the only thing returned to the client. I assume this is for the sake of speed that the implementations are not the same. But... I am assuming a lot here so I may be wrong. I think it has less to do with configuration now than it does with how LS works internally.

Here is the log from LS rails site:

67.188.175.53 - - [17/Sep/2006:19:14:50 -0700] "HEAD / HTTP/1.1" 404 - "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"
67.188.175.53 - - [17/Sep/2006:19:15:15 -0700] "GET / HTTP/1.1" 200 2912 "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"

And here is the log from a lighttpd/fcgi rails site (thats all i had at the moment):

67.188.175.53 - - [17/Sep/2006:19:52:32 -0700] "HEAD / HTTP/1.1" 200 0 "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"
67.188.175.53 - - [17/Sep/2006:19:53:03 -0700] "GET / HTTP/1.1" 200 7969 "-" "curl/7.15.3 (powerpc-apple-darwin8.6.0) libcurl/7.15.3 OpenSSL/0.9.7i zlib/1.2.3"

Again, I think its just that on LS, HEAD returns the immediate response without following the error handler (404), where on lighttpd/fcgi, HEAD seems to process the error-handler and then returns its response (200).

Am I right/wrong? If right, does it really matter?

 

[mistwang]

You are right.

I overlooked that it is "HEAD" request. We will change it to follow 404 handler. As HTTP protocol requires that a HEAD request should return the same status code as a GET request.

Thanks for the bug report.