Recently in computers Category

Capistrano and svn prompts

| No Comments
I've been configuring rails 2 apps for automation with Capistrano 2.6 lately and I encountered a pretty irritating bug with svn. When you run the deploy:update command, Capistrano prompts you for your svn password and your machine password - like it should - and then it hangs forever on the actual deploy step. The problem occurs only when svn continually asks if you want to save your password with one of these dialogs:
-----------------------------------------------------------------------
ATTENTION!  Your password for authentication realm:
    Subversion Repository
can only be stored to disk unencrypted!  
...
-----------------------------------------------------------------------
Solution: make svn stop asking you if you want to save your password. To do this, modify your ~/.subversion/config so that it has (at least) this section:
[auth]
store-passwords = no
store-auth-creds = no
If you're interested, read more to view the error messages from Capistrano.

Debian/Shorewall routing issues

| No Comments

I've recently deployed a new router running Debian Squeeze and Shorewall. The problem started like this:

I have my network configured with five vlans for different areas (servers, workstations, sandboxing, guests, and DMZ) and a /28 subnet assigned for each zone all the fancy network math done.

Everything worked fine until I had to get machines to talk to the outside world.

Shorewall has configurations for 5 zones (corresponding neatly to the vlans) and a few filtering rules for each one. One is behind NAT and the others all have public addresses for which shorewall handles proxyarp.

With and without shorewall running, the router could ping machines both inside and outside the perimeter. In both cases, machines on the inside of the perimeter couldn't ping machines on the outside, but could ping each other.

Something, clearly, was wrong.

I'm pretty sure I spent at least 6 hours trying everything from verifying the upstream switch configuration to checking sysctl variables (mental note: make sure to enable the .forwarding keys!) to checking arp caches everywhere to even swapping out cables and hardware.

As it turns out, the problem was with shorewall. Since shorewall ultimately ends up making a lot of the routing decisions, a badly configured value will ultimately foobar everything else you work so hard to configure. Just remember that even though every other almost value in shorewall.conf is {Yes|No}, IP_FORWARDING takes {On|Off|Keep}.

This was a most extreme case of not RTFMing closely enough.

Passenger on Solaris

| No Comments

So today I set about installing passenger on a Solaris 10 machine. This turned out to be way harder than it seemed.

I went down the "install via gem" road, so it wasn't too painful to being with. When I reached the phase of actually installing passenger to apache, I discovered that the configuration tool is a bit confused by Solaris, especially the apache APU component.

Here's the problem:

* GNU C++ compiler... found at /opt/csw/gcc4/bin/g++
* Ruby development headers... found
* OpenSSL support for Ruby... found
* RubyGems... found
* Rake... found at /opt/csw/bin/rake
* rack... found
* Apache 2... found at /opt/csw/apache/bin/httpd
* Apache 2 development headers... found at /opt/csw/apache/bin/apxs
* Apache Portable Runtime (APR) development headers... not found
* Apache Portable Runtime Utility (APU) development headers... not found

No big deal - we'll just pass it some options, right? Wrong. The CLI only accepts arguments for apxs2 and apr, but not apu.

Running with --apr-config-path /opt/csw/apache2/bin gets us slightly further:


* GNU C++ compiler... found at /opt/csw/gcc4/bin/g++
* Ruby development headers... found
* OpenSSL support for Ruby... found
* RubyGems... found
* Rake... found at /opt/csw/bin/rake
* rack... found
* Apache 2... found at /opt/csw/apache/bin/httpd
* Apache 2 development headers... found at /opt/csw/apache/bin/apxs
* Apache Portable Runtime (APR) development headers... found at /opt/csw/apache2/bin/
* Apache Portable Runtime Utility (APU) development headers... not found

Finally the APU. This is a bit tricky to track down.

This was my process:

passenger-install-apache2-module secretly runs this file:

/opt/csw/lib/ruby/gems/1.8/gems/passenger-2.2.9/bin/passenger-install-apache2-module

Right away we see the following lines:

37 require 'phusion_passenger/dependencies'
52 Dependencies::Apache2,
53 Dependencies::Apache2_DevHeaders

So we follow them:

lib/phusion_passenger/dependencies.rb reveals that it secrely calls PlatformInfo to look for the APU:

304 if PlatformInfo.apu_config.nil?

So we jump into lib/phusion_passenger/platform_info.rb and discover (with some casual grepping):

284 if env_defined?('APU_CONFIG')
285 return ENV['APU_CONFIG']

This means that if we do the following, we can convince the installer that we do, in fact, have the necessary files:

setenv APU_CONFIG /opt/csw/apache2/bin/apu-1-config
passenger-install-apache2-module --apr-config-path /opt/csw/apache2/bin/

Hopefully this will make your life easier.

Drupal GCal Events module and time zones

| No Comments

If you've been using the GCal Events module, you might have noticed a problem where the timezone causes event times to be displayed incorrectly.

In my situation, the time is correct in Drupal, but incorrect in the corresponding GCal link. To force the calendar to interpret the timezone correctly, you can pass the current timezone argument as a GET parameter with the link.

In your GCal Events block, configure the Event Template like this:


<P><A HREF="#URL#&ctz=America/Chicago">#TITLE#</A><br>#LOC##DATE##TIME#

This should force GCal to interpret the times as specific to Central Time.

See the Google API Atom Reference for more information about the ctz option. An example of the link in practice is visible on the Minnesota Population Center (shameless department promotion!) website (http://www.pop.umn.edu) on the right-hand events sidebar.

Drupal and Shibboleth

| No Comments

With the imminent switch to Shibboleth, I've decided to try out hooking up a Drupal instance...to...Shibboleth. Exactly as the title implies.

I'm using Ubuntu Server 10.04 for my server and rather then authentication against a real IdP, I'm using the public TestShib Two service as an IdP and configuring the SP myself.

Here's how I did it:

0) Register with OpenIdP and get a machine ready for testing.

1) Install the necessary packages:
apache mpm-prefork
php5
libapache2-mod-shib2
mysql (for drupal)!
...and all their friends.

2) Get drupal and install that properly. There are plenty of tutorials out there on how to do this.

3) Configure your apache server so that it looks something like this:



ServerAdmin webmaster@localhost
ServerName x-128-101-79-21.pop.umn.edu
UseCanonicalName On

DocumentRoot /var/www

Options FollowSymLinks
AllowOverride None

Alias /secure /var/www/drupal

Options Indexes FollowSymLinks Multiviews

Order allow,deny
Allow from all

AuthType Shibboleth
ShibRequireSession On
require shibboleth


...and a whole bunch more as needed. You can actually base it off the default-ssl site (...like I did.)

4. Somewhere in your shibboleth2.xml (probably in /etc/shibboleth), you have something that looks like this:









This is the really important part. It's telling you exactly which paths on the server are going to be secured by

vhosts with modperl

| No Comments

This post deals with configuring mod_perl environments with vhosts. Our objects for the post are to configure an Apache2 prefork server with mod_perl 2.0 with the following specifications:

1) Each vhost should have its own interpretter pool with its own paths.
2) vhosts do not need to use custom directives during startup.
3) Custom liraries in each vhost should not be shared with the others.
4) Users should be able to execute mod_perl scripts only from docs/bin.

Here's a snip from my apache2ctl status: "Server Version: Apache/2.2.11 (Ubuntu) mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0"

Let's get started!

I have three vhosts {a, b, c} each listening on ports 5050, 8080, and 9090 respectively.

Here's my initial /etc/apache2/sites-available/vhosts.conf file:


<VirtualHost *:5050>
DocumentRoot /var/www/a
#PerlPostConfigHandler MyApache2::Test->post_config
</VirtualHost>

<VirtualHost *:8080>
DocumentRoot /var/www/b
#PerlPostConfigHandler MyApache2::Test->post_config
</VirtualHost>

<VirtualHost *:9090>
DocumentRoot /var/www/c
#PerlPostConfigHandler MyApache2::Test->post_config
</VirtualHost>


Here's my initial /etc/apache2/httpd.conf file:

ServerName localhost


PerlSwitches -I/var/myperl/dep_lib
PerlPostConfigHandler MyApache2::Test->post_config

/var/myperl/dep_lib/MyApache2::Test.pm looks like this:


package MyApache2::Test;
use Apache2::ServerRec;
use Apache2::Const -compile => OK
use strict;
use warnings;

sub post_config {
my ($self, $conf_pool, $log_pool, $temp_pool, $s) = @_;
warn "HELLO " . $s->port . "\n";
}

1;

When we start the server, this is the output we get:

root@larx:~# tail -fn 0 /var/log/apache2/error.log
HELLO 0
HELLO 0
[Tue Mar 09 19:09:00 2010] [notice] Apache/2.2.11 (Ubuntu) mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0 configured -- resuming normal operations
[Tue Mar 09 19:09:00 2010] [info] Server built: Nov 13 2009 21:59:28
[Tue Mar 09 19:09:00 2010] [debug] prefork.c(1032): AcceptMutex: sysvsem (default: sysvsem)

This shows that the global configuration (*:0) is saying hello when it calls the PostConfigHandler. After the first call to PerlPostConfigHandler, Apache restarts itself to test its graceful restart ability. [1]

Now what happens when we try to implement a PostConfigHandler in a VirtualHost? According to the documentation, since PerlPostConfigHandler is valid in the SRV scope, we should be able to do this:

/etc/apache2/httpd.conf:


ServerName localhost
PerlSwitches -I/var/myperl/dep_lib
#PerlPostConfigHandler MyApache2::Test->post_config

/etc/apache2/site-available/vhosts.conf:


<VirtualHost *:5050>
DocumentRoot /var/www/a
PerlSwitches -I/var/myperl/dep_lib
PerlPostConfigHandler MyApache2::Test->post_config
</VirtualHost>

<VirtualHost *:8080>
DocumentRoot /var/www/b
</VirtualHost>

<VirtualHost *:9090>
DocumentRoot /var/www/c
</VirtualHost>


and the startup:

root@larx:~# tail -fn 0 /var/log/apache2/error.log
[Tue Mar 09 19:19:23 2010] [notice] Apache/2.2.11 (Ubuntu) mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0 configured -- resuming normal operations
[Tue Mar 09 19:19:23 2010] [info] Server built: Nov 13 2009 21:59:28
[Tue Mar 09 19:19:23 2010] [debug] prefork.c(1032): AcceptMutex: sysvsem (default: sysvsem)

Notice that the PerlPostConfigHandler hasn't said anything. This is actually by design, contrary to what the documentation might say. In reality, PerlPostConfigHandler is only valid in the server-wide configuration context, NOT in vhosts. There are two ways to get around this.

1) Patch your mod_perl installation using a patch (possibly outdated) available on the mod-perl mailing lists here [2].
2) Use startup.pl files in a per-host configuration scheme.

Since I have less than no desire to go hacking around inside the monster, I've opted for choice 2.

To achieve this, we need to provide each vhost with a way to load some file at server startup. In /var/www/(a|b|c) we place a file called startup.pl that looks remarkably like this: (just make sure to s/a/(b|c) when and where appropriate)

/var/www/a/startup.pl:


#!/usr/bin/perl

use lib qw( /var/www/a/lib /var/www/a/lib/dep_lib );

use Data::Dumper;

use warnings;
use strict;

warn "Dumping the A startup.pl:\n";
print STDERR Dumper(\@INC);
warn "Finished processing A startup!\n";

1;

Notice that "1;". That's pretty important since the script is effectively being require()'d by mod_perl and therefore must return true.

Next we need to configure the vhost:

/etc/apache2/sites-available/vhosts.conf:


<VirtualHost *:5050>
DocumentRoot /var/www/a
PerlPostConfigRequire /var/www/a/startup.pl
</VirtualHost>

<VirtualHost *:8080>
DocumentRoot /var/www/b
</VirtualHost>

<VirtualHost *:9090>
DocumentRoot /var/www/c
</VirtualHost>


This is the outcome from a startup run:

root@larx:~# tail -fn 0 /var/log/apache2/error.log
[Thu Mar 11 16:54:12 2010] [info] removed PID file /var/run/apache2.pid (pid=4057)
[Thu Mar 11 16:54:12 2010] [notice] caught SIGTERM, shutting down
Dumping the A startup.pl:
$VAR1 = [
'/var/www/a/lib',
'/var/www/a/lib/dep_lib',
'/var/myperl/dep_lib',
'/etc/perl',
'/usr/local/lib/perl/5.10.0',
'/usr/local/share/perl/5.10.0',
'/usr/lib/perl5',
'/usr/share/perl5',
'/usr/lib/perl/5.10',
'/usr/share/perl/5.10',
'/usr/local/lib/site_perl',
'.',
'/etc/apache2'
];
Finished processing A startup!
Dumping the A startup.pl:
$VAR1 = [
'/var/www/a/lib',
'/var/www/a/lib/dep_lib',
'/var/myperl/dep_lib',
'/etc/perl',
'/usr/local/lib/perl/5.10.0',
'/usr/local/share/perl/5.10.0',
'/usr/lib/perl5',
'/usr/share/perl5',
'/usr/lib/perl/5.10',
'/usr/share/perl/5.10',
'/usr/local/lib/site_perl',
'.',
'/etc/apache2'
];
Finished processing A startup!
[Thu Mar 11 16:54:15 2010] [notice] Apache/2.2.11 (Ubuntu) mod_apreq2-20051231/2.6.0 mod_perl/2.0.4 Perl/v5.10.0 configured -- resuming normal operations
[Thu Mar 11 16:54:15 2010] [info] Server built: Mar 9 2010 21:04:15
[Thu Mar 11 16:54:15 2010] [debug] prefork.c(1032): AcceptMutex: sysvsem (default: sysvsem)


Notice that we see the Dumper being called twice. This is, again, because immediately after starting up for the first time Apache starts once, and then restarts to test its graceful restart ability. [1]

Really interesting side note: when you reload (aka "graceful restart," "-k graceful," SIGUSR1) the apache server will start new children with the new configuration settings. On the other hand, restarting (aka "restart now, "-k restart", SIGHUP) stops all children, reloads the parent, and then loads all children with the new configuration. This means that when you send "reload" you get one startup pass from modperl instead of two (since the server is already running it doesn't test it's ability to restart) whereas with restart, the server (for all intents and purposes) stops and starts again meaning that you see the output from modperl twice. [3]

Now that we know what's going on with the modperl startup, let's finish our setup. The remaining adjustments are fairly straightforward:

/etc/apache2/httpd.conf:


ServerName localhost
PerlSwitches -I/var/myperl/dep_lib
#This option allows us to say things like print "Content-Type: text/plain\n\n"
PerlOptions +ParseHeaders
#We still might want to do something server-wide
PerlPostConfigHandler MyApache2::Test->post_config

/etc/apache2/sites-available/vhosts.conf:


#I've actually ignored one of the criteria I set earlier (because I can, and partially to show a point):
#vhost a allows scripts to run under modperl from docs/bin even though the documentroot is under ExecCGI.
#The PerlOptions +Parent gives vhost a it's own perl interpreter pool complete with its own @INC.
<VirtualHost *:5050>
DocumentRoot /var/www/a/docs
Options ExecCGI +Includes FollowSymLinks +MultiViews Indexes
DirectoryIndex index.html

PerlPostConfigRequire /var/www/a/startup.pl
PerlOptions +Parent

<Directory /var/www/a>
AllowOverride All
</Directory>

<Directory /var/www/a/docs/bin>
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
</Directory>
</VirtualHost>

#vhost b is configred identically to vhost a.
<VirtualHost *:8080>
DocumentRoot /var/www/b/docs
Options ExecCGI +Includes FollowSymLinks +MultiViews Indexes
DirectoryIndex index.html

PerlPostConfigRequire /var/www/b/startup.pl
PerlOptions +Parent

<Directory /var/www/b>
AllowOverride All
</Directory>

<Directory /var/www/b/docs/bin>
SetHandler perl-script
PerlResponseHandler ModPerl::Registry
</Directory>
</VirtualHost>

#vhost c is a little different. This vhost allows scripts to be run under modperl from anywhere it its document root.
<VirtualHost *:9090>
DocumentRoot /var/www/c/docs
Options ExecCGI +Includes FollowSymLinks +MultiViews Indexes
DirectoryIndex index.html

PerlPostConfigRequire /var/www/c/startup.pl
PerlOptions +Parent
SetHandler perl-script
PerlResponseHandler ModPerl::Registry

<Directory /var/www/c>
AllowOverride All
</Directory>

</VirtualHost>

There are two things that should be explained:

PerlOptions +Parent
...creates a separate interpreter pool for each of the vhosts. These pools do not inherit any settings from other pools.

SetHandler perl-script
...tells Apache to parse all files in that Directory (also valid for Locations) as perl-scripts. ("SetHandler modperl" also works just as well here). More on this in a week or two.

PerlResponseHandler ModPerl::Registry
...tells modperl that everytime it needs to process something, use ModPerl::Registry to generate the response by compiling/parsing/executing your code when it needs to.

Finally, to prove that this actually works (in addition to having the error log), here's a script that you can actually run under modperl in each of the vhosts. Notice the lack of a #! line and a 1;. They're not necessary [in this particular instance]!

test.pl:


use Data::Dumper;

print "Content-type: text/plain\n\n";
for(sort keys %ENV) {
print "$_ => $ENV{$_}\n";
}

print Dumper(\@INC);

Finally, my directory structure looks like this:

root@larx:/var/www# tree
.
|-- a
| |-- docs
| | |-- bin
| | | `-- test.pl
| | `-- index.html
| |-- lib
| | `-- dep_lib
| `-- startup.pl
|-- b
| |-- docs
| | |-- bin
| | | `-- test.pl
| | `-- lib
| | `-- dep_lib
| |-- lib
| | `-- dep_lib
| `-- startup.pl
`-- c
|-- docs
| |-- bin
| | `-- test.pl
| |-- lib
| | `-- dep_lib
| `-- test.pl
|-- lib
| `-- dep_lib
`-- startup.pl

19 directories, 8 files

Try visiting http://localhost:8080/bin/test.pl and http://localhost:9090/test.pl . Now you have mod_perl working with isolated vhosts!

Join me next week as I delve further into mod_perl. Possible topics include authentication and authorization, ModPerl::Registry and/or ModPerl::PerlRun, custom environment variables, and further per-vhost configuration options.


References:

[1] http://perl.apache.org/docs/2.0/user/handlers/server.html#Server_Life_Cycle

[2] http://www.gossamer-threads.com/lists/modperl/modperl/60796#60796

[3] http://httpd.apache.org/docs/2.2/en/stopping.html#graceful

CGI::Application and POST forms

| No Comments

So you're writing a webapp using CGI::Application, you add a form to the application and set its submission method to POST and all of a sudden you can't seem to get a runmode to handle it. Instead of mixing GET and POST parameters, here's a way to convince C::A to work with POST data.

I usually use CGI::Application::Plugin::TT and CGI::Application::Plugin::AutoRunmode in my applications.

Suppose your application has two screens: statistics and administration.

In the administration screen a user can add or remove other users.
In the statistics screen a user can generate reports based on a type and some parameters.

Each of these screens corresponds to a run mode. Here's the general logic behind the runmodes:

Enter the mode and create useful objects (esp. $self = shift;)
Test to see if the form has been submitted.
If it has, process the associated information and return something useful.
If it hasn't, return the information to display.
Return

Both methods use POST for form submission, but can be accessed by GET URL parameters by default.

When your clients visit the website http://example.com/cgi-bin/myapp?rm=stats they get sent to the statistics page and visiting http://example.com/cgi-bin/myapp?rm=admin they get sent to the administration page.

Here's the problem: When the user clicks on the "submit" button on either page, the runmode parameter (rm=stats) becomes part of the CGI environment variable QUERY_STRING (which, by definition, contains URL parameters).

Suppose your instance script (cgi-bin/myapp) looks like this:


my $app = MyApp::Admin->new();
$app->run();

It may not appear obvious, but without any fancy processing, passing rm?=stats will cause the value of rm to be the runmode. Read more about that here: http://search.cpan.org/~markstos/CGI-Application-4.31/lib/CGI/Application.pm .

Anyway, if you try that with a POST request, it will blow up in your face (I suggest using use CGI::Carp qw(fatalsToBrowser) to have it explode cleanly).

Oracle and Jaspersoft

| No Comments

Usually I think of connecting to a data source as a relatively straightforward process involving something like Perl's DBI or .NET's System.Data.Odbc namespace and a connection string. When using a product like Jaspersoft, however, this task becomes slightly more difficult.

  1. Download the appropriate driver from Oracle here: http://www.oracle.com/technology/software/tech/java/sqlj_jdbc/index.html. There are a lot of really gory details about Java versions, so make sure to read the readme very carefully.

    For this example I'm using the 11g Release 1 (11.1.0.7.0) JDBC Drivers, download the file ojdbc6.jar.
  2. You'll want to install the file into the /webapps/jasperserver/WEB-INF/lib directory and then chown/cmod it as appropriate. (You may need to reload the application or reload the server after this.)
  3. Next, you're probably familar with this screen:

    jaspersoft.JPG

    Let's fill in the values for the form:

    Name: MyConnection
    Description: This is a description of my connection.
    Driver: oracle.jdbc.OracleDriver
    URL: jdbc:oracle:thin:@host.example.com:1521:mysid
    User Name: user123
    Password: notagoodpassword

  4. At this point, you should be able to connect to the Oracle server from JasperSoft.

Unfortunately, I haven't entirely gone through all the log4j settings that JasperSoft has to argue, so I can't really say "go check the logs" in good faith. My diagnostic method was writing a quick java program to make sure that my connection strings were good and then just moving the driver around until the java vm detected it based on where the other JDBC provider drivers were.

About this Archive

This page is an archive of recent entries in the computers category.

Amusement is the previous category.

History is the next category.

Find recent content on the main index or look in the archives to find all content.