The People

I'm almost starting to take for granted the fact that Drupal will be sending the University of Minnesota Libraries a handful of authentically brilliant and unusually good-natured developers to visit, this being our second high-profile Drupal project. But yeah, we're "Live from Minnesota" as Doug Green points out (I like the meme, no doubt).

The Work

If all goes well with the new search API, I believe it won't be long before we start seeing c|net style faceted drill-down e-commerce sites in Drupal. We already have a well laid-out Apache Solr Module, but a search API will provide more granular control, better integration with core search and ease the long-term burden of maintenance on Robert by moving portions of his code into core.

Personally, this is a chance for me to really think deeply about how to best harness the wonderful work of Doug Green and others within the context of core search features, especially as regards Solr integration. On an even more practical level, this is my own little unit testing bootcamp as I help to write tests for the work being done here this weekend.

Off to bed and on to day two.

Development, Staging and Production Environments - Source Code Control

One panel member (Neil Giarratana of Lucidus) actually lives down the street from a lead engineer at Subversion and has had dinner with him a couple of times to chat about potential "best practices" for a Drupal shop. The result of these conversations was a pretty straight-forward Subversion folder setup: httpdocs, docs (project documentation) and dbdumps (MySQL db dumps) - all three of these folders being replicated on a per-project basis. What is interesting about this practice (to me) is that they avoid using the conventional the conventional branch, trunk, tree setup. The Subversion developer made the case that these conventions are used primarily when you need to collaborate with a large community of people and that they don't make a heck of a lot of sense for internal projects. Neil's gang push out all updates via checkout to there prod, staging and dev environments.

Another Drupal shop, Advomatic (a very, very compitent bunch) check out Drupal core from CVS and then check that into subversion. They then have separate "sites' folders for each project. In Drupal, virtually 100% of unique coding happens in the sites folder, so this makes a lot of sense to me.

Automated Deployments

The OSU Open Source Labs (where Drupal.org is hosted), have adopted the Cfengine configuration engine to automate their deployment process. Cfengine basically provides systems administrators with an Object-Oriented language that allows the to set constraints on pretty much any kind of server configuration. These can be pushed out to multiple machines and can also do things like allow for single command deployment of complex server environments. It sounds really great, but apparently Cfengine is a major pain to get up and running and has a fairly high learning curve. Narayan Newton (OSUOSL) said "We would not be able to administer our systems with our staffing numbers without Cfengine."

Code vs Content Migrations

Content Migration: basically, it's ugly (for now). There have been various efforts to clean this process up in Drupal, but it's still not quite there yet. The key is to backup and have a very clearly articulated rollback policy/practice. If Dries has his way, Drupal 7 will be RDF compliant and this issue will more or less go away. Not only will you be able to easily migrate content, external content will (theoretically) be easy to feature in real-time in any Drupal installation.

Most shops either use rsync and/or straight-up tarbals to deploy code from staging to production. Some also deploy directly from svn checkouts.

QA Best Practices

Neil Giarratana started with an interesting point: automated load testing is of limited interest to his shop. They are pretty familiar with Drupal performance issues, for the most part. Lucidus has a full time QA person that does a great deal of manual functional testing (clicking through the UI), and that is their chief method of finding issues. Neil said the most important thing that they have learned is to have clear and standardized communication processes set up for their customers: indicate (a) when upgrades will happen, (b) when they are actually happening, (c) your fallback plan (in detail), (d) when you are done with the upgrade...for example. He mentioned a Cisco paper on this topic (no reference yet...I need to Call the Science and Engineering Library and have them help me dig it up!).

Other shops do take advantage of automated tools such as Watir + firewatir (Ruby-based) and Firewater for functional testing. The good news, IMHO, is that unit tests AND functional tests will become standard practice in Drupal core; Dries proposed this change in his keynote yesterday. This announcement brought cheers from some core developers, particularly as Dries indicated that unit testing would allow for a much more compressed code freeze period, which would leave more time for active development, new features, etc.

Eliminating (uh, Reducing) Unscheduled Down-Time

One of the more interesting comments concerned *testing* backups. Advomatic apparently had, at one point, made backups of a site that had an exceedingly large sessions table (implementation issue). It was in the millions of records and caused the recovery process to go quite slowly when they did need to recover from a system failure. If they had tested their backups occasionally, they would have caught this problem and corrected it ahead of time.

At least a couple of the shops on the panel make use of systems monitoring applications like Nagios and Cacti. These sorts of tools can provide 'forensic evidence' for systems failures, for example.

We just heard Dries' "State of Drupal" keynote. Here are a couple of things that I found very interesting.

Better APIs in Core
Drupal will, among other things, place a high level of focus upon improving internal and external APIs. For starters, Dries would like to see good (and scalable/fast) support for RDF. This would exist on two levels: (1) how Drupal stores data in general and (2) within the more specific context of RDF triples. The latter would provide for extreme flexibility in terms of creating relationships between system objects. To make the system even more flexible, Dries is suggesting that we raise CCK fields to "first class" object level, rather than at the node level. We could, therefore build n number of relations between field-level objects. It could hardly be any more flexible.

For general storage, this means the Drupal would instantly become compatible with hundreds (thousands?) of external repositories (ex GeoNames) - all repositories that can output to RDF. As Dries is also promoting other formats as JSON, that number goes up even more dramatically (ex Google Spreadsheets can output to JSON).

User Experience
Dries put Usability Testing on the top 11 keys to a "Killer Release" of Drupal 7. I guess his experience at the U of M Libraries was a good one! It's exciting to be part of Drupal's future.

Why are people interested in Flex? One of the other developers in the U Libraries pointed me to Bruce Eckel's article on RIAs and Flex, as an example of how developers are beginning to see this technology.

I like Flex for creating bomb-proof, embeddable and highly interactive widgets; backend administrative interfaces; and the possibility of building cross-platform desktop applications that can easily speak to web-based applications. There are still plenty of things that Javascript would solve more easily and more elegantly, I should add. One always has to be very careful when purchasing a big hammar.

And, being the agile community that it is, Drupal is already on it with tutorials, modules, podcasts and screencasts.

The following, very, very crude example was derived from the tutorial resources listed above as I modified code presented there to create what follows - not much time in my schedule to say more here, but this definitely has got me thinking....

More information at Adobe - and check out their Accessability Page (I'd like to dig more deeply into this aspect).

Apart from what I learned in the seemingly endless stream of useful sessions, by attending a Drupalcon I realized just how much of Drupal is locked up in the heads of core maintainers. That is not a criticism of the project, it's just a fact of life. It's also a case study on why "knowledge management" is so hard. Even when you have the tools (Drupal), capturing knowledge is as hard as effectively representing it to users, and that's not an easy thing. It's also a reason why developers interested in utilizing the platform should seriously consider attending the conference. I found the community to be open-minded, tolerant of newbies and full of brilliant people. An especially effective tactic, I've found, is to corner a developer over a pint.

With that, here's what I learned in school:

Early on, she started to customize her status message throughout the day, often with very funny observations on personal states of being: "Snacking." I started to keep mine fairly up-to-date too, inspired by her creativity. I generally reference the web project de jure in my own messages.

I've also noticed that a number of my librarian buddies have gotten in the customizing habit: "Busy - Off Getting Coffee." I find this sort of thing both mildly amusing and somewhat helpful.

So, there are obvious differences between customizing your status message on your chat account and keeping a twitter message updated (don't know what Twitter is? Have a Look):

• Twitter offers more lines of text for your message


• Chat systems, well, they allow you to chat


• And so on...
  

What is the XMLReader Library, you ask? It’s a “thin PHP layer� on top of a ridiculously fast C/C++ XML pull parsing utility, and I thank my lucky stars for bumping into it (Pull parsing XML in PHP - IBM).

I’ve been working on various projects that require lots of citations from PubMed. And, I wrote some scripts to download them en masse via PubMed’s Entrez Programming Utilities, which offer a simple XML API to PubMed data. So, I downloaded a batch of 50k for my latest project, and casually noticed that the resulting file was 250+ megs. I needed to take that data and transform it into BibTeX. “Oh Dear,� I thought, “Will SimpleXML handle all that data?� Nope, it laughed in my face, a deep, long, belly laugh. Then it walked away.

Rats. I could have broken the file down into baby files, but it sort of annoyed me to have to do so, especially as I was inevitably going to have to rinse and repeat this process every time I needed different data. So, I cast about to find a more acceptable solution. And behold, right under my nose, PHP 5 had gone and bundled the XMLReader Library into its core offerings. Yay.

Instead of laughing at me and walking off into the distance, XMLReader chewed right through that massive document like a Minnesotan with a turkey leg at the state fair.

Enough background. For those of you who do PHP and XML, I’m including my script below. Read on if you’re interested. If you gag at the site of code, move along, nothing to see here.

http://www.huddletogether.com/projects/lightbox2/

I slapped something together on my staging server:

http://webstaging.dewey.lib.umn.edu/image/tid/224

It could hardly have been easier. I also love that this gracefully degrades. With Javascript turned off, users are simply taken directly to the image itself. I can imagine a number of uses for this utility, but I'm most interested in looking at integrating it into online exhibits, where we might have an image that triggers a pop-up that allows users to page through various images. This is a common feature on newspaper sites and the like.

Can't Access This Page? Go to the Original

"Following on a huge amount of discussion and planning at the Internationalization Group, Jose A. Reyero, Károly Négyesi (chx) and Gábor Hojtsy (myself) prepared a big patch to introduce better language support into Drupal 6.x-dev. We got significant code reviews and correction tips from Dries Buytaert, Steven Wittens and Doug Green. As a result, the development version of Drupal 6 now includes a generic language setup screen with support for right-to-left written scripts, native language names, path and domain based language selection and browser language detection. We also added support for language dependent path aliases, so you can have addresses like "example.com/espanol/contacto" and "example.com/english/contact", or alternatively "english.example.com/contact" and "espanol.example.com/contacto".

See the Anoucement: http://drupal.org/node/131516

Can't Access This Page? Go to the Original

Innovate is a bimonthly, peer-reviewed online periodical (ISSN 1552-3233) published by the Fischler School of Education and Human Services at Nova Southeastern University. The journal focuses on the creative use of information technology (IT) to enhance educational processes in academic, commercial, and governmental settings. Our basic assumption is that innovative uses of technology in one sector can inform innovative uses of technology in each of the other sectors.

We welcome submissions from a wide community of practitioners—from professors, teachers, trainers, and support specialists to program administrators, software designers, and leaders in the information technology industry. Articles generally fall into one of the following categories:

Go to Innovate