Jeff Silvis's Blog

Problem getting to localhost website on my Mac after snow lepord

I got a 403 when trying to reach localhost: Solution from: techtrouts.com
Mac OS X 10.5: Web Sharing – “Forbidden 403″ on http://localhost/~username
If, usually after installing/upgrading your mac, you can’t access your ~/Sites folder on your browser
( http://localhost/~username , where username is your mac username):
* Open a Terminal and $ sudo nano /private/etc/apache2/httpd.conf ;
* ctrl+w (where is) : “<Directory />” ;
* Alter the line below from “Deny from all” to “Allow from all“;
* ctrl+x and y to save;
* now $ sudo httpd -k restart to restart your Apache 2 server;

Then I got the error:
httpd: Syntax error on line 459 of /private/etc/apache2/httpd.conf: Syntax error on line 15 of /private/etc/apache2/extra/httpd-userdir.conf: Syntax error on line 8 of /private/etc/apache2/users/+entropy-php.conf: Cannot load /usr/local/php5/libphp5.so into server: dlopen(/usr/local/php5/libphp5.so, 10): no suitable image found. Did find:\n\t/usr/local/php5/libphp5.so: no matching architecture in universal wrapper
Fixed this by changing :
LoadModule php5_module /usr/local/php5/libphp5.so
to
LoadModule php5_module libexec/apache2/libphp5.so
in
/private/etc/apache2/users/+entropy-php.conf

Posted by Jeffrey Silvis at 4:04 PM | Permalink | Comments (0)

Reaching the Fedora Repository from a remote box ... localhost issue

I was trying to contact a fedora box from a remote box and I had the problem that when I hit links within Fedora, the word "localhost" kept appearing in the URL. This resolved to a "404" because Fedora is not on my box. I fixed this by editing:
$FEDORA_HOME/server/config/fedora.fcfg and changing the line: <param name="fedoraServerHost" value="localhost"> In the line above, I replaced the word "localhost" with the actual IP address.

Posted by Jeffrey Silvis at 3:01 PM | Permalink | Comments (0)

Turning XACML ON and OFF in Fedora

2.1 Enabling/Disabling XACML Policy Enforcement To enable/disable XACML policy enforcement in Fedora, use the Fedora configuration file (fedora.fcfg). Whether Fedora uses XACML for authorization decisions is controlled by the ENFORCE-MODE parameter in the Authorization module: <param name="ENFORCE-MODE" value="enforce-policies"/> The ENFORCE-MODE parameter can contain one of three values, with the following meanings: 1. enforce-policies – enable XACML enforcement to determine whether a request is permitted or denied
2. permit-all-requests – disable XACML enforcement; PERMIT every request by default
3. deny-all-requests – disable XACML enforcement; DENY every request by default
The enforce-policies setting is used to enable the enforcement of XACML policies, and is the default setting for a Fedora repository. The permit-all-requests setting can facilitate testing code independent of security. The deny-all-requests setting can be used to quickly shut down access to the server, but requires a server restart to affect this. Tomcat container security is, of course, still a first barrier to authentication/authorization (i.e., Fedora's Tomcat web.xml specifies access protection earlier than XACML. Tomcat container security is always in place regardless of the setting for parameter ENFORCE-MODE. see Fedora Commons on XACML

Posted by Jeffrey Silvis at 10:40 AM | Permalink | Comments (0)

DC Metadata fields used by UMN DSPACE instances:

dc.title
dc.title.alternative
dc.contributor.author
dc.contributor.editor
dc.subject
dc.subject.other
dc.date.issued
dc.identifier.citation
dc.relation.ispartofseries
dc.description.abstract
dc.description
dc.identifier.govdoc
dc.identifier.uri
dc.identifier.isbn
dc.identifier.issn
dc.identifier.ismn
dc.identifier
dc.relation
dc.format.extent
dc.language
dc.extent

Posted by Jeffrey Silvis at 3:53 PM | Permalink | Comments (0)

Last step in getting perl drivers for Postgres

While trying to install new DBD::Pg drivers on mac os x Snow Leopard I got the error below, when running test code:
dyld: lazy symbol binding failed: Symbol not found: _PQconnectdb Referenced from: /opt/local/lib/perl5/site_perl/5.10.1/darwin-2level/auto/DBD/Pg/Pg.bundle Expected in: dynamic lookup I found that the Makefile.PL that cpan made for DBD::Pg was pointed at the wrong directories for libraries. I made the changes below to Makefile.PL (located in ~/.cpan/build/XML-DOM-1.44-tM9lK0):
#$POSTGRES_INCLUDE = $ENV{POSTGRES_INCLUDE} || $pg->inc_dir || "$ENV{POSTGRES_HOME}/include"; $POSTGRES_INCLUDE = '/opt/local/include/postgresql83'; #$POSTGRES_LIB = $ENV{POSTGRES_LIB} || $pg->lib_dir || "$ENV{POSTGRES_HOME}/lib"; $POSTGRES_LIB = '/opt/local/lib/postgresql83'; Then
sudo perl Makefile.PL
sudo make test
sudo make install

And the error was gone.

Also I had to switch
#!/opt/local/bin/perl # /usr/bin/perl

Posted by Jeffrey Silvis at 4:29 PM | Permalink | Comments (0)

utf-8 utf-16 and UNIX

1) Convert utf-16 to utf-8.
iconv -f utf-16 -t utf-8 bell-map-IMAGESspreadsheet.xml > tt
2) Have less read utf-8
export LESSCHARSET='utf-8'
cat just works.

Posted by Jeffrey Silvis at 2:55 PM | Permalink | Comments (0)

Some useful drupal sql

Find uid given user naa (in this case naa)

select uid from users where name='naa';

Get the number of uploads by user 'naa'

select count(upload.fid) from upload, node where node.nid=upload.nid and node.uid=(select uid from users where name='naa');

Get the number of files owned by user 'naa'

select files.filepath from files, upload where upload.fid=files.fid and upload.nid=68652;

Find path to uploaded files

select files.filepath from files, upload where upload.fid=files.fid and upload.nid=68652;

Posted by Jeffrey Silvis at 12:44 PM | Permalink | Comments (0)

title element wrong for media ingest

The IMAGES xml files used to ingest data into the the media repository contain a flaw.
Bad version (current): <title main="Duplex House" variant="Residence project" variant="Exterior presepctive"/> Good: <title type="main" > Duplex House </title> <title type="variant" > Residence project </title> <title type="variant" > Exterior perspective </title> Effected files: bln-dcugranting2007.xml
botanical-dcugranting2007.xml
cbi-dcugranting2007.xml
ellis-dcugranting2007.xml
mno-dcugranting2007.xml
mss-alexanderBros.xml
mss-dcugranting2007.xml

Some more files (all the rest):

s 001-bell-historicalmaps
005-ymca-wwiPhotos
006-map-19thCent
008-eas-ming
011-mss-purcellMasonite

Posted by Jeffrey Silvis at 12:21 PM | Permalink | Comments (0)

David Naughton gave me some fast perl code for hashes

#!/exlibris/sfx_ver/sfx_version_3/app/perl-5.8.6/bin/per use strict; use warnings; # Takes export file from SFX and generates host definitions to append to # the ezproxy.cfg file. # Open SFX export file for manipulation my $sfx_url_file = shift @ARGV; open (my $fh_sfx, '<', $sfx_url_file) or die "File Open Failed: $!"; # get hostnames into a hash my %hostnames; while (<$fh_sfx>) { my $line = $_; next if $line =~ /^#/; my ($sfx_target, $sfx_url) = split /\t/, $line; if ($sfx_url =~ m/\:\/\/(.*?)\//) { # Hash keys are always unique, so if a key for $1 # already exists, this line will clobber its value: $hostnames{$1} = undef; # If you want to keep track of how many times each # hostname appears, you can use this magic: # $hostnames{$1}++; # More verbose version of the code above: # if (!(exists $hostnames{$1})) { # $hostnames{$1} = 0; # } # $hostnames{$1} = $hostnames{$1} + 1; } } # print each unique hash key, with some added text for my $hostname (keys %hostnames) { print "HJ $hostname\n"; } # close file handle close $fh_sfx or die "File Close Failed: $!";

Posted by Jeffrey Silvis at 10:47 AM | Permalink | Comments (0)

dspace batch ingest

Format of files to ingest

Here is a breakdown of the files : batch_files - top level directory (name does not matter)
   I
   Ingest1 - This a prototype for the directories that you will create. Give these directories any name you want
   I
   contents - contains  the asset name. The fields are separated by a single tab. File must be called "contents"
   dublin_core.xml - contains the DC metadata. This file must be called "dublin_core.xml"
   UDCsubmissionguidelines.pdf - This is the asset. You may use whatever name you want. However the name of the
   asset must appear in the "contents" file.

tarball that gives working sample of the directory structure.

command

/dspace/dspace-ir/bin$ ./dsrun org.dspace.app.itemimport.ItemImport -a -c CollectionHandle -e Eperson -s /PATH_TO_BATCH_FILES/batch_files -m /home//PATH_TO_BATCH_FILESs/Ingest1/mapfile.txt

Resources

Dorothea Salo's EXCELLENT blog
ingest-export.ppt ARD Prasad
ScalabilityIssues - DSpace Wikis

Posted by Jeffrey Silvis at 3:38 PM | Permalink | Comments (0)

putting captchas into dspace using jcaptcha

ingest-export.ppt

code needed for a captcha

The file form.jsp had to be modified so that if the captcha was not set or was not correct the form for the email form was not produced.
If the email form was not called then a jsp was called that produced the captcha: captcha_main.jsp The jsp captcha_main.jsp called in order the following java classes to make the captcha image:
ImageCaptchaServlet.java
CaptchaServiceSingleton.java
MyImageCaptchaEngine.java
Also the file dspace-web.xml had to be modified.

X11 not on the box where tomcat runs

If X11 is not the box with tomcat, you will get an error like "port 6000 not available". To fix this put:

-Djava.awt.headless=true

into the catalina.sh file.

Helpful links

How to use jsp-forward tag
How do I perform browser redirection from a JSP pages
Breaking a Visual CAPTCHA

Posted by Jeffrey Silvis at 4:26 PM | Permalink | Comments (0)

Error in DSPACE search ... handles from index not having items

Symptom

Hi, I was checking abstracts for Volume 41 No. 2 of Journal of Agricultural and Applied Economics. When I searched on "Race, Gender, School . . ." I got an error message saying that the website had experienced an internal error and I tried it again w/ the same results. Since the same message requested letting you know of the problem I am responding. Sincerely,

Error Message

Trying the search above generated the error message: An internal server error occurred on http://ageconsearch.umn.edu: Date: 8/28/09 10:59 AM Session ID: 6AAEA1B2D8AADD0C7F1BE28731AE9083 -- URL Was: http://ageconsearch.umn.edu/simple-search?sort=date&query=%28%28keyword%3Arace%29%29&from_advanced=true&query2=&field1=keyword&conjunction2=AND&query1=race+&field2=keyword&query3=&conjunction1=AND&field3=ANY&SortDirection=descending -- Method: GET -- Parameters were: -- field3: "ANY" -- field2: "keyword" -- field1: "keyword" -- sort: "date" -- query3: "" -- query2: "" -- SortDirection: "descending" -- query1: "race " -- query: "((keyword:race))" -- from_advanced: "true" -- conjunction2: "AND" -- conjunction1: "AND" Exception: java.sql.SQLException: Query "((keyword:race))" returned unresolvable handle: 53087 at org.dspace.app.webui.servlet.SimpleSearchServlet.doDSGet(SimpleSearchServlet.java:271) at org.dspace.app.webui.servlet.DSpaceServlet.processRequest(DSpaceServlet.java:151) at org.dspace.app.webui.servlet.DSpaceServlet.doGet(DSpaceServlet.java:99) at javax.servlet.http.HttpServlet.service(HttpServlet.java:689) at javax.servlet.http.HttpServlet.service(HttpServlet.java:802) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148) at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:199) at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:282) at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:767) at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:697) at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:889) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684) at java.lang.Thread.run(Thread.java:595)

Outline of Solution

1) A handle was found by search that was not in the database (probably a deleted file and IndexAll has not run yet).
2) I found the segment of code that collects into a list all the items found by search.
3) When any word that was on this deleted record was input, the search routine would throw an error and halt.
4) I rewrote the code to ignore handles that are not in the database.
5) Ran a few basic tests on Odin (our private DSAPCE setup).
6) Deployed it to strip1 (production site)

Modified Code

// FROM SimpleSearchServlet.java /* resultsItems = new Item[numItems]; for (int i = 0; i < numItems; i++) { String myhandle = (String) itemHandles.get(i); Object o = HandleManager.resolveToObject(context, myhandle); resultsItems[i] = (Item) o; if (resultsItems[i] == null) { throw new SQLException("Query \"" + query + "\" returned unresolvable handle: " + myhandle); } } */ // The code below will only add handles to the list if a well defined // item is associated with it. Item[] resultsItems_temp = new Item[numItems]; int item_count =0; for (int i = 0; i < numItems; i++){ String myhandle = (String) itemHandles.get(i); Object o = HandleManager.resolveToObject(context, myhandle); if (o != null){ resultsItems_temp[item_count] = (Item) o; item_count++; } } resultsItems = new Item[item_count]; for (int i = 0; i < item_count; i++){ resultsItems[i] = resultsItems_temp [i]; }

Posted by Jeffrey Silvis at 9:53 AM | Permalink | Comments (0)

Ames collection ingest ... multiple metadata per file

The problem

While ingesting the metadata for the AMes collection I found a problem. Below is a list of metadata sets that map to the same image. s

identifier local	Image
ama00711	ama00711.jpg
ama00712	ama00711.jpg
amp00259	amp00259.jpg
ap00259	amp00259.jpg
amp00435	amp00435.jpg
amp00436	amp00435.jpg
amp00448	amp00449.jpg
amp00449	amp00449.jpg
amp00513	amp00531.jpg
amp00531	amp00531.jpg

Jason's solution

identifier local Image ama00711 ama00711.jpg ama00712 ama00711.jpg - error in the data. Should point to ama00712.jpg (I've fixed it in IMAGES) amp00259 amp00259.jpg ap00259 amp00259.jpg - delete this version amp00435 amp00435.jpg amp00436 amp00435.jpg - error in the data. Should point to amp00436.jpg (I've fixed it in IMAGES) amp00448 amp00449.jpg - error in the data. Should point to amp00448.jpg (I've fixed it in IMAGES) amp00449 amp00449.jpg amp00513 amp00531.jpg - error in the data. Should point to amp00513.jpg (I've fixed it in IMAGES) amp00531 amp00531.jpg

What to do from here

Bill will need to pull the ames collection from IMAGES. I will need to re ingest it and delete ap00259.

Posted by Jeffrey Silvis at 2:49 PM | Permalink | Comments (0)

Problem with a dspace user and solution

Problem

A problem that happened once before. Although Erin George has successfully uploaded files to some of the collections in the Univeresity Archives community, she is now if a cycle where she logs in, goes to a collection, and whatever she does gets cycled back to the log in screen with her x500 there but no password. She enters her password, and goes through the same thing again and again. This happened to one of our students.

Solution

This command fixed it:
update eperson set netid = 'georg038' where eperson_id = '190';

Posted by Jeffrey Silvis at 2:52 PM | Permalink | Comments (0)

svn notes ... media_repo Externals Switch

Our svn site

https://chaucer.lib.umn.edu/svn/dldl/

Externals

Chad is using svn externals to have several projects at the DLDL link to the same set of files within the svn tree. From the command line to view the externals use the commands: svn propedit svn:externals . svn propget svn:externals FileName

Switch

The svn switch command allows a user to update to a new version of a package.
Example from John Barneson: First make drupal go offline. https://dldl00.lib.umn.edu/admin/settings/site-maintenance svn switch https://chaucer.lib.umn.edu/svn/dldl/drupal/releases/acquia-drupal-1.2.13/ .

Useful URLs

Jeremy Knope dot com OS X svn client

Posted by Jeffrey Silvis at 1:58 PM | Permalink | Comments (0)

Jeff Silvis's Blog

November 17, 2009