Main

November 16, 2011

Cron to do mysqldump

Create a mysql user that will do the backups

GRANT LOCK TABLES, SELECT ON *.* TO 'backup'@'%.oit.umn.edu' IDENTIFIED BY PASSWORD;

The Script to run under cron

#!/usr/bin/perl

printf("Running...");

####################
# Backup settings #
###################

my $mysql_user = "backup";
my $mysql_pwd = "password";
my $mysql_server = "mysql_host";
my $dir_base = "path_to_backup_dir";

my ($y, $m, $d) = (localtime)[5,4,3];

$y += 1900;
$m += 1;

if ($m < 10) { $m = "0" . $m; }
if ($d < 10) { $d = "0" . $d; }

$stamp = $y . "_" . $m . $d;
$file_dumpname = $stamp . "_mysqldump.sql";

# Format:
#/swadm/assetstore_stage/Backups/mysql/2009_0921_mysqldump.sql
#/swadm/assetstore_stage/Backups/mysql/2009_0922_mysqldump.sql

##################
# Handling MySQL #
##################

# Making mySQL dumps
$starttime = `date`;
printf("mysqldump -h$mysql_server -u$mysql_user -p$mysql_pwd drupalstage > $dir_base/$file_dumpname\n");

`mysqldump -h$mysql_server -u$mysql_user -p$mysql_pwd drupalstage > $dir_base/$file_dumpname`;

#############################
# Chmod -- owner/group only #
#############################
`chmod 660 $dir_base/$file_dumpname`;
`gzip -fq $dir_base/$file_dumpname`;


##############
# Concluding #
##############


# The "done" message
printf("...complete.\n");
$finishtime = `date`;

####################
# Mail the results #
####################

$title='MySQL backup';
$to='your email';
$from= 'crontab@yourhost';
$subject='[cron] vm02 mySQLdump';

open(MAIL, "|/usr/sbin/sendmail -t");

## Mail Header
print MAIL "To: $to\n";
print MAIL "From: $from\n";
print MAIL "Subject: $subject\n\n";
## Mail Body
# $dumpsize = `du  -h $dir_base/${file_dumpname}.gz`;
#print MAIL "Start:\n$starttime\nFinish:\n$finishtime\nDumpsize:\n  $dumpsize";
print MAIL "Start:\n$starttime\nFinish:\n$finishtime\nFile $dir_base/${file_dumpname}.gz";
close(MAIL);

January 13, 2011

ABBY OCR System does not work well in a VM

I tried to run ABBY 3.0 in a VM. I used a 34 page pdf as input to ABBY:
MRC-72-3 A consumer test of canned, seasoned salad tomatoes.pdf I ran the microsoft performance monitor while using ABBY to produce the plot below.
FinalperformanceTest.gif
red line is \\DLS-OCR\Processor(_Total)\% Processor Time
green line is \\DLS-OCR\Memory\Page Faults/sec ( this maxed out at above 60K)


Also we have a csv version of the data This run was completed after several modifications of the VM were made to enhance performance. However even at this point, it still takes almost 2 seconds per page.
Because We will not be using a VM with ABBY.

December 29, 2010

Environment variables on windows 2008 box

How to find the Environment variable menu in windows 2008



Start



Administration Tools



System Configuration



tools



System Properties (Launch)



Advanced Settings



Environment Variables

December 8, 2010

SQL to find handles that correspond to a specific language.iso in DSPACE

select handle,metadatavalue.text_value from handle, item, metadatavalue where metadatavalue.item_id=handle.resource_id AND metadata_field_id = 38 AND handle.resource_type_id=2 AND text_value='French' AND handle.resource_id=item.item_id AND item.withdrawn='f';

September 3, 2010

Tables populated by Node Relationships Module

Tables populated by Relationships module .jpg

August 20, 2010

using Node Relationships module

Having a cck fields point at another data type.
1) Install the Node Relationships 6.x-1.6 module..
2) Create a new content type to point at (ContentTypeChild). .
3) Now in a content type "ContentTypeParent" create a new cck field (NewCCK). Have this be type: "Node Reference".
4) In the cck field, check the ContentTypeChild to point at..
5) Go to cck field -> configure -> Relationships -> Node reference extras.
Click:.
View reference in new window.
Edit reference.
Create and reference .
and set to "table style".
6) When you try to upload to ContentTypeParent the NewCCK will have some . icons next to it. These will allow you to fill in values for ContentTypeChild.

August 11, 2010

Instructions to allow harvesting of log files by URCHIN

Instructions to allow harvesting of log files by URCHIN
OIT referenced URCHIN page General plan: A log file called daily.log will be produced each day that is in a location where it be harvested through http by the OIT URCHIN service.
Steps:
I) Add lines to the MEDIA Archive httpd.conf and ssl.conf files that will create log files in the correct format. For these lines to run a UNIX utility, cronolog, may have to be installed.
II) Write a cron job that creates the daily.log file and transferres that file to a directory in the MediaStats domain.
III) Add an .htaccess file to the directory mentioned ion step II. This .htaccess file will allow the OIT Urchin appliance access to the daily.log file.
I)
1. Lines to add to Media Archive’s http.conf file:
########### Lines for Media Archive httpd.conf file ####################
CustomLog "|/usr/sbin/cronolog /etc/httpd/logs/www/%Y/%m/%d/access.log" "%h %l %u %t \"%r\" %>s %b"
ErrorLog "|/usr/sbin/cronolog /etc/httpd/logs/www/%Y/%m/%d/errors.log"
########### End Lines for Media Archive httpd.conf file ######### #######
2. ssl.conf
########### Lines for Media Archive ssl.confs file ####################
CustomLog "|/usr/sbin/cronolog /etc/httpd/logs/www/%Y/%m/%d/ssl_access.log" "%h %l %u %t \"%r\" %>s %b"
ErrorLog "|/usr/sbin/cronolog /etc/httpd/logs/www/%Y/%m/%d/ssl_errors.log"
########### End of lines for Media Archive ssl.confs file ####################
Comments:
1) If /usr/sbin/cronolog does not exist on your box, then the app must be installed see cronolog.org . Most likely this will involve someone at OIT.
2) The directory “/etc/httpd/logs/” is the root for the log files. If you change this it will effect the cron job in the next section.
3) It is not required that the error logs be created since these logs will not be transferred to URCHIN. They are merely created for a comparison to the *access* files.
4) URCHIN wants Common Log Format, for more info on this see: Common Log Format and Customlog


II) Write a cron job to create the daily.log file and move it to a location (MediaStats domain that you created) where OIT can access it.
########### Cron to create daily.log  ################
#!/bin/bash
# Path to directory that URCHIN will harvest the files from.
URL_PATH=" /var/www/html/chaucer/urchin_daily "
# Location where /usr/sbin/cronolog will create the daily log files
LOG_PATH="/etc/httpd/logs/www/"
YESTERDAY=$(date --date="yesterday" +"%Y/%m/%d")

echo "Removing existing logs from $URL_PATH"
[ -f $URL_PATH/daily.log ] && rm -f $URL_PATH/daily*.log*

if [ -d $LOG_PATH/$YESTERDAY ]

then
	echo "Looking for access logs in $LOG_PATH/$YESTERDAY/"
	for file in $LOG_PATH/$YESTERDAY/*access.log*
	do
		if [ -s $file ]
		then
			echo "cat $file to the Urchin log "
			cat $file >> $URL_PATH/daily.log
		fi
	done
fi
# Change perms so URCHIN app can get daily.log
[ -f $URL_PATH/daily.log ] && chmod 664 $URL_PATH/daily.log

########### end of Cron to create daily.log  ################
Comments:

1) If you made no changes in step I, then you will not have to change LOG_PATH. However if you made any changes to the path in step I, you must change LOG_PATH.
2) URL_PATH must be changed. It points to a directory that can be reached by the MediaStats Domain that you created. It is possible that this is the only variable that you will need to change.
3) This cron job should be run once a day, sometime in the early morning.
4) The user that runs the cron must have read privileges in LOG_PATH and write in URL_PATH.
III. Put an .htacess file in the directory defined by: URL_PATH, so the OIT URCHIN app can harvest daily.log files.

########### .htaccess file in the directory URL_PATH ################
RewriteEngine On
RewriteCond %{SERVER_PORT} 80
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}


order deny,allow
deny from all
allow from 128.101.65.14
allow from 128.101.29.84
allow from urchin.umn.edu
allow from mousetrap.software.umn.edu 

######### end of  .htaccess file in the directory URL_PATH ###############
Comments:
1) This will allow just OIT in
2) I will need to know what the URL is the maps to the directory: URL_PATH.

August 5, 2010

how to get rid of '\n' in an sql call

Select metadata_field_id , metadata_schema_id , element , qualifier , translate(scope_note, '\n',' ' ) From metadatafieldregistry ;

June 10, 2010

script to take a sql query and turn it into a tab delimited file

This will convert an sql query to a tab delimited file. I am sure there is a more elegant way to do this, but this works.
# Name sql2tab.sh
#
# dbFlavor - dspace_sr or dspace_ir
# sql_query - a query for the database
#
# Example:
#  ./sql2tab.sh dspace_ir 'select handle from handle;'
#
# Bug the '*' in an sql query produces an error because bash
# expands it as an las of the directory
#
# J. Silvis
# June 2010
#*******************************************************
flavor=$1
sql_query=$2

echo $sql_query > sql_temp_junk
# run the sql queryy and put it in tempData
psql -U $flavor $flavor  < sql_temp_junk  > tempData

# Replace the pipe (|) delimiter with tab
perl -p -i -e 's/\s+\|\s+/\t/g' tempData
#clear out the leading whitespace
perl -p -i -e 's/^(\s+)(.*)/\2/g' tempData
# dump the results out to be either directly viewed or sent to a pipe.
cat tempData
# get rid of junk files
rm tempData
rm sql_temp_junk


May 14, 2010

location of firefox's executable on mac OS X

/Applications/Firefox.app/Contents/MacOS/firefox-bin

find all cron files

for user in $(cut -f1 -d: /etc/passwd); do crontab -u $user -l; done
run it form root

February 18, 2010

Script to curl a list of files

#!/bin/bash if [ ! -d ./dayofpeace ] ; then mkdir ./dayofpeace fi while read inf do curl "http://URL/path/$inf" > "$inf" done << EOF dayofpeace/00612.tif dayofpeace/00613.tif dayofpeace/00614.tif dayofpeace/00615.tif

February 10, 2010

Very simple pojo example for an axis2 webservice

Introduction

Below are three critical files to writing a simple pojo for AXIS2. The files are listed just to get a feel for the system. At the bottom is a tarball with an ant build file in it. If you want to actually use this download the tarball and install axis2 under tomcat.

The webservice pojo

The user needs to create a pojo that will be made into an aar file. An example is: UMN_CMA_Service.java. This has a java method with two strings as arguments and returns a string.

The rpc client

Here is a client that will talk to the webservice after it has been deployed. UMN_CMA_ServiceRPCClient.html The client bundles up two strings and ships them to the webservice.

service.xml

There is also an xml config file. You must add a row to to this file for each method you create: services.html

Tarball for the client/webservice

Download this to run the example:
axis2simple_pojo_20100210_15_47.tar.gz

January 22, 2010

Got a Ruby CGI script up

Leaned this site

November 17, 2009

Problem getting to localhost website on my Mac after snow lepord

I got a 403 when trying to reach localhost: Solution from: techtrouts.com
Mac OS X 10.5: Web Sharing – “Forbidden 403″ on http://localhost/~username
If, usually after installing/upgrading your mac, you can’t access your ~/Sites folder on your browser
( http://localhost/~username , where username is your mac username):
* Open a Terminal and $ sudo nano /private/etc/apache2/httpd.conf ;
* ctrl+w (where is) : “<Directory />” ;
* Alter the line below from “Deny from all” to “Allow from all“;
* ctrl+x and y to save;
* now $ sudo httpd -k restart to restart your Apache 2 server;


Then I got the error:
httpd: Syntax error on line 459 of /private/etc/apache2/httpd.conf: Syntax error on line 15 of /private/etc/apache2/extra/httpd-userdir.conf: Syntax error on line 8 of /private/etc/apache2/users/+entropy-php.conf: Cannot load /usr/local/php5/libphp5.so into server: dlopen(/usr/local/php5/libphp5.so, 10): no suitable image found. Did find:\n\t/usr/local/php5/libphp5.so: no matching architecture in universal wrapper
Fixed this by changing :
LoadModule php5_module /usr/local/php5/libphp5.so
to
LoadModule php5_module libexec/apache2/libphp5.so
in
/private/etc/apache2/users/+entropy-php.conf

October 21, 2009

utf-8 utf-16 and UNIX

1) Convert utf-16 to utf-8.
iconv -f utf-16 -t utf-8 bell-map-IMAGESspreadsheet.xml > tt
2) Have less read utf-8
export LESSCHARSET='utf-8'
cat just works.

October 6, 2009

Some useful drupal sql

Find uid given user nae (in this case naa)
select uid from users where name='naa';
Get the number of uploads by user 'naa'
select count(upload.fid) from upload, node where node.nid=upload.nid and node.uid=(select uid from users where name='naa');

Get the number of files owned by user 'naa'
select files.filepath from files, upload where upload.fid=files.fid and upload.nid=68652;
Find path to uploaded files:
select files.filepath from files, upload where upload.fid=files.fid and upload.nid=68652;

September 17, 2009

David Naughton gave me some fast perl code for hashes

#!/exlibris/sfx_ver/sfx_version_3/app/perl-5.8.6/bin/per use strict; use warnings; # Takes export file from SFX and generates host definitions to append to # the ezproxy.cfg file. # Open SFX export file for manipulation my $sfx_url_file = shift @ARGV; open (my $fh_sfx, '<', $sfx_url_file) or die "File Open Failed: $!"; # get hostnames into a hash my %hostnames; while (<$fh_sfx>) { my $line = $_; next if $line =~ /^#/; my ($sfx_target, $sfx_url) = split /\t/, $line; if ($sfx_url =~ m/\:\/\/(.*?)\//) { # Hash keys are always unique, so if a key for $1 # already exists, this line will clobber its value: $hostnames{$1} = undef; # If you want to keep track of how many times each # hostname appears, you can use this magic: # $hostnames{$1}++; # More verbose version of the code above: # if (!(exists $hostnames{$1})) { # $hostnames{$1} = 0; # } # $hostnames{$1} = $hostnames{$1} + 1; } } # print each unique hash key, with some added text for my $hostname (keys %hostnames) { print "HJ $hostname\n"; } # close file handle close $fh_sfx or die "File Close Failed: $!";

July 27, 2009

svn notes ... media_repo Externals Switch

Our svn site

https://chaucer.lib.umn.edu/svn/dldl/

Externals

Chad is using svn externals to have several projects at the DLDL link to the same set of files within the svn tree. From the command line to view the externals use the commands: svn propedit svn:externals . svn propget svn:externals FileName

Switch

The svn switch command allows a user to update to a new version of a package.
Example from John Barneson: First make drupal go offline. https://dldl00.lib.umn.edu/admin/settings/site-maintenance svn switch https://chaucer.lib.umn.edu/svn/dldl/drupal/releases/acquia-drupal-1.2.13/ .

Useful URLs

Jeremy Knope dot com OS X svn client

May 8, 2009

Basic solr works

The two requests below will get a record from solr that has "hal" in the title. http://128.101.29.84:8080/solr/select/?q=hal&start=0&rows=10&version=2.2&indent=on
http://128.101.29.84:8080/solr/select/?q=hal

Using this solr schema and ustore code

March 13, 2009

truncated dates in ISO 8601

I have looked into the truncated date issue. From wikipedia article on ISO 8601 you find:
"The standard also allows for calendar dates to be written with reduced precision. For example, one may write "1981-04" to mean "1981 April", and one may simply write "1981" to refer to that year or "19" to refer to that century." So if we use the date "1981", we will be compliant to iso 8601 ( latest version : iso 8601:2004). The only issue raised was that the software developer must decide on a way to compare dates of different precision , i.e. is 1981 before or after 1981-02. I did not look into this very deeply but I could find no standard for this. I won't spend any time on this comparison issue right now. Here are a couple of other URLs
FAQ on ISO 8601 Allowed values for MODS attributes (including datetime) ISO 8601 examples

February 27, 2009

Description of OAI properties

OAICat - An OAI-PMH v2 Repository Framework
OAICat - An OAI-PMH v2 Repository Framework A Project of OCLC Research

Description of Entries in oaicat.properties

  • OAIHandler.extraXmlns=[xmlns attributes] (Optional): This string gets inserted into the root element in ListRecords and GetRecord responses. It was added to make large-scale harvesting of DSpace METS records more efficient.
  • OAIHandler.serviceUnavailable=[true|false] (Optional): Forces the server to return a SC_SERVICE_UNAVAILABLE code to inform the user that it is currently being worked on.
  • OAIHandler.styleSheet=[path-to-xsl-stylesheet] (Optional): If present, a reference to this XSLT stylesheet will be included in the OAI responses.
  • OAIHandler.renderForOldBrowsers=[true|false] (Optional): If present and true, the XSLT stylesheet will be applied to the OAI response on the the server before sending back the response, if the client is an old browser. This is a good idea if you've defined OAIHandler.styleSheet because some old browsers don't do a very good job of rendering XSLT on the client side.
  • OAIHandler.forceRender=[true|false] (Optional): If present and true, the XSLT stylesheet will be applied to the OAI response on the the server before sending back the response. This is useful if you want to use OAICat as a web application framework, but want to disable OAI harvesting because the data is proprietary (e.g. The GSAFD Thesaurus).
  • OAIHandler.baseURL=[OAI baseURL] (Optional): Force OAI responses to include the specified baseURL instead of getting it from the HttpServletRequest. This may be necessary if your firewall/router/port-mapper is messing with the request in some way.
  • AbstractCatalog.oaiCatalogClassName=[package.classname] (Required): The fully qualified Java class name that implements ORG.oclc.oai.server.catalog.AbstractCatalog.
  • AbstractCatalog.recordFactoryClassName=[package.classname] (Required): The fully qualified Java class name that implements ORG.oclc.oai.server.catalog.RecordFactory.
  • AbstractCatalog.secondsToLive=[# of seconds] (Optional): The number of seconds a resumptionToken is retained for reuse. The default is -1, which means that resumptionTokens are stateless and can be resent anytime.
  • AbstractCatalog.harvestable=[true|false] (Optional): If present and false, the ListRecords and ListIdentifiers verbs will send back an error condition. This might be useful if you want to provide access to the other verbs but want to discourage general harvesting.
  • AbstractCatalog.granularity=[YYYY-MM-DD|YYYY-MM-DDThh:mm:ssZ] (Required): The supported level of datetime granularity for from/until parameters.
  • Identify.repositoryName=[text] (Required): Text to use as the Identify verb's repositoryName value.
  • Identify.adminEmail[mailto:abc@xyz.org] (Required): Text to use as the Identify verb's adminEmail value.
  • Identify.earliestDatestamp=[datetime stamp of earliest] (Required): Text to use as the Identify verb's earliestDatestamp value.
  • Identify.deletedRecord=[no|transient|persistent] (Required): Text to use as the Identify verb's deletedRecord value.
  • Identify.description.[0-9]*=[xml fragment] (Optional; Repeatable): XML fragments (<description>) to include in the Identify verb's response.
  • Crosswalks.[metadataPrefix]=[package.classname] (Required; Repeatable): Java class(es) that implement the ORG.oclc.oai.server.crosswalk.Crosswalk class.
  • ExtensionVerbs.[verb name]=[package.classname] (Optional; Repeatable): Java class(es) that extend the ORG.oclc.oai.server.verb.ServerVerb class. This allow you to easily create new verbs to perform application functions beyond the set of OAI verbs. These verbs can be accessed by adding "/extension" to the baseURL.
  • Other properties that support various combinations of classes mentioned above.
OAICat - An OAI-PMH v2 Repository Framework    
Jeff Young
Last modified: Fri Jul 29 10:03:16 Eastern Daylight Time 2005

December 1, 2008

logrotate config files

Intro

Redhat uses the logrotate to manage the rotation of log files. The files end up in /var/log or a subdirectory of that directory.

Main logrotate config file ( /etc/logrotate.conf)

# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
}

Specific file for httpd (/etc/logrotate.d/httpd ) overrides file above

/var/log/httpd/*log {
# keep 52 weeks worth of backlogs  
    rotate 52
    missingok
    notifempty
    sharedscripts
    compress
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

November 26, 2008

Eliminating the host table from awstats

Within the directory that contains the config files enter the command: perl -i -pe 's/ShowHostsStats=PHBL/ShowHostsStats=0/g' *.conf
Location of on walter desktop:
/opt/awstats-6.8/wwwroot/cgi-bin/awstats.ageconsearch.umn.edu.conf

October 24, 2008

Simple Java code to dump an xml document using DOM

This zip file contains java code and an xml file to test it.
When it is run you will see: [silvi003~/xmlDom_play]$ javac DomEcho01.java [silvi003~/xmlDom_play]$ java DomEcho01 slideSample01.xml <xml version="1.0"> <slideshow author="Yours Truly" date="Date of publication" title="Sample Slide Show"> <slide type="all"> <title>Wake up to WonderWidgets!</title> </slide> <slide type="all"> <title>Overview</title> <item>Why <em>WonderWidgets</em> are great</item> <item></item> <item>Who <em>buys</em> WonderWidgets</item> </slide>

September 29, 2008

Regex to pull GET POST commands from apache logs

The regex: perl -p -i -e 's/^.*\s\"(GET|POST)(.*)\"\s.*\".*\".*\"$/\1\2/' Will extract:
GET /bitstream/34171/1/sp06pa01.pdf HTTP/1.1

from
99.239.52.187 - - [21/Sep/2008:09:38:24 -0500] "GET /bitstream/34171/1/sp06pa01.pdf HTTP/1.1" 200 16368 "http://www.google.ca/search?hl=en&q=positive+effects+on+canada+from+immigration&start=10&sa=N" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1"

September 11, 2008

Finding the x.500 fields in an ssl cert

The command below will reveal all the x.500 fields in tools.lib.umn.edu.crt

openssl x509 -noout -text -in tools.lib.umn.edu.crt
openssl rsa -noout -text -in server.key
openssl req -noout -text -in server.csr
openssl rsa -noout -text -in ca.key
openssl x509 -noout -text -in ca.crt

August 22, 2008

how to strip a passphrase from an openssl key


cp filename.key filename.key.encrypted
openssl rsa -in filename.key -out filename.key.nopass
cp filename.key.nopass filename.key

February 15, 2008

Ports that must be set in server.xml so that more than one instance of Tomcat runs on a box

The following ports must changed if a new instance of tomcat is going to be set up on the same box:



Excepts from server.xml (Ports that must be changed)

<!-- A "Server" is a singleton element that represents the entire JVM, which may contain one or more "Service" instances. The Server listens for a shutdown command on the indicated port. Note: A "Server" is not itself a "Container", so you may not define subcomponents such as "Valves" or "Loggers" at this level. --> <Server port="9035" shutdown="SHUTDOWN">

<!-- Define a non-SSL HTTP/1.1 Connector on port 8080 --> <Connector port="9030" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" redirectPort="8043" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true" URIEncoding="UTF-8" />

<!-- Define a SSL HTTP/1.1 Connector on port 8443 --> <Connector port="9031" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="25" maxSpareThreads="75" enableLookups="false" disableUploadTimeout="true" acceptCount="100" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" />

<!-- Define an AJP 1.3 Connector on port 8009 --> <!-- <Connector port="9039" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> -->

January 7, 2008

rsync and expect

Using expect to rsync the directory /Users/silvi003/Documents/workspace/dspace-sr
to
silvi003@odin.lib.umn.edu:/home/silvi003/src
over ssh using the password $ThePassWord



#!/usr/bin/expect spawn rsync -a --delete -e ssh /Users/silvi003/Documents/workspace/dspace-sr silvi003@odin.lib.umn.edu:/home/silvi003/src expect "password:" send "$ThePassWord\r" expect eof

November 21, 2007

bash functions to run more than one instance of tomcat on a box

# The functions below are used to manage a box where two or more instances # of tomcat are running simultaneously. # # Ports used by Tomcat (see $TOMCAT_BASE/conf/server.xml): # 1) <Connector port="8080" maxHttpHeaderSize="8192" # maxThreads="150" minSpareThreads="25" maxSpareThreads="75" # enableLookups="false" redirectPort="8443" acceptCount="100" # connectionTimeout="20000" disableUploadTimeout="true" /> # 2) <Server port="8005" shutdown="SHUTDOWN"> # # 3) <!-- Define an AJP 1.3 Connector on port 8009 --> # <Connector port="8009" # enableLookups="false" redirectPort="8443" protocol="AJP/1.3" /> # # The first two ports need to be modified to set up a new instance of tomcat. # the third port is for a ajp server that talks to aphache and it does not need to be # changed. # http://tomcat.apache.org/tomcat-4.0-doc/config/ajp.html # # # # I) Creating the new set of code. # # # Suppose you want an instance of tomcat to run on port 8070 # and the primary set of tomcat files are in # TOMCAT_BASE=/usr/local/apache-tomcat-5.5.23 # # 1) Create a duplicate set of the tomcat files # cp -R /usr/local/apache-tomcat-5.5.23 /usr/local/apache-tomcat-5.5.23_8070 # # 2) Modify the file /usr/local/apache-tomcat-5.5.23_8070/conf/server.xml # a) set <Connector port="8080"... to <Connector port="8070" # b) <Server port="8005" shutdown="SHUTDOWN"> to <Server port="8006" shutdown="SHUTDOWN"> # or whatever port you like for shutdown. # # # II) Examples of using the scripts # tomstart - will start an instance of tomcat on 8080 # tomstart 8070 - will start an instance of tomcat on 8070 # tomkill - will kill -9 an instance of tomcat on 8080 # tomskill 8070 - will kill -9 an instance of tomcat on 8070 # ect. # # III) I have the line # source ~/TOMCAT_SCRIPTS.sh # in my .bashrc file where TOMCAT_SCRIPTS.sh is this file # # # J. Silvis Nov 2007 # ********************************************************************************************** export TOMCAT_BASE=/usr/local/apache-tomcat-5.5.23 function tom { if [ -n "$1" ]; then cd ${TOMCAT_BASE}_${1} else cd $TOMCAT_BASE fi } # Start tomcat server: # tomstart portnumber will start a server that is is # within the directory $TOMCAT_BASE_$portnumber and will be # on the port: portnumber Otherwise stop the server on 8080 function tomstart { export CATALINA_HOME=$TOMCAT_BASE port=8080 if [ -n "$1" ]; then port=${1} export CATALINA_HOME=${CATALINA_HOME}_$port fi export CATALINA_PID=${CATALINA_HOME}/temp/tomcat.pid ${CATALINA_HOME}/bin/catalina.sh run & sleep 4 # wait for CATALINA_PID file to be written termtitle "Tomcat on port $port pid `cat $CATALINA_PID`" } # Stop tomcat server: # tomstop portnumber will start a server that is is # within the directory $TOMCAT_BASE_$portnumber and will be # on the port: portnumber Otherwise stop the server on 8080 function tomstop { export CATALINA_HOME=$TOMCAT_BASE if [ -n "$1" ]; then export CATALINA_HOME=${CATALINA_HOME}_${1} fi $CATALINA_HOME/bin/shutdown.sh termtitle "$USER" } # kill -9 the tomcat server: # tomkill portnumber will start a server that is is # within the directory $TOMCAT_BASE_$portnumber and will be # on the port: portnumber Otherwise stop the server on 8080 function tomkill { export CATALINA_HOME=$TOMCAT_BASE if [ -n "$1" ]; then export CATALINA_HOME=${CATALINA_HOME}_${1} fi export CATALINA_PID=${CATALINA_HOME}/temp/tomcat.pid kill -9 `cat $CATALINA_PID` rm $CATALINA_PID termtitle "$USER" } # gives the pid of the tomcat server: # tomkill portnumber will start a server that is is # within the directory $TOMCAT_BASE_$portnumber and will be # on the port: portnumber Otherwise stop the server on 8080 function tompid { export CATALINA_HOME=$TOMCAT_BASE if [ -n "$1" ]; then export CATALINA_HOME=${CATALINA_HOME}_${1} fi export CATALINA_PID=${CATALINA_HOME}/temp/tomcat.pid cat $CATALINA_PID } # Go to the root of the dspace development tree function dsrt { cd ~/Documents/workspace/trunk_8070 } # Copy dspace war files into tomcat function tomds { export CATALINA_HOME=$TOMCAT_BASE dsCodeRoot=~/Documents/workspace/trunk if [ -n "$1" ]; then export CATALINA_HOME=${CATALINA_HOME}_${1} dsCodeRoot=${dsCodeRoot}_${1} cp ${dsCodeRoot}/build/*war ${CATALINA_HOME}/webapps/. else echo -n "You are putting war files into the 8080 server ... proceed (Y/N) ? " read PROCEED ymatch=`echo $PROCEED | grep -Ei '^y(es)?$' | wc -l` if [ $ymatch == "1" ] ; then cp ${dsCodeRoot}/build/*war ${CATALINA_HOME}/webapps/. fi fi }

November 7, 2007

Add a user to a group

The line below adds the user silvi003 to the group dspace:

sudo dscl . merge /groups/dspace users silvi003

Make a Lucene field sortable and searchable

The java code snippet below will add searchable (TOKENIZED) fields and
sortable (UN_TOKENIZED) fields to a Lucene document.


if (value != null)
{
doc.add(new Field(key, value, Field.Store.YES, Field.Index.TOKENIZED));
if (key.equalsIgnoreCase("author") || key.equalsIgnoreCase("title") || key.equalsIgnoreCase("date") )
{
doc.add(new Field(key + "_sortable", value, Field.Store.YES, Field.Index.UN_TOKENIZED));

}
}


This is from the ag econ project file:

./src/org/dspace/search/DSIndexer.java

October 1, 2007

Things needed to set up dspace

- download eclipse
- eclipse svs plugin subclipse
- Tomcat
- postgress.jar
- config dspace.cfg files
- also see dspace.org

Continue reading "Things needed to set up dspace" »