The issue

Location of code

The bug

$new_url = 'http'

<?php                   
   $url = 'https://ddd';
    if (strpos($url, 'http://') == 0) {
        $new_url = 'http';
    }
    elseif (strpos($url, 'https://') == 0) {
      $new_url = 'https';
    }
    else {
        drupal_set_message(t('Invalid URL: !url', array('!url' => $url)));
        return NULL;
    }
   echo $new_url . "\n";
?>

<?php                   
   $url = 'https://ddd';

    $url_start = substr ($url , 0, 5 );

    if ($url_start == 'http:') {
        $new_url = 'http';
    }
    elseif ($url_start == 'https') {
      $new_url = 'https';
    }
    else {
        drupal_set_message(t('Invalid URL: !url', array('!url' => $url)));
        return NULL;
    }
   echo $new_url . "\n";
?>

Posted by Jeffrey Silvis at 11:51 AM | Permalink | Comments (0)

Some issues with bad characters

http://128.101.146.59:8080/solr/select/?version=1.2&start=0&rows=10&indent=on&wt=standard&q=fullText:food&sort=modstitle desc&qt=mods

Should be:

http://128.101.146.59:8080/solr/select/?version=1.2&start=0&rows=10&indent=on&wt=standard&q=fullText:food++%26sort%3Dmodstitle%2Bdesc&qt=mods

Fields that Solr searches on should be single valued
Bill found out that fields that are being used for sort must be single values.  Our title field is not.  In the past SOLR would handle multivalued fields for sort.

The MODS schema

	<titleInfo>
 	 	<title>[dc.title]</title>
 	</titleInfo>
<titleInfo type="translated">
 	 	<title>[dc.title.alternative]</title>
 	</titleInfo>


The XSLT to make SOLR indices
Note the XSLT will not distinguish between different values of the type attribute, so there is more than version of title, it will no longer be single valued.

/Users/birage/fedora/tomcat/webapps/fedoragsearch/WEB-INF/classes/config/index/GSearch_solr/demoFoxmlToSolr.xslt  

    <xsl:for-each select="foxml:datastream[@ID='MODS']/foxml:datastreamVersion[last()]/foxml:xmlContent//mods:titleInfo/mods:title">
         <xsl:if test="text() [normalize-space(.) ]"><!--don't bother with empty space-->
            <field>./WEB-INF/classes/config/index/GSearch_solr/demoFoxmlToSolr.xslt
               <xsl:attribute name="name">
                  <xsl:value-of select="concat('mods.', 'title')"/>
               </xsl:attribute>
               <xsl:value-of select="text()"/>
            </field>
         </xsl:if>
      </xsl:for-each>


Possible values for the type attribute of title element in MODS

From:  
http://www.loc.gov/standards/mods/v3/mods-userguide-elements.html  


		type - This attribute is applied when it is necessary to identify what type of title is recorded. 

        For the main title (MARC 21 field 245), no type is indicated. The following values may be used with the type attribute:

		abbreviated (equivalent to MARC 21 field 210)

		translated (equivalent to MARC 21 field 242, 246)

		alternative (equivalent to MARC 21 fields 246, 740)

		uniform (equivalent to MARC 21 fields 130, 240, 730)

                              
                              

                                 Posted by Jeffrey Silvis at  4:56 PM | Permalink
                                 | Comments (0)
                                 
                              
                           

The bug

Unable to connect to Solr server 

Islandora code connected to the problem

/**
 *
 * @param String $solr_url
 * @return boolean
 *
 * Checks availability of Solr installation
 *
 */
function solr_available($solr_url) {
  // path from url is parsed to allow graceful inclusion or exclusion of 'http://'
  $pathParts = parse_url($solr_url); 
  $path = 'http://' . $pathParts['host'] . ':' . $pathParts['port'] . $pathParts['path'] . '/admin/file';
  $test = @fopen($path, "r");
  if ($test) {
    return true;
  }
  return false;
}
    

The fix (upgrade SOLR)

"/admin/file"

Posted by Jeffrey Silvis at 12:03 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 5:16 PM | Permalink | Comments (0)

List of datastreams in a Fedora Object

curl example

browser example

MODs Datastream

curl example

browser example

RDF Datastream

curl example

browser example

Get all triples

curl example

browser example

All triples with 36284 handle as a predicate

curl example

browser example

All triples with 36284 handle as a subject

curl example

browser example

Determine if 36284 is a collection

curl example

Find the title of 36284

Posted by Jeffrey Silvis at 4:40 PM | Permalink | Comments (1)

Summary

Biraj pulled some xml directly from his repository but it could not be ingested into the new Fedora repository on https://umetadata-stage.lib.umn.edu:8443/fedora/admin/. However I could ingest into umetadata-stage the examples that the Fedora repository people gave us.

Original file that would not load and modified file that does load

Changes

Posted by Jeffrey Silvis at 10:06 AM | Permalink | Comments (0)

# Clean up the databases nightly
20 0 * * * vacuumdb -U dspace_ir --analyze dspace_ir > /dev/null 2>&1
40 0 * * * vacuumdb -U dspace_sr --analyze dspace_sr > /dev/null 2>&1

# Backup the databases nightly
2 1 * * * /var/lib/pgsql/backup.sh

# MySQL backup
5 0 * * * /opt/mysql/bin/backup.sh

Posted by Jeffrey Silvis at 2:01 PM | Permalink | Comments (0)

techConf2011-roySilvis.ziptechConf2011-roySilvis.ziptechConf2011-roySilvis.zip

Posted by Jeffrey Silvis at 11:20 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 11:02 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 12:57 PM | Permalink | Comments (0)

public void insertSequenceData(String NewXMl, String PID, String LogMessage) throws Exception {
  byte[] normalarr = NewXMl.getBytes("UTF-8");
  String[] altIds = new String[1];
  altIds[0] = "";
  // Use an axis client to call the Fedora webserver
  Service service = new Service();
  Call call = (Call) service.createCall();
  call.setOperationName(new QName(APIM_NS, "modifyDatastreamByValue") );
  call.setTargetEndpointAddress( new URL(URL_API_M));
  call.setUsername(fedoraUser);
  call.setPassword(fedoraPswd);
  // if the WRAPPED stlye is not used you get the evil error:
  // SAXException: Found character data inside an array element while deserializing
  // see 
  // http://www.opensubscriber.com/message/axis-user@ws.apache.org/1855611.html
  // for the solution.
  call.setOperationStyle(org.apache.axis.constants.Style.WRAPPED);
  Object[] obj_arr = new Object[] {
    PID, // The PID of the object.
    "STRUCT",        // The datastream ID.
    altIds, // Alternate identifiers for the datastream, if any.
    "METS StructMap for this object", //  The label for the datastream.
    "text/xml", //  The mime type.
    "",  //  Optional format URI of the datastream.
    normalarr,  //  The content of the datastream.
    null, //  The algorithm used to compute the checksum. One of "DEFAULT", "DISABLED", "MD5", "SHA-1", "SHA-256", "SHA-385", "SHA-512".
    null,  //  The value of the checksum represented as a hexadecimal string.
    LogMessage, //  A log message.
    false // Force the update even if it would break a data contract.
    };
  call.invoke( obj_arr); 
}

Posted by Jeffrey Silvis at 3:12 PM | Permalink | Comments (0)

 
import org.apache.axis.client.Service;
import org.apache.axis.client.Call;
import javax.xml.namespace.QName;
import java.net.*;

public class Axis2ClientToFedora
{

 public static void main(String[] argv){
  try{
    Service service = new Service();
    Call call = (Call) service.createCall();
    call.setOperationName(new QName("http://www.fedora.info/definitions/1/0/api/", "purgeObject") );
    call.setTargetEndpointAddress( new URL("http://chaucer.lib.umn.edu:8080/fedora/services/management") );
    call.setUsername("FedoraUserName");
    call.setPassword("FedoraPassword");
    Object[] obj_arr = new Object[] {
        "basic:data",
        "purge basic:data",
        false
        };
    call.invoke( obj_arr);
  }
      catch ( Exception e ){
         e.printStackTrace();
  }
  return;
}
}

Posted by Jeffrey Silvis at 2:20 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 10:52 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 4:59 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 11:24 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 1:41 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 11:21 AM | Permalink | Comments (0)

I. ESciDoc Content Models

II. TOC Description

III. Example of TOC

III. Used by VIRR

IV. escidoc and CMAs

Posted by Jeffrey Silvis at 11:15 AM | Permalink | Comments (0)

Instructions from tomcat install

Lines added to /etc/profile

Posted by Jeffrey Silvis at 11:24 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 3:01 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 10:40 AM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 12:21 PM | Permalink | Comments (0)

The problem

Jason's solution

What to do from here

Posted by Jeffrey Silvis at 2:49 PM | Permalink | Comments (0)

a. Managed Content (M): Datastream content is stored and managed within the Fedora repository’s persistent storage. The content can be any MIME type including XML. b. Inline XML (X): A special case of M, restricted to well-formed XML. In this case the datastream content is stored as part of the XML structure of the digital object itself and is thus included when the digital object is exported (e.g., for archival purposes). c. Externally Referenced (E): Datastream content is external to the Fedora repository and is referenced by a URL that is recorded within the digital object. The content can be any MIME type including XML. d. Redirected Content (R): Like E, but datastream content is delivered to the client without any mediation by Fedora; i.e., via an HTTP redirect. You should use this datastream type when the external content is a web page with relative links or it is streaming audio or video. The content can be any MIME type including XML.

Posted by Jeffrey Silvis at 5:53 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 3:04 PM | Permalink | Comments (0)

Overall Plan

Posted by Jeffrey Silvis at 4:36 PM | Permalink | Comments (0)

Version of ingestFormat that fails in Fedora 3.0

 
import fedora.server.types.gen.RepositoryInfo;
import java.io.*;

public class FedoraIngest {

    private static final String protocol = "http";
    private static final String host = "localhost";
    private static final int port = 8080;
    private static final String usr = "fedoraAdmin";
    private static final String pwd = "pass";

    private static final String collection = "swhp";
//    private static final String foxmlSrc = "/Users/bill/projects/fedora/" 
//                                         + collection + "/";

    private static final String foxmlSrc = "/Users/silvi003/Desktop/bill" 
                                         + collection + "/";


    public static void main(String[] argv) throws Exception {

	String[] dir = new java.io.File(foxmlSrc).list(new FOXMLFilter());
	FedoraSOAPClient caller = new FedoraSOAPClient(protocol, host, port, usr, pwd);
	// test client connection status with the most basic call...
	for (int i = 0; i< dir.length; i++) {
	    String pid = dir[i];
	    System.out.println("FedoraIngest " + pid);
	    try {
		FileInputStream fis = null;
		String fedoraPid = null;
		File foxml = new File(foxmlSrc + pid);
		fis = new FileInputStream(foxml);
		fedoraPid = caller.ingest(fis, "foxml1.0", "ingest of " + pid);
		System.out.println("new fedora object: " + fedoraPid);
	    } 
	    catch (Exception excp) {
		System.out.println("ingest error: " + excp.getMessage());
		excp.printStackTrace();
	    }
	}
    }
}

Version of ingestFormat that works in Fedora 3.0

 
import fedora.server.types.gen.RepositoryInfo;
import java.io.*;

public class FedoraIngestOneFile {

    private static final String protocol = "http";
    private static final String host = "localhost";
    private static final int port = 8080;
    private static final String usr = "fedoraAdmin";
    private static final String pwd = "pass";

    public static void main(String[] argv) throws Exception {

	FedoraSOAPClient caller = new FedoraSOAPClient(protocol, host, port, usr, pwd);
	String FileName = "/Users/silvi003/Desktop/umndob_msp01688";
	    try {
		File foxml = new File(FileName);
		FileInputStream fis = new FileInputStream(foxml);
		String fedoraPid = caller.ingest(fis, "info:fedora/fedora-system:FOXML-1.1", "ingest of " + FileName);
		System.out.println("new fedora object: " + fedoraPid);
	    } 
	    catch (Exception excp) {
		System.out.println("ingest error: " + excp.getMessage());
		excp.printStackTrace();
	    }
    }
}

Posted by Jeffrey Silvis at 2:08 PM | Permalink | Comments (0)

MTOM: way of sending binary in soap

Possible PHP library for MTOM

WSO2 WSF/PHP features proven interoperability with Microsoft .NET, WSO2 WSAS (Apache Axis2/Java based Web services application server) and other J2EE implementations. The basic SOAP level interoperability as well as WS-* specification implementations have been tested and proven to interoperate.

Downloading a Binary File from a Web Service using Axis2 and SOAP with Attachments

Posted by Jeffrey Silvis at 11:21 AM | Permalink | Comments (0)

No Problem with server

Issues with the client

Useful Links

Posted by Jeffrey Silvis at 4:32 PM | Permalink | Comments (0)

Location of my svn space on chaucer:

/var/svn/projects/jeff

web

svn:chaucer

Posted by Jeffrey Silvis at 4:27 PM | Permalink | Comments (0)

Summary

Qmaster and clustering

Compressor used by Qmaster

Codec and clustering

Posted by Jeffrey Silvis at 1:47 PM | Permalink | Comments (0)

Posted by Jeffrey Silvis at 12:08 PM | Permalink | Comments (0)