Sundry Miscellanea

Programming with Generators

Thu, 03 Apr 2008 13:54:50 -0600

I gave a talk about programming with generators in the remarkable programming language Icon courtesy of the local ACM chapter. Here are the small programs I used to illustrate Icon's expression evaluation mechanism. Here is the flyer! As you can see at the bottom of the flyer, I have written a dynamic 'glue' library for the Icon virtual machine in C++. This library makes writing dynamic libraries for Icon in C++ very simple. The current documentation for the glue is here. which I am currently writing it as of 2008/4/1. The glue is complete and being tested and cleaned up as I do this.

=================

Title: Paradigm Lost?
Subtitle: Programming without Booleans: using Generators in Icon
Author: Carl Sturtivant, Dept. of CS & E, U of M
When & Where: 5:30pm Thursday April 17, EE/CSci 3-115

Abstract

The otherwise conventional programming language Icon is designed around a powerful control mechanism called generators. Instead of testing whether something can be done, and if so attempting to do it, generators enable a much more natural programming style: simply attempt to do it, and the success or failure of the attempt will automatically be propagated to the surrounding context. There may be more than one way to succeed, and if the first way found leads to the failure of something else, then control can automatically backtrack to get a second way to succeed and then go forward with a second attempt to do the something else, and so forth. For this reason expressions in Icon may implicitly produce a sequence of results: an empty result sequence is interpreted as failure, and a non-empty one as success.

A small, mature and powerful language, Icon is implemented through a 300K virtual machine, iconx, implemented in C. The source has been placed overtly in the public domain. An Icon program is compiled into an icode file with an executable header, that when run will automatically invoke iconx. Benchmarks indicate that Icon runs at a similar speed to Perl or Python, and about twice as fast as Ruby. Icon was developed at the University of Arizona over a period of two decades, and the implementation is both robust and available for a wide variety of platforms. Designed for the purpose of manipulating text effectively, Icon has sufficient expressive power that regular expressions are unecessary. In this talk I will demonstrate programming with generators in Icon with applications, as well as giving a survey of the features of the language.

Those who wish to try out some of the ideas in his talk may pick up Icon 9.4.3 The Icon 9.5x binary distribution is currently only available for cygwin, macintosh and linux, though source is available with builds for many platforms. MSWindows users will need to install cygwin and will need to get Icon 9.5x which has vital bug fixes.

Icon related bio: Dr. Sturtivant has built an experimental dynamic 'glue' library for Icon in C++ that enables additions to Icon to be written cleanly in C++. Such an addition can then be compiled into its own dynamic library and loaded by an Icon program at runtime, without any modification to iconx or the icode compiler. The glue library enables further C++ to routinely do esoteric things such as call an Icon procedure, or when called by Icon act as an Icon generator that suspends a sequence of results. He also inspired and co-wrote the new external value datatype for Icon, that will be a part of the standard distribution of version 9.5.

Innumeracy. Why the blog is it tolerated?!

Fri, 05 Jan 2007 10:10:04 -0600

If you don't have a little mathematics or coding up your sleeve or more conveniently placed in your head, then you won't like the overhead necessary to understand this blog. Sorry --- for some reason liberal arts ignorance (illiteracy being the ultimate form) is socially and culturally unacceptable, but scientific ignorance (of which the inability to read and understand simple mathematics or code is the ultimate form) is considered the norm. The latter is called innumeracy by the very literate, and given that the root of this word gives a completely misleading impression that it's all about numbers, I am convinced that it was coined by an innumerate. So I say sorry to apologize for your ignorance for you, if you're one of those people, since for cultural reasons you're unlikely to endeavor to fix the problem for yourself, or perhaps even to realize there is a problem. Don't take this personally --- the educational apparatus around the world strongly supports this strange situation, especially primary and secondary education.

Mathematics is an international language evolved to talk of the world with far greater intellectual economy than the best so-called natural languages. The syntax and semantics of mathematics have been much more consciously honed than those of natural languages. A person who does not speak at least a basic core of mathematics is ignorant indeed, unable to grasp simple attributes of artifacts or nature, or communicate without immense difficulty about such. It's like going through life with a paper bag over your head. And just like any other language it's best learned when you're young. Eradicating illiteracy in adults is difficult. Eradicating innumeracy in adults is likewise difficult. So a better path would be to eliminate it at source during primary and secondary education. And given that innumeracy is the default, we have a catch-22 roadblock to doing that: innumerate parents, teachers and peers.

If you do have some active calculus and some mathematical maturity, then you speak a second language and I hope you proclaim "I speak mathematics", and use it wherever possible with others. Like any minority language only use and a cultural perception that speaking it is important will have an effect on its spread. Add your vote to the widespread occurrence of mathematical speech by speaking mathematics. It occupies a unique position among natural languages, making it the most important second language.

Add so as to multiply (part 2)

Fri, 05 Jan 2007 00:40:19 -0600

Add so as to multiply looked at differentiable solutions to the functional equation

and showed that they were all of the form

i.e. they are all logarithms in some base. But now what if we drop differentiability and just look for a function of the positive real numbers, and proceed from first principles? (We stick to positive numbers for now because g has a problem at 0. If we set y to 0 in the functional equation, we get

so if g ever takes on a non-zero value we have a contradiction, unless g is undefined at 0.) We pursue an exploratory argument that could be made a lot more compact, but I won't do that for exploration's sake. Perhaps I will write a very compact version as a later blog article.

First consider the functional equation when we set x and y to 1.

So

Now what if we set y to be x to the power n-1 where n is a positive integer. Then

so we may decrement the exponent of x by adding g(x). Decrementing it by n gives

Note that this also works when n is 0 because g(1) is 0. Call this the natural exponent rule. So if x is a positive natural number and we factor it into a product of distinct prime powers

then we may use the original functional equation and the natural exponent rule to find that

So the value of g at each positive natural number x depends upon its values at all of the prime numbers. In fact as we will see, we may define g(2), g(3), g(5), g(7), g(11) and so forth quite arbitrarily. So unless we impose some constraint upon g, it can be quite nasty. However, we will now show that if we insist that g is a continuous function, then fixing one value of g fixes all others ... ok, back to the details. Suppose we set y to be the reciprocal of x in the original functional equation. This gives

and since g(1) is zero we have

Call the last rule the quotient rule. So now we have the rule

for any integer a. Now what about rational exponents? Let's start with an exponent of 1/n, where n is a natural number.

So now we know that the rule

works when a is the reciprocal of a positive natural number. What about any positive rational exponent, m/n ? Applying the rules we know so far, we get

So now we know that the rule

works for any positive rational exponent. What about any negative rational exponent -m/n ?

Putting all these together we know that the rule

applies whenever a is rational. Call this the rational exponent rule, and note that there's nothing here forcing x to be rational, just positive by our current convention.

What about g(2), g(3), g(5), g(7), g(11) and so forth? Roughly speaking these are independent in general, because the primes 2,3,5,7,11 etcetera are not mutually related by rational exponents, only by real ones. And unless g is required to be continuous, we do not know that the rule

applies for all real exponents. If we knew this, then we could conclude for example that

where (lg denotes the base 2 logarithm as usual) and so only one g value would be arbitrary, say g(2) and then the rules would determine g on any rational argument (you might like to check this). So suppose we fix g(2) and assert that g must be continuous. Let p be an even prime (i.e. not 2). We know that

but we need to approximate this arbitrarily closely with rational exponents, so we can use the rational exponent rule. Here's how we may play that game! Let n be an arbitrarily large natural number, then

Notice that the two exponents have numerators that differ by 1, as we can definitely use < because the log of p is irrational. But the denominators of the exponents are both n, so the error in each exponent is at most 1/n. So by choosing sufficiently large n we may approximate the genuine but irrational exponent as closely as we please. That is to say

If g is continuous (which informally means approximable i.e. g of an approximation to p is approximately g of p, and this can be made as accurate as you like by choosing better and better approximations of p), then applying g to both sides gives

where continuity allows us to move the limit. (Notice how this is just a limit of approximations as in the parenthesised statement above.) Now using rational exponent rule for g gives

and as we take the limit the floor makes a vanishing difference, so

So we may conclude that

if g is continuous, for any odd prime p. So now if x is any natural number, g(x) is determined from its factorization as before

but now we know that for each i

and so

Now the quotient rule gives g for x any positive rational number m/n because

and because g is continuous, this defines g on all positive real numbers as well (by rational approximation in the spirit of continuity as exemplified above). So we conclude that the functional equation has continuous solutions on the positive reals only of the form

i.e. all solutions are differentiable, and there are no more than before.

What happens for negative x ? Well if we set both x and y to -1 in the original functional equation we get

So if we just set y to -1 in the original functional equation we get

So the final conclusion is

Finding the right one

Thu, 28 Dec 2006 08:14:15 -0600

Here's my way to find the position in a 32 bit word of the rightmost 1 bit. Back in late '96 I used this for certain purposes, other portable methods being too slow. The technique works equally well for 64 bit or larger word sizes. It uses the well known trick that given a word x, we can compute a word containing a single 1 bit in the position of the rightmost 1 bit of x by bitwise anding x with its arithmetic negation. Here's why: if I add a word to its bitwise negation, there are no carries and I get a word of all ones, which is -1 in standard two's complement representation of signed words. Why is this word of all ones the representation of -1? Well just imagine adding 1 onto it --- carries ripple all the way through the word and fall off the end, leaving a word of all zeros, which is 0. Just as 999 behaves like -1 on a three digit decimal counter.

Now the arithmetic negation of a word, on the other hand, obeys this obvious rule.

Comparing the two, we see the relation between the two kinds of negation.

So here's what happens when you add one onto the bitwise negation to get the arithmetic negation, followed by bitwise anding that with the original word, illustrated by an example.

A non-zero word consists of an arbitrary sequence of bits followed by the last 1 bit, followed by all zeros. (In the example there are two such zeros.) So its bitwise negation consists of the negation of the arbitrary sequence of bits followed by a 0 bit followed by all ones. Note that if we anded the logical negation with the original word we would get zero, because in each bit position we would be anding together a 0 bit with a 1 bit giving a 0 bit. However, now we add 1 onto the bitwise negation first, to get the arithmetic negation. This has the effect of causing a carry to ripple through the sequence of ones at the right hand end, putting a 1 bit into the position of the 0 bit to the left of that sequence, which is in the position of the rightmost 1 bit in the original word. So the arithmetic negation consists of the negation of the arbitrary sequence of bits followed by a 1 bit followed by all zeros. Clearly anding it with the original word causes the arbitrary leading bits to be anded with their negations, thus making all upper bits of the result be 0, and the zeros on the right hand end lead to zeros in the result. So the only bit position that is 1 in both is that of the last 1 in the original pattern. So we may conclude that

where k is the position of the rightmost 1 bit in the word x, with position zero defined to be the rightmost position.

Now to the real problem --- how do we find k given this? We could keep shifting it to the right until we get 1, but that looks clumsy and inefficient. Instead we use some mathematics! The integers modulo a prime number p form a finite field, and the multiplicative group in any finite field is cyclic. Here is a concrete example where p=7 of what this all means. First the multiplicative group of integers modulo 7 is just the multiplication table of the possible non-zero remainders when dividing by 7. The term "modulo 7" just means "ignoring multiples of 7", so everything is a remainder on dividing by 7. We throw out zero because it behaves badly with multiplication. The result is as follows.

This is the "multiplicative group of the integers modulo 7", and it is cyclic, which means that there is an element that generates all of the others. In this case 3 does the job, as follows.

The powers of 3 cycle through all values. This does not work for 2 in this case. We say that 3 is a generator and two is not, for modulo 7 multiplication.

Now for my little trick: if we choose a prime p bigger than the word size (32 bits say) where 2 is a generator, then modulo p the different powers of 2 that appear when we and a word with its arithmetic negation will all be different modulo p. 37 is such a prime (for 32 bit words). So the value of the following expression has 32 distinct values, all less than 37, for each different position of the last 1 bit. Here "mod 37" just means "take the remainder upon dividing by 37", sometimes written "% 37" in some programming languages.

So if we create a constant array with a maximum index of 36 that contains a table as follows, we may use it for table lookup of the position of the rightmost 1 bit, given the value of the previous expression. The array should contain the following values, which are discrete logarithms modulo 37.

Note that it is the fact that values are now small (less than 37 in fact) that makes the array practicable. Now we may write the final expression for the position of the rightmost 1 using this array.

If the word size is 64 or 128 or 256 bits, we may use the same method mutatis mutandis with the respective primes 67, 131 and 269.

Add so as to multiply

Fri, 15 Dec 2006 15:22:33 -0600

In a discussion of entropy it may be pointed out that when physical systems are paired the total number of microstates multiplies, since a microstate of the composite system is a pair of states of the separate systems. And yet the entropy, (which is a measure our ignorance of the exact state of a system) adds under these conditions. Boltzmann took this to mean that the entropy is proportional to the logarithm of the number of microstates, a famous result that endures today. This is because any logarithm satisfies the following functional equation, mapping multiplication of numbers to the addition of their logarithms.

But now we may ask which functions g actually satisfy this equation! If g is differentiable, then we may proceed as follows. First partial differentiate with respect to y.

Now set y to 1, and divide by x.

Look familiar? Integrating both sides gives a logarithm.

And we may write k for the coefficient of the logarithm, and note that if this is to satisfy the original functional equation, the constant of integration C must be zero. So the general differentiable solution is

But what if we care only about continuous functions: are there any more solutions? More on this soon!

Alternating three ways without branching

Thu, 14 Dec 2006 09:48:24 -0600

Suppose you want a variable to cycle between three different values in a loop, like this.

If this code is executed in a loop, then x will alternate between the values a, b, and c repeatedly. But how may we achieve the same effect without any branching? Tricky...here's a little scheme I've cooked up to do just that. It cheats by using two variables x and y. The mysterious "plus-in-a-circle" operation is bitwise exclusive-or (exor).

First, before the loop starts, initialize x and y, and precompute the magic constant s as follows.

Then inside the loop execute the following non-branching code.

Note that x cycles between the following three values, because of the self-inverse property of exor (see Alternating without a branch).

So if we exor x with the magic constant s we get a value that cycles between a, b, and c, because of the self-inverse property of exor, once again.

There's a secret connection between this scheme and the Fibonacci numbers...

Alternating without a branch

Wed, 13 Dec 2006 14:16:36 -0600

Suppose you want to achieve the effect of a variable repeatedly alternating between two values

(for example inside a loop) without using a branch. If the variables are just bit patterns such as integers or references (pointers) then you may use bitwise exclusive-or (exor) as follows, and eliminate the branch.

Of course, the exor of a and b can be precomputed outside the loop. This relies upon exor being not only commutative and associative, but also acting as its own inverse as follows.

Informally speaking, exor acts as its own inverse; it is both addition and subtraction in its domain.

Soon (after doing some real work) I will say how to alternate cyclically between three different values in a loop, without branching ... this is a little more amusing.

We blog

Wed, 13 Dec 2006 13:15:48 -0600

therefore we cyber-exist. This 'blog will contain a rag-bag of fragments of information, useless by default, serendipitously assembled. Examining this 'blog will be less like watching paint dry than like checking the content of an obscure storage closet in a sub-basement.