What I wanted was a password generator that could output a configurable length password using only easily distinguishable letters and numbers, so I wrote one. As usual I place this code in the public domain, feel free to use it any way you want.

Features:

1. The entropy of the password is as good as the underlying operating system. If you use a recent Linux or OSX version, the data returned from /dev/random is quite good.
2. The code is simple and it is easy to verify that the program actually uses the entropy that it reads.
3. The resulting passwords are easy to type on keyboards and write on paper without confusing the reader with similar characters such as 1 (one) and l (lower case l).
4. The length of the password is configurable.

#!/usr/bin/python
 
import sys
 
# alphanumeric chars minus l, I, O, 0, 1
alphabet = "abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ23456789"
# Some expeimentation told me that 2 ** 5.8 = 55.7
BITS_PER_CHAR = 5.8
 
# The default password length has the capacity of a bit more
# than 64 bits of entropy.
DEFAULT_LEN = 12
 
def main(args):
	count = DEFAULT_LEN
	if len(args) > 1:
		if args[1] == '-h':
			usage()
			return
		elif args[1] == '-c' and len(args) > 2:
			count = int(sys.argv[2])
		else:
			usage()
			return
	print(create_password(count))
 
def string_to_bignum(s):
	num = 0
	for c in s:
		num = ord(c) + (num << 8);
	return num
 
def create_password(length):
	random = open("/dev/random", "r")
	needed_bytes = (int)(length * BITS_PER_CHAR) / 8 + 1
	n = string_to_bignum(random.read(needed_bytes))
	random.close()
	s = ""
	for i in xrange(length):
		s = s + alphabet[n % len(alphabet)]
		n = n / len(alphabet)
	return s
 
def usage():
	print """mkpasswd [-h] [-c COUNT]
Create a random password using the operating system's entropy pool
using a 57 character alphabet of letters and numbers. The characters
in the alphabet excludes characters and letters easily confusable such
as I and 1. 
 
Each password character holds about 5.8 bits of entropy, so the
standard 12 character password can hold a theroretical maximum of
69 bits of entropy.
 
The actual entropy present in any generated password is a function
of the entropy gathering algortihm present in the kernel of your
operating system.
 
  -h        display this help text
  -c COUNT  create a password with COUNT characters."""  
 
if __name__ == '__main__':
	main(sys.argv)

A driver for this card is included in the standard CentOS 5 kernel (tested with kernel-2.6.18-164.9.1.el5) under the name rt61pci, however to be able to function a binary firmware is needed. Some searching revealed that the firmware is available in Fedora's rt61pci-firmware package. I rebuilt the package and put it in my CentOS playground repository.

So, if you have such a card and want to make it work in CentOS5 you might want to try running rpm -ivh http://rpm.resare.com/centos5-playground/i386/rt61pci-firmware-1.2-6.el5.noarch.rpm as root. Once that is done, running modprobe rt61pci should do the trick.

This is useful for example when you are about to sign up to an online service that requires your email to be validated by sending an email  message with a link for you to click to activate your account. If you spell your email address incorrectly the registration will fail, but you will not typically get any feedback that the email delivery failed.

With rjmailer it is possible to build an online service that tells the users if an email delivery failed right away in the web form that was used to register. Having this information can help the user to correct a spelling error or remove messages from a full inbox.

I released the initial public version of this piece of software back in april, but I haven't really had any time to start using it at work until now. This past week I spent a few hours modifying one of our projects, http://biblesearch.org, to use rjmailer when sending out account activation links and report back to the user if there was an issue sending out the mail. Feel free to try it out if you want, registering is free. Just fill out the form at the new user registration page.

While adding this feature I found a few small issues in rjmailer, so there is a new version available for download

DNSSEC protects the DNS system against a certain group of security problems such as the kaminskybug, where an attacker tricks a DNS server to return the wrong data to end users. If an attack against the DNS system is successful that means serious trouble, since we depend on it to work reliably in a vast number of online activities. An attacker that controls the DNS system can trick people to for example supply their account information to their online bank and use that to steal money. Whenever there is the potential for large scale fraud you can pretty much be sure that someone will try to break it, and that is why DNSSEC is important.

So, we need DNSSEC. What's stopping us from using it? A few things, but the most important obstacle in my opinion is that it is a complex set of standards and that it is difficult to understand. There are some presentations and HOWTO documents online that attempts to explain and help people get started, but the learning curve is steep. One thing that I ran into when experimenting with my own zones was that somehow I managed to corrupt the signatures of one zone and I couldn't easily pinpoint what the problem was.

When confronted with this I got the idea to build an online service that tries to answer a simple question. What data was used and what cryptographic operations was performed to actually verify one specific DNS record? The answer to that question can be thought of as a chain of operations and records where one link connects to the other from all the way from the record being verified down to the DLV root key.

I decided to write the service in Python and it was one of the most fun programming projects that I have worked on in years. In a way it was basic research but with a clear application and an end result that I think could be a useful contribution. I even wrote my own RSA signature verification functionality, with a lots of help from Python's excellent large integer support.

The service can be found at http://dnssec.resare.com Feel free to give it a spin. There are no doubt bugs and errors that will be fixed and other modifications that will be made, but the basic functionality is in place.

Thanks to Alex for the beautiful HTML design,  to the python dns library dnspython that I use extensively and the airspeed templating library.

I've been missing that functionality on my Mac, so I wrote a small wrapper to the openssl command that provide the same basic functionality using Python. Python is really handy when it comes to writing small scripts like that does some string handling and calls other programs and since the basic checksumming functionality already is available in the openssl package it simple, short and neat.

As usual, feel free to use this any way you want.

 
#!/usr/bin/python
 
import subprocess
import sys
 
def checksum_file(filename):
    sp = subprocess.Popen(["/usr/bin/openssl", "sha1", filename],
                          stdout=subprocess.PIPE)
    retval = sp.communicate()[0]
    return retval[retval.find("= ") + 2:-1]
 
def verify(checksumfile):
    f = open(checksumfile, "r")
    for line in f:
        line = line[:-1]
		(sha1, fn) = line.split("  ")
		calc = checksum_file(fn)
		if calc != sha1:
            print "%s: FAILED" % fn
            sys.exit(1)
        else:
            print "%s: OK" % fn
 
def usage():
    print "Usage: sha1sum [-c CHECKSUM_FILE] [FILE]..."
    sys.exit(1)
 
if __name__ == '__main__':
    if len(sys.argv) == 1:
        usage()
	if sys.argv[1] == '-c':
        if len(sys.argv) != 3:
            usage()
        verify(sys.argv[2])
    else:
        for f in sys.argv[1:]:
            print "%s  %s" % (checksum_file(f), f)
 

Unlike most of the internet i use Unbound instead of bind as my nameserver. It offers great DNSSEC support as well as a well maintained code base. One recent feature is the use of mixed case labels when sending queries to other nameservers, as outlined in the DNS0x20 document. This is one countermeasure to the DNS Spoofing attacks that is an increasing problem on the internet these days, and it depends on the fact that name servers should treat queries that only differs in the case as if they were equals. In other words, mobizoft.qbrick.com and MobiZoft.Qbrick.com should be treated as the same.

The exact wording of the specification can be found in RFC1035 section 2.3.1:

Note that while upper and lower case letters are allowed in domain
names, no significance is attached to the case.  That is, two names with
the same spelling but different case are to be treated as if identical.

Unfortunately, Qbrick's nameservers fail to implement this specification, and mixed case questions gets answered with the NXDOMAIN reply code, which means that there is no data for the given domain name. I hope that Qbrick will get their act together and fix this soon, but in the meantime it can be a good idea to use the use-caps-for-id: no directive if you are using unbound.

In summary it is a bit annoying that errors like these are so hard to find and correct. Most video displaying flash plugins will not report a meaningful error, and the fact that SVT uses an external provider for their streaming video solution puts the problem even further away from the end user.

Update 091104: I have now gotten in contact with Qbrick. They recognize the problem but state that they have an ongoing project to replace the DNS solution and they will not address this issue until the new solution is in place.

I have tested it with the XMPP and SMTP protocols and should run on python 2 variants from 2.3 onwards. Feel free to use it any way you want. Comments and suggestions for improvements are welcome, as always.

1. The text was encoded in UTF-8 in the old database, but mysql thought that it was what it calls latin1. What I had entered as å the database perceived as Ã¥, but the transformation was applied on both write to and read from the database, so the characters turned out to be correct when displayed in bugzilla again.
2. The default behavior of mysqldump is to treat data it knows to be latin1 into UTF-8 in the output file. Since my data was really UTF-8, but mysql was under the impression that it was latin1, it encoded the UTF-8 into UTF-8 once more.
3. To make matters even more complicated, what mysql calls 'latin1' is not actually ISO-8859-1 but rather a slightly modified variant of the Windows-1252 character encoding. A result of this is that in some instances the double application of the UTF-8 transformation a single input character results in 5 output characters.
4. The solution to this mess is a curiously named option to mysqldump named --default-character-set. It can be used to override the default behavior of encoding strings marked as latin1 into UTF-8. mysqldump --default-character-set latin1 outputs my UTF-8 correctly. Once the database is in a file, just search and replace default charset=latin1 with default charset=utf8 and import the data.
5. At this point, the data that was UTF-8 all along is now correctly understood by mysql to be UTF-8.
6. Next problem: when starting up bugzilla with UTF-8 settings the characters still gets mangled.
7. It turns out that the bridge between mysql and perl in CentOS 5, the perl-DBD-MySQL package, is too old to support the mysql_enable_utf8 connection parameter. As a result, strings coming out of perl-DBD-MySQL containing non-ascii is not marked as utf8 strings.
8. So, why didn't checksetup.pl tell me this when I ran it? It turns out that there is a patch in the bugzilla shipped with EPEL to remove the check for the proper perl-DBD-MySQL version to make it runnable on CentOS 5. Perhaps a reasonable tradeoff, but a bit annoying when trying to find out what fails.
9. So I compiled a recent perl-DBD-MySQL and put it in my playground repository and now my bugzilla displays all sorts of strange characters correctly.

There is a Random class in java that returns pseudorandom numbers. However, there is no guarantee that when calling Random.nextInt() a number of times the same number will not be returned twice.  In other words, the simple strategy filling a small list with elements from the large list picked at random with the help of Random.nextInt() will probably produce a result list when the same element is added twice. Not good enough.

One defense against that problem could be to keep track of which numbers the random generator has returned and simply discard any duplicates. For most cases that strategy is probably efficient, but sometimes it could lead to having to discard a whole bunch of random numbers. Moreover, execution times of such a solution would differ a lot depending on which numbers Random.nextInt() returned.

Another solution would be to copy all of the large list to a temporary list and remove each element taken to the smaller list from the temporary list. That way, no item would be added twice to the result list. However, if the source list is long, say a million entries, creating a temporary list to fetch holding all the million entries just to extract 100 random entries would be inefficient.

The elegant solution to this problem in my mind would instead be to create a list of offsets into the large list as long as the small list. The list of offsets would then be modified to emulate the effect of having a copy of the large list and removing entries from it. If the first entry of the offset list is offset 1, and the second offset also is 1, modify that to 2 instead, the element that would have been at position 1 if the original element at that position was removed.

I wrote a class implementing this solution, Randomizer.java. Feel free to use it or modify it anyway you wish.

However, I really enjoy the challenge of writing efficient and elegant code that solve a particular problem, so sometimes when I get a bit of free time and want to do something relaxing I spend time writing solutions to small programming puzzles that I run across in my day job working for Voxbiblia.

The next post contains the solution to one such puzzle and the solution I just wrote. It might be interesting to some.