The making of Unicode Keyboard #3
So we’ve covered how to make an alternative keyboard layout, how to intercept the key presses from our keyboard and send them where necessary, and how to send characters to the view being edited. The keyboard is functional, it just needs polishing up now.
In order to do that, we really want to know what characters are being entered as the Unicode range is huge and users can’t be expected to remember all those hex codes. So we created a database of all the Unicode codepoints, and inserted that into our application and used it to lookup the codes as they were entered to get the textual name of the dialed in character. This is also helpful since the default Android fonts don’t support most of Unicode and characters show up as square blocks, so its nice to show users what they’re entering when they can’t actually see it.
Thankfully Unicode provide various files describing all the characters, we used one they call NamesList. This describes all the characters and their alternative names. Obviously we can’t use this file as-is, so we converted it into a CSV file that can be imported into an SQLite database. This turned out to be a relatively straightforward task. In fact our resident tester, designer, grumbler and general odd-job-man Ted managed to code this up in PHP in a few hours and he claims not to be able to code at all!
We ended up arranging our CSV file with the integer value of the codepoint for the character, followed by the official name for the character, then the alternative names all separated by tabs:
... 117 LATIN SMALL LETTER U 118 LATIN SMALL LETTER V 119 LATIN SMALL LETTER W 120 LATIN SMALL LETTER X 121 LATIN SMALL LETTER Y 122 LATIN SMALL LETTER Z 123 LEFT CURLY BRACKET OPENING CURLY BRACKET, left brace, opening curly bracket 124 VERTICAL LINE VERTICAL BAR, vertical bar 125 RIGHT CURLY BRACKET CLOSING CURLY BRACKET, right brace, closing curly bracket 126 TILDE ...
This format is rather handy, since we can just import it directly into an SQLite database. Due to the integer field being first we can also use this as the primary key for our table, that makes lookups really fast. For our app we’ve also converted the Blocks file into a similar format so that we can have a list of all the code blocks within Unicode to make our browsing experience nicer.
If you haven’t got SQLite yet you should go download it here. We used the following commands to create our database tables and import our data:
CREATE TABLE codepoints (_id INTEGER PRIMARY KEY, name TEXT NOT NULL);
CREATE TABLE codeblocks (_id INTEGER PRIMARY KEY, name TEXT NOT NULL, lowerRange INTEGER NOT NULL, upperRange INTEGER NOT NULL);
CREATE TABLE "android_metadata" ("locale" TEXT DEFAULT 'en_US');
INSERT INTO "android_metadata" VALUES ('en_US');
.separator \t
.import UnicodeData.csv codepoints
.import UnicodeBlocks.csv codeblocks
Couple of things to note here, we named our primary key fields ‘_id‘, thats to make it compatible with Android since it will look for this field in our tables. We’ve also added an ‘android_metadata‘ table, this is also required by Android. We’ve then set the separator and imported our CSV files into our tables. We now have a database of all the Unicode information we need to create a nice browse and search experience.
Next up we needed to add our database as an asset for our application, and have it copied into place when our app starts for the first time. There’s a handy little class called SQLiteOpenHelper which we’re going to extend to do most of our database access, and we’ll add our copy behaviour to that. Juan-Manuel Fluxà made this great post about how to extend this class to copy databases from your assets folder. We based most of our database code off this so go take a look, but we did find we had to fix one small issue. Before each call to getReadableDatabase() in his class we added a call to close(). This fixes a rather odd bug that crops up on first installation of the database, it appears something ends up in an incorrect state causing an exception, but a quick call to close fixes that.
So when our keyboard is started, we need to copy over the database if necessary, and open up a connection to it. We added a call to createDatabase and openDatabase in our onCreate method to accomplish this:
@Override public void onCreate() {
super.onCreate();
mWordSeparators = getResources().getString(R.string.word_separators);
dbHelper = new DatabaseHelper(this);
try {
dbHelper.createDataBase();
dbHelper.openDataBase();
} catch (IOException ioe) {
throw new Error("Unable to create database");
}
}
We also added some methods to our database helper that let us search and list entries from the database. This seemed like a convenient place to store these, and they’re relatively simple queries :
public String getName(int codepoint) {
Cursor cursor = myDataBase.query(TABLE, new String[] { "_id", "name" }, "_id = " + String.valueOf(codepoint), null, null, null, null);
if (cursor.moveToFirst()) {
return cursor.getString(1);
}
return "Unknown";
}
public ArrayList<CodeBlock> getCodeBlocks() {
ArrayList<CodeBlock> codeBlocks = new ArrayList<CodeBlock>();
Cursor cursor = myDataBase.query(BLOCKSTABLE, new String[] { "_id", "name", "lowerRange", "upperRange" }, null, null, null, null, null);
if (cursor.moveToFirst()) {
do {
CodeBlock newBlock = new CodeBlock();
newBlock.id = cursor.getInt(0);
newBlock.name = cursor.getString(1);
newBlock.lowerRange = cursor.getInt(2);
newBlock.upperRange = cursor.getInt(3);
codeBlocks.add(newBlock);
} while (cursor.moveToNext());
}
return codeBlocks;
}
public ArrayList<CodePoint> getCodePoints(int lowerRange, int upperRange) {
ArrayList<CodePoint> codePoints = new ArrayList<CodePoint>();
Cursor cursor = myDataBase.query(TABLE, new String[] { "_id", "name" }, "_id >= " + String.valueOf(lowerRange) + " AND _id <= " + String.valueOf(upperRange), null, null, null, null);
if (cursor.moveToFirst()) {
do {
CodePoint newPoint= new CodePoint();
newPoint.codepoint = cursor.getInt(0);
newPoint.name = cursor.getString(1);
codePoints.add(newPoint);
} while (cursor.moveToNext());
}
return codePoints;
}
These let us list our entries from the codeblocks database, and lookup entries from the relevant ranges as well as lookup individual codepoints. We added added a call to the getName() method to our handleHexChar method so that we could lookup the character every time we dialled in a new character:
private void handleHexChar(int code) {
// if the character is a hex character append the char to our existing value
if (isHexChar(code)) {
// codepoint stores our currently dialled in number
String hex = Integer.toHexString(this.codepoint);
if (hex.length() >= 5) {
// only want to enter up to 5 hex characters
return;
}
hex += String.valueOf((char)code);
// convert back into an integer
this.codepoint = Integer.parseInt(hex,16);
// uppercase for display purposes
hex = hex.toUpperCase();
// convert codepoint into an actual unicode character as a string
int[] cp = {codepoint};
String s = new String(cp,0,1);
// lookup the name for the character
String name = dbHelper.getName(codepoint);
// add the unicode character onto our string for display in the preview area
// output in the form 'U+2605 ★ Black Star'
this.mPreviewView.setText("U+" + hex + " " + s + " " + name);
}
}
So I guess that covers the basics of how to setup a working SQLite database from a pre-existing asset. We now have a working keyboard and can now see what character we’re entering even though Unicode support on Android is patchy at best. In the next post we’re going to cover something slightly more tricky, showing dialogs within a keyboard, this proved non-trivial as the keyboard isn’t your standard activity and so doesn’t behave quite the same when showing dialogs. More on that next time…