3.0.7s to 3.2: what next?

Today I read Nick's post which links to a 2006 thread on the Death of Angband, which happened around the same time as I released 3.0.7s1, a version of 3.0.6 that included squelch and some minor UI tweaks. That's four years. I didn't think back then that in 2011 I would still be here, but I'm quite happy that I am. :)

3.0.7s

I released that first patch version because I felt there was a lot of negativity and discussion but no-one was actually coding, so in the grand tradition of open source, I effectively forked the code and hoped other people would jump on board.

Looking back, there's three reasons for the fork. First, improve the interface. Second, rewrite/refactor a lot of the code. I'm still working on those.

The third was to change the culture of development so that the maintainership model couldn't come back. The game is quite old and if it was going to survive it needed more than maintainership, it needed what you might call "culture change".

I was part of the email conversations that took place when Robert announced he was searching for a new maintainer, and as I recall, the idea of moving to a more open model wasn't really considered—it was always going to be "pass it on to the next pair of hands", which as far as I was concerned meant stagnation and eventual death, and a lack of involvement from the community which sustains it.

So, I tried hard at the beginning to switch from Angband being freely available to being Open Source. I think I'd just read Karl Fogel's book Producing Open Source Software, so I was a flash hip young go-getter up on OSS norms, and since I'm an infrastructure geek I enjoyed setting up and sorting out the various tools and spaces that open source projects use.

3.2.0

Fast-forward a bit, and it's 2011. The culture change seems to have paid dividends, given that there is now a vibrant, outward-facing community once again. I'm particularly happy that we're using Git now. In 2006, I was using Bazaar to manage my changes for 3.0.6, but back then DVCS was the new kid on the block and it would have been inaccessible to most people. The game itself was still using CVS, which was outdated ven then. GitHub makes things a lot easier.

But when it comes to the code refactoring that I wanted to do and the UI changes that I felt were possible back then, there's been progress but as much as I'd like. Over the past few months, I've finally sat down and had a good think through what would complete the job I set out to do.

4.0

In totality, my plans involve redesigning the game's input and output circuitry, a task I suspect will take a few years. The basic outline goes like this: I want Angband 4 to consist of a core game which is sent commands by a user interface, such as "wear x", "read y", or "attack z". The borg can use this, as can the user-facing frontends. Here's a diagram to show you what I mean:

As for output, I want there to be two user interfaces: a "classic" terminal-emulator style UI and a graphical interface which is usable with both mouse and keyboard.

For the graphical interface, I have no intention of dropping ASCII as the default display mode, but I think there's a lot of things that can be done that break away from displaying everything on a rectangular grid which would make the game more playable: borders around menus, better animated spell effects, better tiles, zoomable maps, variable-width fonts for monster/object descriptions, etc–quite like what the amazing ToME 4 is doing, I guess.

Angband 4 will be as incompatible with existing variant code as Ben Harrison's Angband was to the variants that came before it. There will be pretty much no easy way to port code across from V4 to variants because of how deep the display assumptions in the code are. This makes me sad, but sometimes good things are painful to go through.

Why do I want these things? Well, if we want convincing handheld ports (and I do), at some point the game logic has to be decoupled from its display. The approach outlined above could result in extremely playable Android or iPhone ports, without worrying about virtual keyboards, and making use of native UI on those devices (without adding a load of hacks into the game proper).

It will also be more user-friendly, easier on the eyes, and more newbie-friendly, all of which I rank highly for the game's survival.

Futures

From September, I'm going to try and work part-time hours for about a year, and I'm planning on using that time at least in part to hack Angband. It would be awesome if people wanted to support me financially in that period to do hacking, but I'll cross that bridge when I come to it.

It would be good to know before any of the above kicks off in a serious way: would you want to play a game that had more graphical touches than the game does at present? Would it put you off? Where do you want to see Angband's interface go—or would you rather it stay right where it is?

Thoughts welcomed.

First post! (or: refactoring hell)

So today, I had a quick look at how difficult it would be to get the pref files referencing monsters by name rather than by number, as part of the general effort to remove indexes from the game and make the world friendlier for any budding tileset designers.

The basic sketch should be: make prefs.c output the right file format, convert all the graphics edit files, add a function to get monster race given a name, rename duplicate monsters, and done. Shouldn't take more than an hour.

First step: edit prefs.c to output R: lines with monster names on.

All looks good... oh no it doesn't. I notice lines like:

R:E , the Dark Elf:17:104

That's rubbish, I cry. What's happening here? Well, there's a code module in Angband (to the extent that Angband has code modules) called xchars, which takes strings like "E['o]l the Dark Elf" and turns them either "Eol the Dark Elf" or "Eól the Dark Elf" depending on your platform can support and maybe other factors. In this case, it seems to have failed to do anything useful at all and just be outputting blank spaces.

Why? I do a bit of digging and it turns out that xchars is used when reading anything from any file, so that the monster's name when it's read in from the edit file in the form E['o]l is saved in memory as Eól, in a now-defunct character encoding known as Latin-1.

I assume this has been done to prevent having to make any substantial code changes while gaining a fancy new display feature. A bit like every other feature for the past two decades, then. It would make a lot more sense if the game kept strings in the form they were read in as (with the weird [![?] stuff) and the display code was changed so that some display characters could take up more than one byte in memory.

That would have been the long-sighted, intelligent thing to do that didn't involve just adding another low-level hack that permeates the code. For example, if that had happened, we could make UTF-8 the default encoding of the game with very little difficulty other than wrapping our heads around the Unicode spec and implementing it in C. As it happens, now we have to do that and rewrite the mess that is xchars.

Gragh.

So, in finest Angband tradition, it turns out there's a quick hack around this problem, which is to call xstr_trans(buf, ASCII). That converts a string, in-place, to a new encoding... which presumably may result in a string that is longer than the original string, and since xstr_trans() doesn't take a size for 'buf', could cause overflow and memory bugs. Yay! (There's actually a bare strcpy() in there, I'm not sure how I missed that.)

Looking at the function and the comments above it, to be safe, we need a buffer 1024 bytes in length. OK. We can deal with this if we're strong.

First step, try two: Add xstr_trans() call.

Oh, now the output looks like this:

R:Eol, the Dark Elf:149:214

So we've converted from Latin-1 to ASCII, but not to the 'canonical form' stored in text files. In fact, there is no code to translate from Latin-1 or whatever weird pseudo-native format the game wants to use internally back into the canonical platform-agnostic form that it should have in data files.

Grrraaagh!

So now there are two choices:

1. Add a function that can convert back, temporarily working around the hack and the other hack with a less smelly hack, all in the name of the greater good
2. Fix xchars to happen on display, not on processing.

This is why refactoring Angband takes years. Everything is so entangled that you lose interest before you get to the bottom of the tangle. Or, you ignore the tangle and build stuff on top of it and hope it works.

Also, this lovely comment in x-char.h:

/*
* x-char.c function declarations are located in externs.h.
* x-char-c tables are loctaed in tables.c
*/

What's the point, really?