In Honour of Ben Harrison

I present to you the "blog" of "unnecessary" quotation marks.

Core-UI split: user input

A lump of slate

Here are my current thoughts on how to move forward with the Core-UI split.  At the moment there aren't really layers - it's more like a lump of slate that's starting to crack in two but isn't quite there yet.  The biggest single issue that I see right now in moving towards that goal is user interaction, and that's what this post covers.  This post was written while I was working out the issues, so I hope it makes sense to other people!

At the moment, there is a game command layer which is the intermediary between the UI and the core.  Roughly speaking, the core runs in a loop on a queue of commands.  If the queue is empty, it passes control to the UI and asks it to get a command.  The UI then gets some input, works out what game command that corresponds to, and pushes that command onto the game stack.  When the queue isn't empty, the game just runs through it, executing every command it finds until it runs out of commands to process.

This picture is complicated a bit by the presence of macros.  At the moment, macros are implemented at the UI level as a queue of keypresses.  So there are two queues operating at once - one command queue and one keypress queue (actually, there are three queues - the terminal code keeps another keypress queue).  To illustrate, say we have a macro P:

P = p1bt

And let's start with an empty command queue.  This is the flow inside the game:

Core: Queue is empty -> ask UI for keypress

UI: Player presses P -> put p1b*t into the keypress queue

UI: Process 'p' -> UI command calls prayer book selection code

UI: '1' selects book inscribed @p1 -> select prayer menu

UI: 'b' selects prayer b -> put "pray book:1 spell:b" into the command queue, return control to core

Core: Checks over command to make sure it's got all the data

Core: Calls associated command function - do_cmd_pray(1, b)

Core: 'b' requires a target so it prompts for one

UI: 't' popped from keypress queue, ends prompt

Core: execute prayer

The problem here is that the game core isn't given enough information up-front to execute the prayer without user input.  So the question is: what do we have to do to get the command layer running without user input?

First off, there is a problem at the moment in that the game command queue handling code sometimes fills in the paramaters given to the functions that execute those commands.  So, for example, when the game core comes to execute an 'open door' command, but no direction is provided, it will check to see how many doors can be opened around @.  If more than 1, it will prompt.  This is bad layering, and the way to fix it is to make the game command code refuse to run incomplete commands, and put that logic in a level a bit higher up.

The bigger problem, though, is the one in the prayer example.  Different commands require different input, but not based on their command type (aim, read, pray, etc) but on what the effect the object they are used with will invoke.  It doesn't matter whether you're making a fireball from a want, spell or rod - they all need a direction.  Enchant Weapon requires an item regardless of whether you're reading from a scroll or a spellbook.

To make this work, we evidently need close tie-in between the command and effects code.  Effects basically need to be specified in such a way that their input requirements and implementation are coded up separately. How to do this in C is quite beyond me at present - while many effects only need a target, some portion want to select an object, which requires a prompt, a filter function and an error message if the effect can't be invoked because there is no appropriate object.  All this needs to be specified in some declarative way so that the UI, game core, and effect code can all access and use it.

Any help appreciated!

After sorting out this (very sticky) point, I think we're pretty close to a proper UI-Core input split.  One useful way of enforcing the constant that the core layer doesn't ask for input would be to add assert()ions in inkey() when a "in the core now" flag is set, and that would help find the remaining instances pretty quickly.  Of course, there's still some output issues - I'm sure some of the core code writes to the screen directly - but they could hopefully be sorted out pretty quickly.

Macros

The ambition, after making all game commands run without user input, is to rid ourselves of the second key queue inside the UI layer, and we can do this by rewriting macros to be a string of game commands sent to the core in one bunch.  So instead of:

\e\ep8d-a\e\ep8d-a\e\ep8d-a\e\ep8d-a\e\ep2f\e\eR\r

Your macro would look like:

cast "Enchant Weapon" item:floor:1

cast "Enchant Weapon" item:floor:1

cast "Enchant Weapon" item:floor:1

cast "Enchant Weapon" item:floor:1

rest

So what exactly is an "ego" item anyway?

I think the concept of the "ego item" in gaming can be credited to Gary Gygax, back in 1E AD&D if not before. Literally, he meant an object with an ego, with its own personality. In gaming terms, these were powerful items whose egos meant they were not to be trifled with, expressed their views and could in extremis refuse to obey the wielder.

As ever with Angband the original intent of the designers is unclear, but over the years the term has come to mean "an item with one or more properties which its base item type doesn't have, excluding plusses to-hit/dam (on weapons) or to AC (on armour pieces)". The original concept of powerful arcane items with their own personalities is closer to what we think of as artifacts, and indeed some roguelikes actually express the personality of some artifacts quite explicitly (e.g. the Singing Sword in Crawl).

Having collected the stats from 25million dungeon levels in 3.3.0 we're working towards a fairly significant overhaul of item generation, to try and rebalance the prevalence of weapons and armour which has crept up on us over the last few years. As many people have observed, there are some deficiencies in the current system - two main ones being the distinction between "special" and other artifacts, and the granularity of the alloc_prob variable. There are also obvious problems with distribution: the vast majority of stuff is findable in the first half of the dungeon, making the second half less interesting and making potentially interesting items obsolete too quickly.

It seems to me that the current system of templates for ego items is quite limiting - you need a new template for every variation of an ego, such as if you want it to be quite common on boots and quite rare on armour, or permit a higher pval in one slot than another. The potential for random variation is currently limited to a single random sustain, random high resist or random ability, and even these are implemented in a spectacularly hackish way. There's also no reliable way of providing a smooth power curve from {average} through {good} to low-end ego to high-end ego. Although you ought to be more likely to find Slay Troll than Holy Avenger on dl20 and the reverse on dl80, that isn't noticeably true.

I've had item affixes in the corner of my mind for a couple of years now. EyAngband introduced a number of useful innovations, this one clearly inspired by the prefix/suffix system used in many modern CRPGs (Diablo, Dungeon Siege, Titan Quest et al.). While EyAngband used them as a bolt-on to the existing framework, it would seem eminently plausible to replace Angband's template system with an affix-driven system for magical and ego items. What follows are my initial thoughts on how this could work and why it would be an improvement - feedback would be most welcome.

First, let's distinguish between craftsmanship and magic. A set of affixes like sharp/keen/brutal of piercing/slashing/bludgeoning could refer to additional damage-dealing properties of weapons which result from superior craftsmanship rather than magic (e.g. +dam, or chance of 2x dam, or whatever). Similarly, affixes like superior or of deflection could imply similarly-derived properties for armour (e.g. increased AC, reduced weight). As an aside, this opens up interesting possibilities for making detection slightly less dominant if these items did not show up as magic.

Next, you have all the magical affixes, which should be divided into groups. These would include slays, brands, resists, stat boosts, protections, movement effects (speed, stealth, FA, lower weight, etc.) and so on. I envisage a one-to-one mapping of affixes to object flags, so that any code for picking affixes can provide any individual flag. (Note to self: get the OFT_ types right in list-object-flags.h and this grouping is already done.)

That gives you the basic framework for creating the equivalent of {good} items and low-end egos. What we now think of as {good} would be those with a non-magical affix (occasionally two) which purely boosted offence or defence, while those with one magical affix would be the equivalent of today's single-slay weapons and other one-trick egos.

Where it gets interesting is combining multiple affixes. You can combine a prefix and a suffix to get items ranging from mildly exciting to endgame (anyone remember Godly Plate of the Whale?). You can combine more than one of each to re-create the current high-end egos, with hundreds of other possibilities. You can combine craftsmanship and magical affixes to get even more permutations (sure, Gondolin weapons are special, but some of the smiths were better than others). The key thing about this is that it provides a route to a smoother power curve - the number of affixes would be a function of depth, with shallow items rarely getting more than one or two and really deep items rarely getting fewer than four or five.

Not all properties are equal so it would be sensible if there was some way of telling the generator that "of Dragonbane" is a more useful affix than "of Troll-slaying". I know that some people intensely dislike numerical power ratings, so I remain open to alternative ways of achieving this. In the meantime, list-object-flags.h already provides a rating for each flag, and a multiplier for each slot, which could be used as a starting point.

To avoid a total mish-mash of random permutations, we could tell the generator to recognise certain combinations and steer its later choices. This would enable us to retain the flavour of existing ego types without their limitations - we could specify the likelihood of each affix on a per-slot basis, and limit the later choices as much or as little as we wanted. (Hmmm, this item has one more affix to go, and it's already got SLAY_ORC and SLAY_TROLL - looks like a weak Westernesse weapon to me, so let's give it STR or DEX ... oooh we rolled a 1_in_6, let's bung in a sustain for free ...) It would also go hand-in-hand with naming items with multiple affixes - so for example anything with two of SLAY_EVIL, SLAY_UNDEAD and SLAY_DEMON gets the prefix "holy".

This proposal will mean rethinking the concept of randarts. With more randomness in non-artifact items, it becomes less important in artifacts. I think there is still a place for randarts, but how to distinguish them from these new ego items is a topic for another day. (My initial instinct is to randomise only *some* of the properties of each artifact ...)

So how are we going to balance this to make sure it doesn't flood the dungeon with egos (er ...) or, even worse, leave people struggling with a +0 whip until 2000'. That's what the stats module is for - rather than agonise over balance before writing it we can write it, run some stats, adjust it and iterate. It's non-trivial (the code for picking names is going to be tricky - fortunately there's a handy precedent in Sangband's class titles), but I think it's possible.

What do you think?

Some thoughts on the Angband economy

I have just been reading a book called Debt by the anthropologist David Graeber - it's a fascinating book, examining in some depth the forms that states, markets, money, credit and debt have existed in. This has prompted me to ask some questions of Angband's economy, some of which I am sure are entirely superfluous and others which might prompt some useful, gameplay-improving answers.

So, first: Angband's economy is based entirely in gold coins and the convertibility of precious metals and minerals directly into gold coins. Note that the game itself never says that you have any gold coins: just so much stuff that is equivalent to so many gold coins.

Where is the adventurer finding enough space to carry (at the beginning of the game) enough chunks of copper to the value of 213 gold pieces or (at the end of the game) enough chunks of precious minerals to the value of tens of thousands of gold pieces? That much stuff would weigh you down. I think an intelligent player would try to maintain a certain amount of tradeable currency, but not pick up everything they found.

Also, the amount of gold that successive adventurers would bring up from the dungeon would surely cause massive inflationary problems...

And why does everyone in the dungeon seem to be carrying around so many precious stones all the time? Their needs as troops (food, water, weapons, armour) will surely be met in some semi-centralised way, since Morgoth needs an army that can fight, that is well-nourished and the like. So in everyday usage, soldiers would make no use of money or precious stones. Why are they carrying them?

Also, how does the town even exist? It is a weird place, atop a massive dungeon full of murderous individuals, all looking to profit from looting and pillaging. The town is nevertheless full of other people, none of whom seem to be particularly interesting or adventurous, all of whom must survive somehow but clearly do not possess the survival skills to fight even a moderately well-equipped player. These people must all eat and sleep somehow. They must also be occupied one way or another. Who is paying them to laze around in a position of extreme danger?

The player has an unguarded, unprotected home in which they can place whatever they like and be sure that it is there even though the building is unoccupied 99% of the time. How does it not get broken into?

Also, why is is cheaper to buy arrows than food? Why are cloaks cheaper than food? Food is essential for life. The idea that a ration of food should cost four gold pieces makes the real-world inflation of food prices look seem tiny by comparison. And why is food handled in-game but drink not?

Why does the player not simply demand what they want from the shopkeepers and threaten to kill them otherwise? There is a total lack of authority or protection to keep this from happening. There is no reason for the shopkeepers not to kill the player, either.

I think a lot of the weirdness of the Angband economy comes down to this: we are all very used to exchanging money for other things we want or need in shops, in line with the rule of law. However, the situation that lets us do this (state authority, police, judicial systems) is pretty much entirely dissimilar to the situation in Angband.

Actually, the entire system of tradeable commodities that Angband has is completely unfitting to the place the game is set. The idea that you can have seventeen identical longswords is somewhat ridiculous, or twenty pieces of plate mail - these things historically would not have been available for trade. You would, at best, be paying someone to make them for you, to your specification. Smiths would not have created massive surpluses because there would have been no reason for them to do so.

That's all very well, but...

So what might happen if we try and provide more reasonable answers to these questions? What happens if we try to re-balance the game's economy? I think that some gameplay issues might start to disappear, mostly those around gold availability, buying, and selling.

From the list above, I would start off with:

1. Make currency weigh the player down. Allow them to drop it, pick it up, etc.

2. Reduce the amount of money that monsters in the dungeon carry, significantly. Some monsters should hoard lots of money (but should maybe not be carrying it - think dragons) but most will have none. The current scaling of amount of GP / drop with dungeon level will need to be rethought.

Pair this up with a reduction in the amount of money required to purchase items. EyAngband started doing this quite well.

3. The home should either be a general item-deposit place that holds your items securely for an access fee, or it should be a home that requires you to pay a retainer to guard it.

After this, what? I think food and drink need to be reconsidered - either they need to become serious elements of the game or they need to disappear. The shops need a plausible back-story; are they all merchants, bringing wares to flog to needy adventurers, or is there a permanent weaponsmith, potionbrewer, etc in residence all the time? How do they survive?

These questions affect the gameplay decisions around them. If the weaponsmith's is actually a weaponsmith's, then maybe you can ask them to make you a new weapon for a given price, but they are unlikely to keep hundreds of weapons in their store. But if they are a merchant, they will surely want to bring some impressive swords along with some cheaper ones - but maybe they will only part with them for a large sum. They certainly wouldn't bring 18 whips and 18 daggers.

Maybe, also, the player could leave a deposit on a particularly handsome item and come back for it later with more money. Combined with a reduction in the availability of gold, this turns into less of a scum-fest and more of something with the potential to be an interesting minigame.

Merchants would also be unlikely to buy unimpressive weapons or items - their job is to turn up with wares, flog them and get lots of precious stones and metals to take back away with them. Unimpressive weapons are worth less to the merchant than the equivalent in gold pieces, because the entire point of the trading is to exchange weapons for more liquid assets. There would have to be regular trips away from the area in order to get more stock and deposit money with a bank, out of danger - this provides a nice explanation for shopkeeper rotation.

And why the hell would there be a temple above Angband? On this thought, I'll leave you.

So many flags, so little time ...

So I finally managed to refactor monster spells into a data table (or

three, in fact). It taught me a lot. As takkaria notes in the first post, the thinking in the original code is not always easy to divine, as layer upon layer of change over the years either obscures it or undermines it (if it was there in the first place).

What we had was half the spells being resolved in monster2.c (the non-projectable ones), and the other half being resolved by the projection code in spells1.c. We now have a separate mon-spell.c which will handle all spells and their side effects, calling out to project() where necessary.

Monster spells, by the way, means anything that isn't a melee attack - so all breaths, balls and bolts as well as the non-damaging spells like teleportation, confusion, summoning etc. Anything that's on an S: line in monster.txt and has an RSF_ flag in the code.

This work led me to two conclusions: the first is that data tables and nice flavourful text output don't mix well. It was hard enough doing all the different verbs for slay_table, but I think it's worth moving to an edit file for the spell text. If I understand takkaria correctly the list-foo.h data tables are a stepping stone towards some sort of scriptable edit file setup anyway, and elly's brilliant parser API makes writing a new parser possible for the syntactically challenged. This way we can keep (and even enhance) the flavour of the different in-game messages without cluttering the list-* files.

The second conclusion is that we ought to go back to first principles and have a think about object flags before we make too many more decisions which are constrained by the existing setup. Object flags are fundamental to the game, because they are used by p_ptr->state to determine resistances and other abilities granted to @ by objects. (Perhaps really going back to first principles would challenge this way of tracking @'s state, but that's a separate essay - for now I'll start from here.)

At the moment object flags are all lumped together in a single group, and we use a load of #defines to create "masks" for particular flag types (slays, brands, obvious flags, etc.). We recently expanded OF_SIZE because we went beyond the limit of 96 flags when the new HATES_FOO flags were introduced in object_base.txt. We can now have up to 255 object flags before we need to change savefiles again. (takkaria has written a new block-based savefile format, which should mean much better savefile compat in future.)

But there are already two different types of flags, and should probably be more. There are binary flags (you either have fire immunity or you don't), and there are pval flags (you have an object which affects your STR, but the precise effect is determined by a pval).

Before I added multiple pvals, there was no structural governance of which flags were connected with an object's pval and which weren't - it was all done by hard-coded rules. Now we have a set of o_ptr->pval_flags for each pval, which simply indexes each flag which is related to that pval. This may seem memory-inefficient at 32 bytes for each pval, but it allows object flag operations to use a single flagset instead of searching over N pval flagsets (+1 for the non-pval flags). It also allows a binary flag to become a pval flag with a minimum of hackery.

As I was acquainting myself with the ID code last year, I was gratified to learn that o_ptr->known_flags is exactly what you expect it to be: a bitflag which tells you which object flags @ knows to be present on the object. I assumed that o_ptr->flags was the actual flags present on the object - and it is in a way, but it's almost never used directly.

Instead a function will create a bitflag f[OF_SIZE] and call object_flags(f). This copies into f the flags of the base object type, those of the object kind, then flags from any ego type or artifact. Instead of being done once at object creation, this is done repeatedly throughout the code. If there was ever a reason for this I can't find it - the object list is global, so the memory for all the o_ptr->flags of all the objects is already in use at any point.

Of course, the object list isn't a single list. There's o_list, which has all the items on the current dungeon level, including those carried by monsters. There's p_ptr->inventory, which is everything @ is carrying and wearing. There's st_ptr->stock, which is all the objects in a store. But they're all global.

In fact, the only flags which *aren't* copied over every time object_flags is called are the curse flags - so when you remove a curse from an object, it stays removed. Now, why doesn't this apply to every other flag too? That way all sorts of spells and effects in the game could add or remove flags from objects, opening up all sorts of possibilities (forging, anyone?) - including the long-gestated new curses. Recent discussion on Oook came down against allowing effects to remove flags from objects, but dispelling timed buffs was considered tolerable.

So we come to the issue of timed flags. The TMD_ flag set is quite recent (2008) and the MON_TMD_ group are even more so. We will have to be careful to keep these separate from object flags, and test for both timed and object flags if necessary. In calc_bonuses we copy TMD_OPP_CONF to p_ptr->state, and likewise with TMD_AFRAID. We should soon be able to do without this, as the new check_for_resist function returns nonzero if either is present. But at the moment it only works for effects whose temporary and permanent effects are linked in gf_table, so something needs redesigning if it is to be more broadly useful.

My suggestions so far:

1) Make o_ptr->flags the definitive source of an object's flags (call object_flags at creation, but not subsequently). This would allow us to do lots of interesting things to add flags before upsetting people by taking any away (but some of the additions could be curses ...)

2) Synchronise OF_FOO and TMD_FOO so that they are linked pairs. This should enable a single check for any flag in the player's state, regardless of whether it comes from an object or a temporary spell effect.

3) Add o_ptr->timed_flags[OF_MAX] so that each object flag can be added or removed temporarily as well as permanently. This would add about 600k to the memory footprint (256x u16b per object, or more if we want timers over 2^16), but should still leave us about two orders of magnitude short of memory-limiting an average smartphone.

4) Create of_table[OF_MAX] to replace all the #defines for OF_FOO_MASK. This would also allow the encoding of other info about a particular flag (e.g. whether it could be found as a random ego power, or on a randart).

None of this directly changes gameplay, but it opens up possibilities that the current code cannot offer. Even if we don't want to use them, variant maintainers might.

3.0.7s to 3.2: what next?

Today I read Nick's post which links to a 2006 thread on the Death of Angband, which happened around the same time as I released 3.0.7s1, a version of 3.0.6 that included squelch and some minor UI tweaks. That's four years. I didn't think back then that in 2011 I would still be here, but I'm quite happy that I am. :)

3.0.7s

I released that first patch version because I felt there was a lot of negativity and discussion but no-one was actually coding, so in the grand tradition of open source, I effectively forked the code and hoped other people would jump on board.

Looking back, there's three reasons for the fork. First, improve the interface. Second, rewrite/refactor a lot of the code. I'm still working on those.

The third was to change the culture of development so that the maintainership model couldn't come back. The game is quite old and if it was going to survive it needed more than maintainership, it needed what you might call "culture change".

I was part of the email conversations that took place when Robert announced he was searching for a new maintainer, and as I recall, the idea of moving to a more open model wasn't really considered—it was always going to be "pass it on to the next pair of hands", which as far as I was concerned meant stagnation and eventual death, and a lack of involvement from the community which sustains it.

So, I tried hard at the beginning to switch from Angband being freely available to being Open Source. I think I'd just read Karl Fogel's book Producing Open Source Software, so I was a flash hip young go-getter up on OSS norms, and since I'm an infrastructure geek I enjoyed setting up and sorting out the various tools and spaces that open source projects use.

3.2.0

Fast-forward a bit, and it's 2011. The culture change seems to have paid dividends, given that there is now a vibrant, outward-facing community once again. I'm particularly happy that we're using Git now. In 2006, I was using Bazaar to manage my changes for 3.0.6, but back then DVCS was the new kid on the block and it would have been inaccessible to most people. The game itself was still using CVS, which was outdated ven then. GitHub makes things a lot easier.

But when it comes to the code refactoring that I wanted to do and the UI changes that I felt were possible back then, there's been progress but as much as I'd like. Over the past few months, I've finally sat down and had a good think through what would complete the job I set out to do.

4.0

In totality, my plans involve redesigning the game's input and output circuitry, a task I suspect will take a few years. The basic outline goes like this: I want Angband 4 to consist of a core game which is sent commands by a user interface, such as "wear x", "read y", or "attack z". The borg can use this, as can the user-facing frontends. Here's a diagram to show you what I mean:

As for output, I want there to be two user interfaces: a "classic" terminal-emulator style UI and a graphical interface which is usable with both mouse and keyboard.

For the graphical interface, I have no intention of dropping ASCII as the default display mode, but I think there's a lot of things that can be done that break away from displaying everything on a rectangular grid which would make the game more playable: borders around menus, better animated spell effects, better tiles, zoomable maps, variable-width fonts for monster/object descriptions, etc–quite like what the amazing ToME 4 is doing, I guess.

Angband 4 will be as incompatible with existing variant code as Ben Harrison's Angband was to the variants that came before it. There will be pretty much no easy way to port code across from V4 to variants because of how deep the display assumptions in the code are. This makes me sad, but sometimes good things are painful to go through.

Why do I want these things? Well, if we want convincing handheld ports (and I do), at some point the game logic has to be decoupled from its display. The approach outlined above could result in extremely playable Android or iPhone ports, without worrying about virtual keyboards, and making use of native UI on those devices (without adding a load of hacks into the game proper).

It will also be more user-friendly, easier on the eyes, and more newbie-friendly, all of which I rank highly for the game's survival.

Futures

From September, I'm going to try and work part-time hours for about a year, and I'm planning on using that time at least in part to hack Angband. It would be awesome if people wanted to support me financially in that period to do hacking, but I'll cross that bridge when I come to it.

It would be good to know before any of the above kicks off in a serious way: would you want to play a game that had more graphical touches than the game does at present? Would it put you off? Where do you want to see Angband's interface go—or would you rather it stay right where it is?

Thoughts welcomed.

First post! (or: refactoring hell)

So today, I had a quick look at how difficult it would be to get the pref files referencing monsters by name rather than by number, as part of the general effort to remove indexes from the game and make the world friendlier for any budding tileset designers.

The basic sketch should be: make prefs.c output the right file format, convert all the graphics edit files, add a function to get monster race given a name, rename duplicate monsters, and done. Shouldn't take more than an hour.

First step: edit prefs.c to output R: lines with monster names on.

All looks good... oh no it doesn't. I notice lines like:

R:E , the Dark Elf:17:104

That's rubbish, I cry. What's happening here? Well, there's a code module in Angband (to the extent that Angband has code modules) called xchars, which takes strings like "E['o]l the Dark Elf" and turns them either "Eol the Dark Elf" or "Eól the Dark Elf" depending on your platform can support and maybe other factors. In this case, it seems to have failed to do anything useful at all and just be outputting blank spaces.

Why? I do a bit of digging and it turns out that xchars is used when reading anything from any file, so that the monster's name when it's read in from the edit file in the form E['o]l is saved in memory as Eól, in a now-defunct character encoding known as Latin-1.

I assume this has been done to prevent having to make any substantial code changes while gaining a fancy new display feature. A bit like every other feature for the past two decades, then. It would make a lot more sense if the game kept strings in the form they were read in as (with the weird [![?] stuff) and the display code was changed so that some display characters could take up more than one byte in memory.

That would have been the long-sighted, intelligent thing to do that didn't involve just adding another low-level hack that permeates the code. For example, if that had happened, we could make UTF-8 the default encoding of the game with very little difficulty other than wrapping our heads around the Unicode spec and implementing it in C. As it happens, now we have to do that and rewrite the mess that is xchars.

Gragh.

So, in finest Angband tradition, it turns out there's a quick hack around this problem, which is to call xstr_trans(buf, ASCII). That converts a string, in-place, to a new encoding... which presumably may result in a string that is longer than the original string, and since xstr_trans() doesn't take a size for 'buf', could cause overflow and memory bugs. Yay! (There's actually a bare strcpy() in there, I'm not sure how I missed that.)

Looking at the function and the comments above it, to be safe, we need a buffer 1024 bytes in length. OK. We can deal with this if we're strong.

First step, try two: Add xstr_trans() call.

Oh, now the output looks like this:

R:Eol, the Dark Elf:149:214

So we've converted from Latin-1 to ASCII, but not to the 'canonical form' stored in text files. In fact, there is no code to translate from Latin-1 or whatever weird pseudo-native format the game wants to use internally back into the canonical platform-agnostic form that it should have in data files.

Grrraaagh!

So now there are two choices:

1. Add a function that can convert back, temporarily working around the hack and the other hack with a less smelly hack, all in the name of the greater good
2. Fix xchars to happen on display, not on processing.

This is why refactoring Angband takes years. Everything is so entangled that you lose interest before you get to the bottom of the tangle. Or, you ignore the tangle and build stuff on top of it and hope it works.

Also, this lovely comment in x-char.h:

/*
* x-char.c function declarations are located in externs.h.
* x-char-c tables are loctaed in tables.c
*/

What's the point, really?