The Future

On the off-chance you didn't recognize the image from my last post, it was part of an animation from Buried in Time. And now you can find the source for running the game in Cabal over here. And, yes, it's completable.

I'll fill in more details on the blog when I get a chance, but I think trying it out is probably a better way to show off what's been done.

Oh, and I went a little overboard with compatibility on this title. Not only is the 24-bit one working, it has full compatibility with the 8-bit game resources too. (This is the ultimate purpose for that Cinepak dithering I spoke of last year.)

Much thanks to Presto Studios yet again for making all this possible and providing the original source code!

Teaser and the Firecat

Soon...

Revisiting Cinepak

While working on an as-of-yet-unnamed engine last year, I realized I needed to dither some videos. My only hope was that it wouldn't be as painful as DrMcCoy had it several years ago (and I'm pretty sure the "beauty" part was sarcastic). Looking at how the game dithers the graphics, I figured out that it relied on Video for Windows to handle dithering. VFW promptly handles it by making the codec handle it.

For this game, that codec was Cinepak. The Cinepak decoder has been in ScummVM since 2009 (I wrote it in late 2007 or so, so it's really even older). I refused to use some other dithering algorithm that would have been different and given different results. If I was going to implement this, I was going to do it to match the original 100%. That meant it was time to figure out what it does.

Basically, the algorithm is based on pre-dithered tables that are for a given hardcoded palette. For custom palettes, it finds the nearest (using a simple distance equation) color in it and maps from the Cinepak palette index to the custom one. It then uses the pre-dithered tables to generate 4x4 blocks based on the contents of the codebook which is then mapped to the custom palette.

I pushed the code for the curious.

QuickTime also does something similar (but with a different dithering algorithm in Cinepak, of course), which I'll be working on for Myst.

Here's the result, using one of the Cinepak samples from the MPlayer samples archive (in this case, the Lara Croft one):

Normal decode to 24bpp

Dither to 8bpp

The result looks pretty decent. I was mostly glad it wasn't a ridiculous amount of extra code.

Hidden in Plain Sight

With the DVD/GOG version of Pegasus Prime, there was a slight problem before release. We had an invalid function call entering the three new chase sequences when compiled in gcc with optimizations. I was unable to figure out the exact cause at the time and I ended up writing a hack around it in final release.

Since a bad function was getting called, I had feared gcc was overwriting a return address somewhere and sending the program counter where it shouldn't be. valgrind wasn't helping and only showed the after-effects of the bad function call. It was pretty hard to pinpoint in gdb too, due to the calling function being called numerous times during execution without breaking. I had shelved the issue for some time so I could return later, perhaps with another idea of tackling it. I found my hope in the AddressSanitizer.

Armed with my shiny new PC and gcc 4.8.1, I recompiled with the address sanitizer to see what I would get. The game would now crash as soon as the sequence would start, due to the sanitizer kicking in. The information the sanitizer gave helped in really one way: I had a way to make it stop as soon as it broke from the stack buffer overflow error. Perhaps not quite the way the tool was intended to be used, but it was enough of a hint for me.

With some logging to a file, I saw that it crashed here the first time _inputHandler changed. Going with LordHoto's suggestion to check the vtable of the pointer, I noticed something funny: It was the vtable for the wrong class!

Once I saw where the _inputHandler field was populated, I quickly saw what my mistake was. Instead of relying on the compiler to upcast from the subclass to the InputHandler class, I had a manual C-style cast in there. The Neighborhood pointer (only known through forward declaration) was being cast to the InputHandler pointer. Normally this would be OK, as long as the compiler knew about the class hierarchy (in this case, with multiple inheritance and virtual functions), and then generate a static_cast. But if it didn't know that, it would have to go with a reinterpret_cast. The code was doing a reinterpret_cast and throwing away the hierarchy, and therefore causing undefined behavior. It just so happened that it called into the wrong vtable in this case.

But why did it happen only during optimization? Probably because the function was getting inlined. If the include order had Neighborhood defined in the translation unit before getting to the constructor of GameInteraction, it would have output the correct static_cast. It's likely one other place had this situation and that version ended up being the actual used function.

Definitely one of the hardest bugs I've had to track down.

I'm not dead! I feel happy!

It's been so long since the last post that Blogger completely changed their editing interface (OK, not really a joke since Google seems to change their GUI with everything at regular intervals to confuse the public at large).

A few things for this update:

1) The Journeyman Project: Pegasus Prime is now supported in daily builds of ScummVM. Rejoice! And stay tuned for more news!

2) ScummVM April Fools Jokes™ will return next year with a vengeance (blame DrMcCoy, probably).

3) If you're able to recall when posts happened more regularly than yearly, you might remember this post, which I followed up saying I uploaded modified ScummVM code to a branch. Then last year I wrote a small player for the videos so I didn't have to hack directly in ScummVM anymore -- and then added support for Outlaws (proper 1/4 sized raw frames), Mysteries of the Sith, and (partially) Rebel Assault SegaCD.

LordHoto's favorite level (Sega CD version)

Mysteries of the Sith, Level 1 Opening

Oh, and I apologize if your comment didn't show up over the past few months. I screwed up with the admin settings on the blog and it never forwarded me e-mails! Sorry!

Getting Pegasus to Run in ScummVM

The Journeyman Project: Pegasus Prime is very much a Mac game. And by that, I mean it uses the gamut of Mac resources it has available to it. Thus, it can be very hard to extract the data on a non-Mac system. This post is an attempt to document some sane way for everyone to play the game in ScummVM. I'll also update this as people give me more info since some information is not provided by myself. Much thanks to eriktorbjorn for his file list and his Linux extraction script. This is all assuming you can compile the source code at the "pegasus" branch of my github fork already.

Extracting the Full Game Data
Since this is different on different platforms, I shall cover the big three here.

Windows
For Windows you need to use either the HFVExplorer or HFSExplorer tools.

If you're using HFVExplorer, you'll want to dump the files according to this document. If you see "M" on a line, make HFVExplorer extract as MacBinary and as a "raw copy, data fork" for ones beginning with "R". Note that you'll have to rename any file or directory with "/" in the name to have an underscore ("_") in its place.

For HFSExplorer, you can pretty much use the same instructions as HFVExplorer, except that you want to extract AppleDouble instead of MacBinary.

Mac OS X
Extracting the data on a Mac is actually the easiest because a) you can use HFS drives directly and b) you can then run the game directly without changing any file names.

First thing you'll need to do is to enable hidden folders. Then merge the PP Data folders from all four discs into one folder on your hard drive. Any files with the same name are identical. That's it, you're done. You can also use the "macbinary" command line tool to make MacBinary versions of the files.

Linux
You'll need to have hfsutils installed on Linux. Then you should run this script provided by eriktorbjorn with this file list (pasted into a file called "filelist.txt"). It should be pretty self-explanatory to run beyond that, I hope.

Extracting the Demo Data
Since StuffIt is a completely awful tool that won't let you extract any of the resource fork data on a non-Mac system, you will have to wait until I upload a version of the demo that can be played with ScummVM directly. Mac users can use it pretty much right away. I will update this space as soon as possible with more details. Sit tight!

About Face

A quick follow-up to my last post: the code is now online.

Filler Post

While I'm sure you're all waiting for a Pegasus Prime update, something completely useless will have to do in the meantime. Hopefully there will be a real update soon, stayed tuned! You can only get a sneak peek by seeing my tool for allowing games made in ScummVM to be played in the original interpreter again. In the meantime, you can hopefully appreciate the fact that you can now properly hear Gehn singing in Riven.

Some more exciting news: I was able to grab a copy of the ultra-rare Return to Zork Macintosh MPEG edition. Pretty much the same as the regular Mac version (and just as unsupported) but with a new MPEG-2 video layer of fun added on top. One day I'll finish RE'ing all those exotic versions of the game.

Speaking of which, I wrote PSX stream decoder for use in Broken Sword 1 and 2 (and Return to Zork) so now one can play those videos in all their half-resolution glory. I know, you're all very excited!

Switching gears.... For the one of you that may be interested, we did have a third idea for April Fools' this year that was replaced by the C# port joke. I came up with this idea after playing through Gabriel Knight: Sins of the Fathers last year and it manifested itself in a photoshop that I happened to like enough to not just outright toss out.

Originally, there would have been a whole set of "advertisements in ScummVM" -- advertisements placed right into the games! The plan was going to insert a few things in such as making the soda machine in The Secret of Monkey Island a real Coca-Cola machine and using the pre-made "advertisement" of Foster's in a Beneath a Steel Sky.

Only one prototype was made of GK1, and hopefully someone out there gets the joke.