Sunday, August 17, 2014

Revisiting Cinepak

While working on an as-of-yet-unnamed engine last year, I realized I needed to dither some videos. My only hope was that it wouldn't be as painful as DrMcCoy had it several years ago (and I'm pretty sure the "beauty" part was sarcastic). Looking at how the game dithers the graphics, I figured out that it relied on Video for Windows to handle dithering. VFW promptly handles it by making the codec handle it.

For this game, that codec was Cinepak. The Cinepak decoder has been in ScummVM since 2009 (I wrote it in late 2007 or so, so it's really even older). I refused to use some other dithering algorithm that would have been different and given different results. If I was going to implement this, I was going to do it to match the original 100%. That meant it was time to figure out what it does.

Basically, the algorithm is based on pre-dithered tables that are for a given hardcoded palette. For custom palettes, it finds the nearest (using a simple distance equation) color in it and maps from the Cinepak palette index to the custom one. It then uses the pre-dithered tables to generate 4x4 blocks based on the contents of the codebook which is then mapped to the custom palette.

I pushed the code for the curious.

QuickTime also does something similar (but with a different dithering algorithm in Cinepak, of course), which I'll be working on for Myst.

Here's the result, using one of the Cinepak samples from the MPlayer samples archive (in this case, the Lara Croft one):

Normal decode to 24bpp
Dither to 8bpp

The result looks pretty decent. I was mostly glad it wasn't a ridiculous amount of extra code.

Saturday, August 16, 2014

Hidden in Plain Sight

With the DVD/GOG version of Pegasus Prime, there was a slight problem before release. We had an invalid function call entering the three new chase sequences when compiled in gcc with optimizations. I was unable to figure out the exact cause at the time and I ended up writing a hack around it in final release.

Since a bad function was getting called, I had feared gcc was overwriting a return address somewhere and sending the program counter where it shouldn't be. valgrind wasn't helping and only showed the after-effects of the bad function call. It was pretty hard to pinpoint in gdb too, due to the calling function being called numerous times during execution without breaking. I had shelved the issue for some time so I could return later, perhaps with another idea of tackling it. I found my hope in the AddressSanitizer.

Armed with my shiny new PC and gcc 4.8.1, I recompiled with the address sanitizer to see what I would get. The game would now crash as soon as the sequence would start, due to the sanitizer kicking in. The information the sanitizer gave helped in really one way: I had a way to make it stop as soon as it broke from the stack buffer overflow error. Perhaps not quite the way the tool was intended to be used, but it was enough of a hint for me.

With some logging to a file, I saw that it crashed here the first time _inputHandler changed. Going with LordHoto's suggestion to check the vtable of the pointer, I noticed something funny: It was the vtable for the wrong class!

Once I saw where the _inputHandler field was populated, I quickly saw what my mistake was. Instead of relying on the compiler to upcast from the subclass to the InputHandler class, I had a manual C-style cast in there. The Neighborhood pointer (only known through forward declaration) was being cast to the InputHandler pointer. Normally this would be OK, as long as the compiler knew about the class hierarchy (in this case, with multiple inheritance and virtual functions), and then generate a static_cast. But if it didn't know that, it would have to go with a reinterpret_cast. The code was doing a reinterpret_cast and throwing away the hierarchy, and therefore causing undefined behavior. It just so happened that it called into the wrong vtable in this case.

But why did it happen only during optimization? Probably because the function was getting inlined. If the include order had Neighborhood defined in the translation unit before getting to the constructor of GameInteraction, it would have output the correct static_cast. It's likely one other place had this situation and that version ended up being the actual used function.

Definitely one of the hardest bugs I've had to track down.

Tuesday, April 02, 2013

I'm not dead! I feel happy!

It's been so long since the last post that Blogger completely changed their editing interface (OK, not really a joke since Google seems to change their GUI with everything at regular intervals to confuse the public at large).

A few things for this update:

1) The Journeyman Project: Pegasus Prime is now supported in daily builds of ScummVM. Rejoice! And stay tuned for more news!

2) ScummVM April Fools Jokes™ will return next year with a vengeance (blame DrMcCoy, probably).

3) If you're able to recall when posts happened more regularly than yearly, you might remember this post, which I followed up saying I uploaded modified ScummVM code to a branch. Then last year I wrote a small player for the videos so I didn't have to hack directly in ScummVM anymore -- and then added support for Outlaws (proper 1/4 sized raw frames), Mysteries of the Sith, and (partially) Rebel Assault SegaCD.

LordHoto's favorite level (Sega CD version)
Mysteries of the Sith, Level 1 Opening
Oh, and I apologize if your comment didn't show up over the past few months. I screwed up with the admin settings on the blog and it never forwarded me e-mails! Sorry!

Thursday, April 12, 2012

Getting Pegasus to Run in ScummVM

The Journeyman Project: Pegasus Prime is very much a Mac game. And by that, I mean it uses the gamut of Mac resources it has available to it. Thus, it can be very hard to extract the data on a non-Mac system. This post is an attempt to document some sane way for everyone to play the game in ScummVM. I'll also update this as people give me more info since some information is not provided by myself. Much thanks to eriktorbjorn for his file list and his Linux extraction script. This is all assuming you can compile the source code at the "pegasus" branch of my github fork already.

Extracting the Full Game Data
Since this is different on different platforms, I shall cover the big three here.

For Windows you need to use either the HFVExplorer or HFSExplorer tools.

If you're using HFVExplorer, you'll want to dump the files according to this document. If you see "M" on a line, make HFVExplorer extract as MacBinary and as a "raw copy, data fork" for ones beginning with "R". Note that you'll have to rename any file or directory with "/" in the name to have an underscore ("_") in its place.

For HFSExplorer, you can pretty much use the same instructions as HFVExplorer, except that you want to extract AppleDouble instead of MacBinary.

Mac OS X
Extracting the data on a Mac is actually the easiest because a) you can use HFS drives directly and b) you can then run the game directly without changing any file names.

First thing you'll need to do is to enable hidden folders. Then merge the PP Data folders from all four discs into one folder on your hard drive. Any files with the same name are identical. That's it, you're done. You can also use the "macbinary" command line tool to make MacBinary versions of the files.

You'll need to have hfsutils installed on Linux. Then you should run this script provided by eriktorbjorn with this file list (pasted into a file called "filelist.txt"). It should be pretty self-explanatory to run beyond that, I hope.

Extracting the Demo Data
Since StuffIt is a completely awful tool that won't let you extract any of the resource fork data on a non-Mac system, you will have to wait until I upload a version of the demo that can be played with ScummVM directly. Mac users can use it pretty much right away. I will update this space as soon as possible with more details. Sit tight!

Monday, April 02, 2012

About Face

A quick follow-up to my last post: the code is now online.

Filler Post

While I'm sure you're all waiting for a Pegasus Prime update, something completely useless will have to do in the meantime. Hopefully there will be a real update soon, stayed tuned! You can only get a sneak peek by seeing my tool for allowing games made in ScummVM to be played in the original interpreter again. In the meantime, you can hopefully appreciate the fact that you can now properly hear Gehn singing in Riven.

Some more exciting news: I was able to grab a copy of the ultra-rare Return to Zork Macintosh MPEG edition. Pretty much the same as the regular Mac version (and just as unsupported) but with a new MPEG-2 video layer of fun added on top. One day I'll finish RE'ing all those exotic versions of the game.

Speaking of which, I wrote PSX stream decoder for use in Broken Sword 1 and 2 (and Return to Zork) so now one can play those videos in all their half-resolution glory. I know, you're all very excited!

Switching gears.... For the one of you that may be interested, we did have a third idea for April Fools' this year that was replaced by the C# port joke. I came up with this idea after playing through Gabriel Knight: Sins of the Fathers last year and it manifested itself in a photoshop that I happened to like enough to not just outright toss out.

Originally, there would have been a whole set of "advertisements in ScummVM" -- advertisements placed right into the games! The plan was going to insert a few things in such as making the soda machine in The Secret of Monkey Island a real Coca-Cola machine and using the pre-made "advertisement" of Foster's in a Beneath a Steel Sky.

Only one prototype was made of GK1, and hopefully someone out there gets the joke.

Thursday, December 15, 2011


Well, hello there. Been a few months since I posted here. Figured I'd post about something, y'know.

Had a busy summer... busy fall too. Played a few games recently, including one classic game -- The Journeyman Project: Pegasus Prime. Let's reminisce a bit (WARNING: Contains spoilers; just skip to the very bottom for the good stuff).

The essential title screen; usually in most games. Let's move onto gameplay.

We should probably wake up, and not be late for work.

Too late! The commissioner doesn't look too happy. Also, bad temporal rip incoming and whatnot.

Time to go recover that log and not jump off the cliff though!

Avoid dinosaurs too, always good. Let's go to Mars instead.

Watch out! (Aside: For fun, you can try to attack this guy with any inventory item -- I bet you didn't know that!)

Freeze the lock, let's take a look at that bomb. Don't just pry it out though! Now, let's get out of here.

The maze isn't the same without the original music, but it's still enjoyable. Let's go catch that robot!

Bang bang! Pew pew! Now, tractor her in and board her!

Need to stop a robot down under too.

Being under the sea more to your liking? This guy doesn't seem to think you're much of a match for him, though. Probably should stay out of his way or maybe crush him or something.

I hope you don't need a map to solve this one.

Watch out! Robot on the loose!

Who needs lockpicks? Let's just blow the door open.

Why are there no tasers in the future?

How come James Bond never had to go through this level of difficulty when diffusing a bomb?

Mission Accomplished!

Well, that was a fun little aside don't you think! Talking about a classic adventure game randomly, just like that. Or was it really random? Maybe there's something common about those screenshots; no, not just in PNG format. They're all taken from within ScummVM. Yeah, you heard correctly. This is definitely happening, and I'll hopefully be done fairly soon. Ah, now you see what I'm getting at? There's still some graphical glitches with the videos, but it's in very good shape overall.

Oh and it's even completable.

More updates to come soon (And no, this code isn't on my github account yet, but will be soon).

And, of course, much thanks to Presto Studios for making all this possible and providing the original source code!

Friday, June 17, 2011

Mac SCI1.1+ Games Update

I really should update here more :P

In an effort to catch up with stuff I've missed, I'll start by going over stuff I coded a few months ago. Remember last year when I wrote up two posts about SCI Mac code? Well, I don't. So, here's two links from last summer.

Back in probably February, waltervn and myself tackled the resource compression of the non-King's Quest VI SCI1.1+ Mac games. King's Quest VI wasn't good enough to receive the compression of the other games. Then we found out that they changed the sprite compression routine again (runs now a word instead of a byte) for QFG1 VGA Mac. The Mac versions of Hoyle Classic Card Games and Freddy Pharkas became playable too, especially after waltervn put in a bunch of work on the hardcoded menu bar of KQ6 and Freddy Mac (though, still not complete).

One of the problems we found with these Mac games is that they have their black and white palette indexes reversed from normal SCI (black = 0x00, white = 0xff). Why? Because on classic Mac OS, white is always at 0x00 and black is always at 0xff. Our solution was to swap black and white pixels so that our palette code from the DOS versions would still work and would keep the complexity down.

Then, I took a look at the SCI2.1 Mac situation (GK1 Mac is SCI2.1 instead of SCI2 like the DOS version). The four SCI1.1 Mac games began to switch various resources to store multi-byte values in big endian order instead of the little endian order of their DOS counterparts. Starting with GK1, Sierra began to, as I like to put it, "big-endian-ify" the remaining resources. I also added support for the high-res GK1 Mac fonts that the DOS version doesn't have. It currently is probably on the same page of support as the DOS version (minus the sound driver and one skippable timing bug in the Day 1 intro). Gabriel Knight 2 Mac also has some small amount of support, but with some more weird transparency issues. The later SCI2.1 Mac games have some Sound class changes (the SCI class, that is) that causes games like Phantasmagoria to not start without a couple hacks.

Going back even further, there's three SCI0 Mac games that we don't support yet either: Space Quest III, Hoyle's Book of Games I and Hoyle's Book of Games II. These three are special in that they run in a higher resolution than the usual 320x200 and have high-res views.

And now for some screenshots:

Clicking on the KQ6 menu

Looked at that awesome greyed out effect!

GK1 high-res fonts = awesome

Yeah, it's Mostly working (I am the king of lame puns)