The "Real" Version of Myst

After about 10 years of not owning it, I finally bought a copy of realMyst. The Macintosh version to be precise. Yes, I know it's on GoG, but I'm a tangible sort of guy and like holding the disc in my hand. Also, I don't want to tempt the Wine Gods who have so callously decided to infuriate me (Not the developers, they have put together a fabulous program. I think X11 on Mac OS X just sucks :P). So, I put it in my mostly reliable MacBook Pro and ran the installer. (Recap: MacBook Pro, June 2009, running Snow Leopard, Rosetta is installed.) Everything was going smooth until I tried running the application. It crashed immediately.

The next logical step was to run the application via the console. However this was no standard Mac OS X app. Running it would give the "cannot execute" error. Running "file" on it gave "header for PowerPC PEF executable." That is the old executable format used in previous versions of Mac OS as well as the early Mac OS X versions (10.0 and 10.1). Looking online, I found that there's an app called "LaunchCFMApp" that can launch the old PEF executables.

Of course, Murphy's Law dictates that this can't possibly work. And of course it didn't. It gave me an error: "Launch failed with error code -2856 (cfragRsrcForkErr)" -- that error means that the application can't find the resource fork. I checked the file, and the resource fork is clearly present. I then asked a friend for help. He was able to get it to run, but I still wasn't.

Searching a bit more on Google revealed that there was a 1.1.1 patch for the game. I installed that, and voila! I saw the "launcher" of sorts where I could choose graphics quality, resolution, and color depth. Nothing is ever that easy. Starting it would crash or just lead to a black screen. Until randomly it "just" worked. Probably something screwy going on in the background, but I'll take a functioning game. I was elated as I saw the SunSoft logo followed by the classic Cyan logo.

I proceeded to beat the game in under 5 minutes, as usual. :P

And now the obligatory screenshots:
EDIT: I also visited the new Rime age. The Riven image is a bit weird, not sure why. It's clearly Temple Island, though.

EDIT 2: Turns out LaunchCFMApp works fine in Snow Leopard as long as you feed it an absolute path and not a relative path. Seems the Resource Manager doesn't like relative paths anymore...

QuickTime Resource Forks

The HFS filesystem (and therefore HFS+ too) used by Mac OS and Mac OS X has two "forks" per file: the resource fork and the data fork. Of course, copying those files to non-HFS systems poses a problem. To make matters worse, QuickTime videos also came in versions split between the resource and the data forks! This complicated some things when copying those videos to a non-HFS drive and/or playing in anything but QuickTime player.

So, time for coding! I wrote a program back last November to combine the resource fork ('moov' chunk) and the data fork ('mdat' chunk) into one video file capable of being played everywhere QuickTime is supported. The code only works on Mac OS X (maybe OS 9, I have no means of testing that) for now (eventually I'll add support for the MacBinary format) and uses the data/resource forks directly.

Finally able to view Pegasus Prime videos on Windows... :)

Urban Runner Support: Postmortem

I'm sure you all saw ScummVM's April Fools' Day joke. It's been long enough since the last one (2007!) and last year's never really got anywhere.

This year, LordHoto and I were talking about it back in early February. He had an idea for one involving Urban Runner and DrMcCoy. Strangerke was quickly brought into the "group." LordHoto's grand plan was to plant images of Dr. McCoy into Urban Runner screenshots, which we all liked. He even wanted to make it of the reboot version of Dr. McCoy.

Then, while looking for some test images... I came to a realization. The actor portraying the new Dr. McCoy's name is Karl Urban! Brilliant!

That night, I quickly made the first two screenshots in Photoshop (to the approval of the other two). And then nothing happened again on it until March 30... I created the last two images (based on screenshots Strangerke made) that night.

Beyond our little April Fools' joke, I hope you all saw xkcd's console interface, YouTube's text mode, and the Monolith Action Figure.

And, for an April Fools' throwback, here are ScummVM's past ones:

• Dune 2 support
• Project Dragon
• CABAL (and its cancellation)
  

March Madness

I've been quite busy the last month, though sadly not much related to coding or reverse engineering.

In the ScummVM world, my main focus was working on rewriting the VideoDecoder system. In the current trunk, the VideoDecoder is limited in what it can be used for. It was designed for fixed-rate, 8bpp videos that do not run in the "background" (instead of say in Broken Sword where the cutscene is the only thing going on in the game and the event loop only handles the video) of the games. Because of the way this works, the QuickTime player in Mohawk does not use this (variable rate, >8bpp, and background videos needed) and I designed a different system to run the Myst and Riven videos.

DrMcCoy also had a similar problem with his Dark Seed II engine where >8bpp videos are used (in AVI files). So, I began working on a redesigned VideoDecoder system which took about a week's worth of my spare time up (with much help from DrMcCoy, especially on bug fixing and input on the new API) and we came up with this patch. Since writing the patch I haven't had time to fix some of the things in it or split it up (it's a whopping 130KB), but hopefully I'll get around to that soon. Also, the QuickTime code in Mohawk will eventually also be adapted to that and moved to common so it can be used in the SAGA and SCI engines.

Outside of the ScummVM world, I've done a bit more experimenting with my DOSBox + Windows 3.1 setup mentioned way back when. Now I'm running it at 640x480x24bpp using the S3 Trio drivers, and now Buried in Time doesn't crash on me at all and works in 24bpp. Buried in Time would be a great game to reimplement one of these, however it is entirely hardcoded and would be difficult to implement. The cursors and main menu graphics are stored in the executable too.

As for media, the audio is standard WAVE files (disguised with a .bta extension), the bitmaps are DIB's (disguised with a .btb extension, usually) and the videos are AVI's with Cinepak encoded frames (disguised with a .btv extension, usually). All of the data is normal, except for one image: TITLE/BKG.BTB. I wasn't able to open it in any image viewer, so I opened it in a hex editor only to see a very weird bitmap compression field: 'cvid' ('divc' when decoding because the field is really LE). Cinepak! I hacked together the Mohawk bitmap decoder and the Mohawk Cinepak decoder to display the image, and it came out fine. Apparently, the BKG.BTB file is displayed in the background so the main title video takes up much less room. Interesting, but again, a project that will probably never be realized because of its hardcodedness.

That's it for now... hopefully more posts soon. :)

Uru Live Again!

As you may or may not know, Uru Live opened to the public again on February 11 for the very high price of $0. It felt weird playing Uru again (not that I played Uru Live in the first place, except for a little bit :P). The cleft, and the main for ages of Uru were pretty much the same as the original. A lot came back to me as I played through them.

However, the Path of the Shell ages (Er'cana and Ahnonay for those keeping score) were a bit different. I ran through Ahnonay the same way as I did in the original... only to find myself not being able to rotate pods with the handle in the "hole in the wall." Turns out, they changed the way it's played to require two people...

A fellow ScummVM dev and myself set forth to help each other through the Path of the Shell here. We beat Ahnonay together and then helped each other yet again to find the way to K'veer Island (the island that Atrus is trapped in in Myst/Riven) and by extension, Myst. Here's Scott's and my avatars in Myst:


For what it's worth, my KI# is 590130 and my nick is, of course, clone2727.

Also, for those still keeping tabs on Riven progress, you can now go free Catherine (not that it's possible to get to the 233rd age yet anyway). Interested in trying? Use these console commands to plant you in Gehn's bedroom.

var agehn 4
changeStack ospit 81

Of course, you still have to make your way to Prison Island afterwards ;)

Snow Leopard Woes

Today I took out the Snow Leopard upgrade DVD that I've had for several months and installed it. I figured it was time for an upgrade.

Positives, positives... It didn't crash when installing... It probably is faster in some way. Ok, now for the negatives:

First thing I noticed: Had to recompile all the libraries. There was libSDL and libMAD present, but broken; or at least compiled for the wrong processor. Not to mention libFLAC and libMAD didn't recognize the host, so I had to force it to "x86_64-darwin". Other than that, the 4.2.1 gcc seems to be an improvement over 4.0.1 from the XCode that comes with 10.5.

Second thing: The definition of KB/MB/GB. I hate the 1000 bytes = 1 KB definition and the 1024 bytes = 1 KiB definition. I'm quite disappointed 10.6 now uses this definition.

And finally, the biggest problem: QuickTime X. To put it plainly, QuickTime X is complete crap. I can't believe Apple would release something like this. It is total trash. As when testing any video in a video player, I like to start with Myst's (the original!) Cyan logo video. First off, QuickTime X has almost no codec support; it has to call upon QuickTime 7 to play the videos. This already took about 10 seconds to load the video. Next problem: The video played at some ridiculously high resolution and wouldn't let me play it at normal size. In addition, the whole interface was ugly.

So, I'm looking around, and alas! QuickTime 7 has been pushed back to the Utilities folder. Following some other guys' advice in this post (of people who agree with me), I switched all QuickTime related files to be played with QuickTime 7. That's much better...

On a completely unrelated note: Background videos in Riven work now! Go watch the awesomeness that is the Tay linking book videos! So epic!

eBay Weirdness

Every once in a while, I check on eBay for a few games that I either find interesting, might help ScummVM in some way, or because I'm trying to track one down. I was looking for Mac versions of Sierra games, and at one point I happened to scroll down to the bottom of the screen and something caught my eye. "Leisure Suit Larry" (along with "Lego Batman") was mentioned on the bottom of the screen:


Bewildered as to what that link could possibly mean, I clicked on it and it lead me to here. Apparently Leisure Suit Larry is a popular item on eBay. Or something. I don't really get it. Not to mention none of the products listed below the description are even related to LSL.

</random blog post>

EDIT (2/3/10): My continued eBay searching has left me somewhat confused. Sometimes the item is just ridiculous, sometimes the price is just ridiculous. Since I don't want to just create another post, here are a few oddities I have found:

- King's Quest I CD: In addition to the high price... "replaced with the CD version"
- King's Quest V FM-Towns: High price. I would love to have it to be able to support it though! (Will accept a donation :P)
- Indiana Jones FM-Towns... Phone Card: Self-explanatory
- LucasArts Japanese Mac Pack: I would love to have this too (not as much as KQ5 FM-Towns), but I don't want to spend this kind of money...
- Indiana Jones and the Last Crusade: Seems normal... until you see what it comes with.

And, the icing on the cake:

- King's Quest VI "Girl in the Tower" CD: My jaw dropped when I saw this one. Yours probably will too. The price pretty much tells all. But it has free shipping!!!!


In other news, I've been tinkering with the Mohawk QuickTime code and working on adapting it to the Graphics::VideoDecoder interface in ScummVM. The more I do this, the more it feels like shoehorning the features that QTPlayer has into VideoDecoder isn't working. In addition, md5 mentioned that IHNM Mac uses QuickTime MIDI files. Curious as to what that involved, I looked up some samples on the internet. They use the good bad ol' QuickTime format to hold its own MIDI format (I had hoped for SMF in a QuickTime wrapper). Here's what Apple's QuickTime specs said about it. Anyone out there have better info than that?

The Year We Make Contact

And so, 2009 comes to a close. For those of you who have a good memory, you might remember a post I made last year about what I had hoped to accomplish this year. Let's take a look:

2009 Goals: (In order of probable completion)
- Fix Cinepak decoder!
- Fix QuickTime IMA ADPCM!
- Get background videos working!
- Figure out image format for other Mohawk games (Current hunch: LZSS similar to Myst format)!
- Finish Riven support (aka all the hardcoded puzzles :P)!

I have crossed out what was accomplished. Technically, I have a WIP patch that allows for background videos to work in Riven and background videos already work in Myst. So, we might consider that one done, even on a bit of a stretch. :P And now for next year:

2010 Goals: (In order of probable completion)
- Finish Riven support (external commands, mostly)
- Finish Myst support (some resource fixes, and then lots of the hardcoded opcodes)
- A look into other games more

Again setting lofty goals! We may even consider them monolithic in size! (Pun intended!) And for one last treat, to see the "good" ending of Riven, open the debug console in ScummVM (ctrl+d) and use these commands:

var ttelevalve 1
var ttelepin 1
var ttelecover 1
var pcage 2
changeStack tspit 137

Turn the lever down and press the button and voila! (Note: The DVD version may have to use a different card number, but just walk over to the telescope if you end up elsewhere :P)

Happy New Year! :)

The End

Today, December 29, 2009 at 23:23:31 (UTC), Mohawk entered the main ScummVM tree. You can now go grab a daily build and try out Myst or Riven. And now Strangerke can make his own gameplay videos and not coerce me into making them. :P

So, this is the end. Mohawk is now in the trunk and everyone can attempt to play Myst and Riven in ScummVM. :)










Or, perhaps the ending has not yet been written...

Sync'd (aka December Update)

After about a year of work, videos finally work and now with full a/v sync (md5 suggested something that could be the problem and that was it). Of course it was about a five line fix, but that's the not the point. ;)

Mohawk progress in the past month, let's see... I don't really remember a lot of it. I've spent most of my time on Uni things. It's now break, so it's time to work on more! digitall has been working on QDesign Music 2 (a complex audio codec used in Myst ME) and then we've taken an early look into the Living Books games. Nothing playable yet, but we can parse the "book outline" files.

I'm sure there's more I've forgotten, but I'll give a quick wrap-up:

Myst: Playable
- Many graphical glitches
- Wrongly disabled hotspots
- Only Stoneship is accessible from Myst Island
- D'ni is accessible! Can't hand a page over though :P

Myst ME: Slightly less playable than Myst (inherits the same problems)
- Less videos work
- Probably some PICT problems that have yet to be encountered

Riven: Very playable
- Almost no status update since last time :P
- Fixed one minor regression from cleanup

Myst is movin' right along now! Unfortunately, both Myst and Riven have hardcoded puzzles which are the biggest impediment in progressing further. But fear not, they will be worked on!

Legacy of Time 10th Anniversary

Earlier this year, Tommy Yune, conceptual artist on most of the Journeyman Project series posted on his Journeyman Project blog that Presto Studios would be re-releasing Legacy of Time for Mac OS X as a 10th Anniversary Special Edition. This was put on hold after the evil QuickTime 7.6/7.6.1 bugs that caused Cinepak videos to not display properly.

On October 30, the game was available to order. I ordered as soon as I possibly could. And, it came yesterday. I played as normal, and as usual, pretty much everything came back to me about playing the game (I'm not done yet, but I did find two parts of the legacy already!).

One of the interviews mentioned there was a new easter egg in the game. It obviously caught my attention. Now, instead of looking through the game for the new one, I set off looking at the data files. Most of the files seem the same from the original PC DVD version (from 1998), but some of the dates are different. For instance, the final cutscene has a date from January of this year, so it was probably modified a bit.

Then, I was looking through some of the El Dorado videos, and lo and behold, a video had a date from 3/13/08. It has been touched since the original release! Looking at it, it's the video with the pool animations (EP/EPanim.mov). At the end of the video is the new easter egg involving a whack-a-rat type joke with the idol heads around the pedestal with the legacy. I haven't figured out how to trigger it in game yet, but that is the next step.

Also, while the interviews seem to suggest improved graphics, the files all have the same 6/25/1998 date (minus those couple videos, unless I missed some), so the files all have the original DVD release's graphics and they haven't been updated since then.

Easter eggs cannot hide from me! :P

The ScummVM Logo: A Lookback

If you didn't notice, ScummVM has an updated logo. So, in honor of that (even though it's a week late), I've taken the liberty of compiling all the ScummVM logos throughout history. Much thanks to archive.org. I just thought it was interesting how far it has evolved. Of course, I came much later and missed out on the old logos... (And, does anyone remember that first one?)

Ancient (2001):



Early 2002: (Website Redesign)



Late 2002:



Late 2005: (Website Redesign)



Late 2009: (Capitalization)

November Already!

Here we are. November. And when was the last blog post? A month and half ago? Bah! It's been too long!

Riven has improved a lot. Loads of hardcoded functions have been filled in, it's slowly becoming more completable. The marble puzzle is just bypassed currently because I haven't gotten around to its logic. So, you can now get up to trying to access one of the lesser domes. And, if you use the debug console, the game is actually completable. Whaddya know?!? Some other bugs (including another QuickTime scaling bug! Egad!) were fixed.

That's great and all. But, really, the biggest piece of progress has been in Myst. Around the time of my last post, a new developer came forward (digitall) and has been working on various portions of Myst bringing it up to a highly playable state. Myst is now explorable and Stoneship is also accessible from the start. Screenshots? Of course! Videos? No!

QuickTime Hell

I hate QuickTime videos. They really are over-engineered. In good news, I have conquered most bugs in our player (mostly based off the ffmpeg player, thank you so much!) but there are still several evil ones left. The last of these screenshots will show the main bug left:

The first three are from the Cyan logo video, the main intro video on D'ni, and then Cho greeting you on Riven (before his timely demise). The last is the door to the temple. Yes, that's a bug. Or, rather an unhandled feature of QuickTime videos (it's in ffmpeg, just not in ours :P) that requires us to cut the width and height by 2. Only a few movies have this, and I'm not sure why. It doesn't make sense to me to even double the resolution in the first place, but alas, it's on the TODO list now. As are the "background" videos which will probably be handled next (they're partially supported now, just need to hook in a few function calls and cut out some screen updates). Not to mention the a/v sync problem still. And, yes, gameplay videos will be coming, just please don't bug me about it now.

EDIT: Fixed the double-resolution stuff (and 4x res stuff too... stupid QuickTime...):

For those interested, the problem was in the display matrix (in the tkhd atom) we didn't handle previously.

Boldly Going

Taking a break from my usual Myst/Riven/Mohawk posts, this one will be different. Back in November of 2008, I took a brief look into the engine behind Star Trek: 25th Anniversary and Star Trek: Judgment Rites. Most of the work was based off of the Xentax Wiki information (as well as other pages on the site). Before long, I had the basic file parser going and then followed by images. I even went to support the MIDI sound and figured out the font format of the PC version before moving onto writing code for the file format of the Amiga and Macintosh formats (which really weren't all that different).

However, I didn't really want to continue to work on it after finding out that the game had some heavily hardcoded parts. And, it just sat on my hard drive for many months... until now. I'm putting the code that I have (slightly cleaned up today and made to compile with the current ScummVM trunk) onto a new Google Code repository. I hope that someone will want to take up the project and continue it to completion. I don't have any plans to work on it myself, currently. If any ScummVM forum dweller remembers, there was a brief mention of another ScummVM engine for the games back in June of this year. As far as I know, nothing ever came out of it. If anything did, I'm sure that probably some of my code could be salvaged if only for the Mac/Amiga code.

Zip-a-Dee-Doo-Dah

Any of you out there who have played Myst or Riven know that the only thing that could possibly relate to that title in those games is Zip Mode.... which is now supported in Riven in the engine. :)

As for videos, I *did* make an attempt to make some yesterday. I actually made one, and then the program for capturing crashed on me when saving. I even beat the "Rebel Tunnel Puzzle" from memory. If anyone out there knows of a good program for capturing video on Mac OS X, leave a note in the comments. :P

Continuing Updates

After a long time coming, the Myst demo has finally been located and secured. I found it on a CD-ROM Today demo disc from July, 1994. Talk about a blast from the past. I quickly started it up in Mohawk (after fiddling around with the fs stuff on the hybrid disc on OS X). The game is about as playable as the full version of Myst. You can access the menu and click on things before an error, while the full version just doesn't do anything to leave the intro. :P

Interesting that it hadn't been found earlier. I had been searching for almost two years for the demo. The Riven demo I found was on the SoftKey release of the Prince of Persia Collector's Edition. Seems Broderbund/Red Orb like to hide their demos fairly well (On a side note, reverse the first six letters of Broderbund for a neat trick :P).

Are you still looking for a Riven gameplay video (*coughStrangerkecough*)? Well.... there is none, because this is all part of

More of a Real Update (Kind of)

Seems everyone is expecting more updates from me, so here's a better one than the last one (which, if you didn't realize, was more of a joke than anything to spite Strangerke who has been pestering me for more blog updates. Besides, we should pester DrMcCoy who hasn't posted in two and a half months :P).

So, here goes. The last post showed off the main things that I had worked on in Mohawk for the past month-ish. Video is still getting better, sync slightly improved (works near perfectly in Myst, worse in Riven) and improved streaming of audio from QuickTime videos. The sync gets worse the longer the video is, but I have a solution to it that I will be implementing soon. Note that only Riven's intro videos currently play, there's still problems with other ones. Riven is indeed playable up to the marble puzzle, which as you might know, is one of the more complex puzzles in the game. I had a script parsing bug left in Riven since who knows when that I finally fixed and finally finished the boiler puzzle. It has so far been just a huge annoyance in working on the game. Hopefully, I won't have as much grief with the marble puzzle.... :P

Indeed the Making of Myst is supported. It's not much of a game, but it has some of the basics of the Myst flavor (yes, ST, flavor!) of Mohawk. It uses one resource/hotspot to show the video, which is now handled. Myst still is not playable beyond the intro.

Hopefully, if you've noticed the trunk, I also had a little SCI action going. Namely (very very very) early SCI32 support at the begging of DrMcCoy (who still hasn't fixed the Urban Runner bug!). I also added AVI/MSVideo1 support to the Windows version of SCI (specifically KQ6 in this case). The actual player I wrote for a dead-ish project/engine of mine that I started a while back, but never got off the ground due to it being completely hardcoded (as opposed to Riven's more complex puzzles only being hardcoded). Maybe after Mohawk (and other things) I'll get back to it. Riven is still my priority, don't worry! It was nice to take a short break and look at an engine that has next to no hardcoded-ness for once. In case you were wondering about SCI32's status, there really is nothing to report. At least GK1 and Torin start their scripts before having a crash with something or other (due to kernel functions not being implemented).

As promised for months and months and months, there will be a new gameplay video. I hope to get it up on YouTube on TUESDAY, so stop bugging me about it! :P

Update for Strangerke

Strangerke has demanded an update from me, so here it is. I fixed some Riven bugs and now the boiler puzzle is completable. The game is playable until the marble puzzle. Added support for changing Myst cursors and the Making of Myst. There were also various video fixes and support for audio support in Riven videos. Riven now shows the intro videos (though slightly out-of-sync). Expect gameplay videos soon.

That is all for now. :P

Taking Inventory

As of yesterday, Riven has full inventory support. Yes, that means you can access Atrus' Journal, Catherine's Journal, and the trap book. Yes, that also means that the first "end" to the game is now accessible. The first two actually. You can now die by using the trap book on Riven or on Tay. Two out of like 20. Not bad.

Images of course!