gekk.info « articles

The Unholy Saga of Phoenix Hyperspace

Please note: I wrote this as an "infotainment" post for social media, and thus it is written with a substantially spicier style than most of my website. I felt that this topic is sufficiently absurd that removing the cussing took the punch out of it; I was genuinely going feral as I performed the research for this article, and it's reflected in my narration. You've been warned.

I have long wondered exactly what it's like working at American Megatrends, Phoenix, or Award (before they were acquired.) As far as i can tell, these companies existed for nearly 40 years as nothing more than long-tail IP trolls: they each made an IBM BIOS clone in the 80s, then continued selling them for decades while adding - seemingly - very, very little

Mostly, these companies existed so that motherboard manufacturers could buy 90% of a BIOS premade, then add the bare minimum customizations for their specific board. This is why so many CMOS setup programs have horrendous typos - they were probably customized by a contractor hired for one week.

Sure, someone working for these corporate entities did Real Work, but that was in, like, 1985. It was probably one guy, and he's not with the company anymore. Whoever was working there in 1996 or 2006 was absolutely profit-squatting - the work done to create their BIOS clones was long, long, long done, with minor exceptions for shit like Y2k compliance updates. They were just siphoning off of its continued necessity for the PC clone market.

So, did they actually employ any developers? Or salespeople? How weird must that job have been? Sitting around in a silent building, no moving and shaking going on, no Deals, just being there out of sheer necessity.

Now, AMI and Award did a bit of dabbling in other markets, but not much. We're talking, like, POST diagnostic cards, and that was in the 80s when there were literally hundreds of companies inexplicably competing in spaces like that. Otherwise, as far as I know, all of these companies had no purpose in existing other than to profit off of a few kilobytes of code that some past employee wrote.

Despite this, in the late 2000s, Phoenix apparently decided they were "failing." How can that be possible? my guess: it wasn't. They'd long since bought Award, meaning there were only two suppliers of BIOS' for the entire clone market: themselves and AMI. I'm positive the market was evenly split between the two, because it simply had to be. So, it is beyond belief that Phoenix could have been unprofitable, given that they were performing no work, required no employees, produced no product, and made literally 100% profit simply by existing.

None of the above has ever mattered to corporations however, so presumably what they meant by "failing" was "not infinitely growing," because capitalism is so brainwormed that it can't accept "making tons of money, forever, for zero work" as a definition of success. And thus, they had to make some Changes.

They hired a new CEO to, uh, "save" them. This always goes the same way: the guy comes in and says "the way you should save this company is by becoming a completely different company. easy peasy. and of course, by 'become', i mean 'buy'." So they proceeded to buy a bunch of dumb crap, including a fucking driver finder app. Yes, as in, "a program you run that supposedly finds driver updates." They tried to "save" their "failing" company by buying and distributing malware.

But what's even wilder than this "fry's electronics begins selling Bazic-brand post-its and counterfeit perfume" style heel turn was that they also began to develop new software, in-house. And, I mean, really, genuinely ambitious software.

One of their products, FailSafe, was apparently a firmware-based tracking app for recovering stolen PCs. I'm guessing that never really worked well, but it's at least novel! The idea that you can't knock it out by just wiping the hard drive seems cool! And I think they actually wrote this from scratch!

Another product, however, was so ambitious that it actually kind of stops you in your tracks. You go, "wait, what" like you heard wrong. They can't possibly have meant what you just heard, right? Because what you heard was:

They developed a consumer-grade hypervisor.

Called Phoenix Hyperspace, it looked like nothing so much as yet another entry in the the then-popular (and also then-reviled-and-ignored) push to find a way to make computers boot faster. There were several companies trying to do this at the time, and it is a far less interesting subject in general than Hyperspace - or, at least, I thought that when I initially wrote this treatise. Since then, I've invested a lot of time into this particular niche of software history.

As part of that investigation, I managed to obtain a machine with Phoenix Hyperspace, and was shocked to discover that, as batshit as the previous statement was, that product (which never really got shipped in anything, AFAIK) doesn't hold a candle to the weirdness of what actually made it into consumer hands. Phoenix invented a completely novel way to abuse the BIOS, and trust me, this article is worth the read if that intrigues you.

First: Instant-On Linux

Let's bottom-line a set of problems that existed in 2009:

Due to all of the above, it was perceived by vendors that consumers were being forced to shut their computers down and start them back up way too often, and wasting too much time doing that. It was believed that they would appreciate the ability to boot their machine directly into an alternate, lightweight OS, one that would be able to cold-boot more quickly than Windows. Obviously, this was Linux, because what alternative was there?

The standard bearer for this "industry" was Splashtop Linux. You might recognize the screenshot above, because Asus shipped it on a number of motherboards and EeePCs. It was developed by a company called DeviceVM, who later rebranded entirely to Splashtop, although their primary product is now a remote desktop product (that's pretty good imo; they did not pay me to say this.)

Splashtop OS was an extremely pared down Linux distro with a web browser, email client, chat client, and very limited photo album and media player apps. They didn't sell it directly; vendors whitelabeled it and embedded it in machines. Asus' was called ExpressGate, Lenovo called it Quick Start, HP called it QuickWeb, and so on.

Above is Lenovo's Quick Start variant. Besides those basic programs (GNU packages, lightly modified to make it impossible to break out and use the underlying OS) you can see other "apps" on the quickstart bar: Gmail, Youtube, banking, and a Games center. Unfortunately, these are all just links to websites.

Splashtop was intended for extremely nontechnical users, to put it mildly, so it was virtually useless to anyone reading this page even when new, but since almost the entirety of its capabilities consists of opening webpages, and the embedded browser is stuck on TLS 1.0 forever, Splashtop is now utterly useless to everyone.

Now, to be clear, Splashtop is absolutely nothing special. It seems intriguing at first, because most of the machines it shipped on have a second power button that causes the machine to boot into the instant-on environment instead of Windows; and because it really does boot very, very fast. On my Lenovo S10-3, it genuinely starts up in less than 3 seconds, off an Atom and (I think) a spinning disk. It's very impressive, but there is no mystery or magic.

All this is is a Linux distro set up to dual-boot with Windows, and pared down so it boots quickly and isn't confusing to people without Linux skills. This, of course, isn't even true - the proverbial "grandma" would be absolutely lost in this UI, and since there's no support for Macromedia/Adobe Flash, half the web in 2009 would have been unusable. It was never a good idea, but it stuck around for a surprisingly long time; Asus shipped Splashtop on motherboards into at least the second or third gen Core i era, if not longer.

Splashtop was not the only shot taken at the problem. Another offering from Dell was called Latitude-ON, based on MontaVista Linux, a distribution best known for being used in a shitload of horrible Linux-powered devices, I think set top boxes and the like. I've written a lot more about it here, but suffice to say, it's just primitive. I could write 10,000 words about what Dell could have done with this but inexplicably didn't.

So, these things were all dumb wastes of time that nobody cared about and nobody used. After two or three years they disappeared and nobody even remembered they existed. Hyperspace was a whole different can of worms.

Dabbling in Hyperspace

I wrote this part of the article before I ever actually saw Hyperspace in person; this is important, so I'm leaving it as-is.

Phoenix Hyperspace was another pared-down Linux distro meant for instant start. As far as I can tell, it had more real apps - including an actual productivity suite - but was otherwise the same basic concept, a very lightweight OS. Hidden underneath this offering, however, was a completely outrageous implementation.

Instead of dual-booting like Splashtop did, Hyperspace lived in a VM. When you installed it, your MBR was adjusted to launch a hypervisor (Xen-based, as far as I can tell) on boot, which then launched Hyperspace and Windows 7 in VMs. You could then jump between them at any time, in about a second. The theory was that you'd use this to actively manage your battery consumption: when you're doing "heavy" tasks, use Windows. When you're ready to Kick Back, Relax And Do Nothing Of Consequence, jump into Hyperspace. When you switch to the Phoenix VM, the Windows one gets suspended, and your TDP (supposedly) plummets.

But what intrigued me even more when reading about this was that, reportedly, it could do data exchange between the VMs. You could, ostensibly, create a Powerpoint presentation in Windows, then switch to Hyperspace and open it. And if you downloaded a file that nothing in Hyperspace could open, it could immediately switch to Windows and open it with the default app. These are incredible concepts. Whether any of it worked, let alone well... I couldn't determine.

It's extremely hard to find a machine with Hyperspace. The only respectable device ever sold with it was the Lenovo Thinkpad X1 Hybrid, which nobody seems to have anymore. They were sold, there are spare parts on ebay, but I can't find any machines. I did learn that Samsung had shipped it on some systems however - a few horrible little netbooks, including one called the N210 which I bought on eBay, hoping to learn the mysteries of Hyperspace.

In the meantime, I continued my research and learned that they had actually offered a trial version, and it was fucking SaaS. Yeah, you could buy Hyperspace as a standalone product, supposedly, with an ANNUAL SUBSCRIPTION. That was pretty novel malfeasance for 2009! But, while I was able to find the trial version installer, it phoned home for the download to servers that are long gone - plus, there were multiple SKU levels, and I doubt the trial version implemented the full hypervisor.

What a goddamn moonshot, though! Especially for a company that had been doing absolutely nothing for 20 years, as far as I know. I can't find any evidence that Phoenix bought any part of this; it seems like they actually developed it, themselves, possibly from whole cloth. That's ballsy! Sure, a company like Apple could get away with something this bold, but... nobody else could.

Now, before continuing, I should note that, given 20/20 hindsight, we can see clearly that this new CEO was a total dry hump and all his ideas were stupid and unnecessary. All his initiatives evaporated within two years, they sold off all their "products" for less than they spent to buy / develop them, and they're back to having no product other than clone firmware. And, wouldn't you know it, they're still around. They developed an EFI firmware, which they've now been milking for a decade, but they didn't go out of business and I'm positive they were never unprofitable for a second.

This entire thing was a huge, stupid waste of time. exactly the same as every other time in history that a CEO has been hired to "fix" a company with which there was nothing wrong. you can set your watch by this shit.

But all of the above is the purely theoretical. A week after I wrote it, my N210 arrived, and I found that Phoenix followed even darker paths than I had thought.

Press Release Meets Reality

As with most netbooks, it's pretty austere. 1024x600 screen, Atom processor. The best thing I can say about it is that the battery still lasts several hours, and honestly I suspect it would be "sorta usable" if I put 2GB of RAM in it (it has 1, and I bet that's how it shipped.) It will never play an HD video. It came with Windows 7 Starter, typical for the time (late 2009) and...

...Phoenix Hyperspace "Instant-On," which I think was Samsung's specific branding.

Finding this machine was crucial. The version of Hyperspace included by OEMs was not SaaS, of course; it carried an infinite license as one would expect. And as far as I can tell, nobody shipped this except Samsung, in just four machines: the N210, N220, and two others I forgot. I couldn't find any of the others on eBay, so, it was either this, or never finding out what this software was all abot.

Naturally, the machine was full of viruses.

Computer over? Virus = very yes? That's not a very good prize.

I ran the Samsung recovery wizard, it wiped the OS and replaced it with an unvirused one, and the machine seems fine now. Most crucially, I was able to run the Hyperspace installer obtained from Asus' website, and it installed!

Hyperspace is not meant to be "Linux" per se. It's a kiosk-style environment, so when it boots, you're put straight into a fixed-function user interface. It boots to a configurable, widget-based "portal" type of thing, and while the idea that anyone ever wanted an "instant view of many things" like this is suspect to me, it's honestly not a bad specimen, if I'm honest. In fact, some of it even still works.

Those are current headlines being pulled from the BBC! I guess they haven't changed their API in 14 years (it's probably RSS.)

This is very "static": It's a tile-based interface, to which you can remove or add widgets, and that's it. You can expand the sections on the left to see more recent apps or bookmarks, or the complete app list, but you're not looking at a windowing system here - all of this is just a webpage in a fullscreened browser.

To do anything more than look at this screen, you have to launch an App.

There's a browser (Firefox,) Skype, and a few gnu tools like galculator and freecell. Shockingly, there is no media player at all, and almost all the other "apps" here are just links to websites, except for - surprisingly - a complete office suite.

It has competent clones of Word, Excel and Powerpoint. This calls itself "Hyperspace Office", though it's just a rebranded version of a Korean Java suite that used to be called Thinkfree, made by Haansoft. It seems quite complete.

This is very limited. There is no desktop, no multitasking, you can't even drag windows around. Everything launches in fullscreen. You certainly can't install your own apps.

This is still actually not as bad as the alternatives on the market. The only actual desktop apps in Splashtop or Latitude-ON were a browser, email client, and Pidgin. Splashtop has a media player, but it's a horribly slow and limited Adobe Flash app. It also has a photo album, but you can't do anything with it other than look at pictures; useless. And while Splashtop seems to have a ton of other apps, they're all just shortcuts to websites. It certainly has nothing like an office suite.

Really though, even Hyperspace just expects you to spend your time in a browser. And hey, at least this version of Firefox has tabs enabled, unlike the others. Just like the others, though, it has a uselessly outdated version of TLS that can't be updated, so it will never be possible to go to 95% of modern websites.

So far, nothing is all that astonishing. This is an incredibly barebones Linux running a very small collection of permanently-out-of-date apps. Well, probably, anyway. I'm actually pretty convinced that Hyperspace was fully capable of updating itself, had Phoenix continued providing it - we'll address that more later.

To see what makes Hyperspace special, you'll want to watch this video clip.

Link if the above video doesn't work.

Clicking an icon in either Windows or Hyperspace swaps between them. I clocked it 6 seconds going one way, 12 seconds to go back.

So yeah, there's the virtualization trick. Except... not. I checked thoroughly: No hypervisor is in use. Windows would have to have generic drivers installed for all hardware, because device passthrough wasn't a thing on this platform or era, and it doesn't. But besides that, I simply know now that this is not the same product I had read about.

Endless Enigmas

I had not realized, when I started my investigation, that there were several SKUs of Hyperspace:

When I realized this machine didn't have Hybrid, I was initially disappointed, but I soon realized that I had actually gotten the far, far, far weirder outcome.

Dual and Hybrid are both straightforward ideas. A hypervisor is clumsy and overkill, but not really that strange a solution. The other one is just a normal Linux with a shortcut in Windows that reboots and selects the other partition, like we had in 1997. Yawn.

It didn't take me long, however, to figure out that Dual Resume was clearly up to some wretched tricks, and as I looked into it it just got stranger and stranger.

First off, I had a very hard time actually figuring out where the files were. Hyperspace isn't burned into the firmware or anything, it's just a program you install. In theory, you could buy it retail and use it on any "supported" (?) machine. So the actual guts of the OS have to be somewhere, but I looked and looked and couldn't find them.

I dug through the install folder and found nothing more than a few "psa" files. They opened in 7zip, but there was very little inside of them; scraps of Linux detritus, but no root FS, and nothing of substance. Curiously, they were VERY large compared to their contents. Hmm.

And despite spending ten minutes installing the OS, I couldn't figure out where all that was going. Both Windows and Linux agreed that there were some odd things going on. fdisk showed overlapping partitions, and Windows showed a 4.36GB partition with no recognizable FS.

Obviously the first guess is that the 4GB partition is simply linux ext3/4. And that's true - but it contains nothing more than a copy of grub and a few drivers. There's no root FS to be found, even when I mount the partition under a full-fat Linux distro. Digging through the grub files revealed nothing salient, just a mysterious call to "psaloader" with no info about exactly what that was loading or how.

Next question: If it's not virtualizing, how is it switching OSes this fast?

What if it's not virtualizing the host OS, only the guest? Well, no point in that - if Windows had to keep running, you wouldn't save any battery power. So that's out, and... what other explanation is there? I couldn't think of anything.

And then, there's this:

That's Hyperspace Write saving a file... into my Windows partition.

While Hyperspace is fairly fixed-function, it's surprisingly not a read-only environment. The other fast-start Linuces were: Splashtop and MontaVista didn't let you save anything at all other than your mail account info. Hyperspace, however, seems to have a complete filesystem. But it's... weird.

The "My Documents" folder you see there is misleading; it's not my Windows documents folder, it's a data partition inside the Hyperspace environment. If I save stuff here, it'll still be there after a reboot, but I can't see it from Windows.

"My Documents" is also the "root" of the filesystem, as far as Hyperspace is willing to admit: All the included apps refuse to go any higher than this, as if it was /.

Furthermore, if I select the C or D drive entries, that DOES open my normal Windows drives. The other folders - Documents, Pictures, etc. - really are in my Windows user folder as well. So I can retrieve files I saved while running Windows... or save stuff straight into My Documents in Windows itself.

Fucking... Qué?

Anyone who Computers Pretty Good can tell you that there is no holy way to do this. No priest would bless whatever is going on here. This is bad and wrong, and someone should have stilled the sinful hands of Phoenix's devs.

So I knew, at this point, that Phoenix had invented multiple novel technologies in pursuit of an incredibly stupid product that nobody wanted, but I was not yet quite aware of how bad it was going to get.

Descent

I had three burning questions and not the slightest answer to any of them:

All of this was baffling, but the OS swap took top billing. I could not think of any possible way to do that without virtualization. If it was just launching Hyperspace inside a fullscreen VM, that would be... stupid and pointless, but explainable within my worldview.

I began by testing whether Windows was, in fact, being shut down. Answer: yes, it is. I wrote a simple loop in Powershell that records the time, then went to Hyperspace for 20 minutes, and came back to find a 20 minute gap. This ruled out the only explanation I could think of: Windows was clearly suspended when it wasn't onscreen.

I noticed, however, that when I selected the Switch To Hyperspace icon, the screen faded out. I remembered that in Win 7, that's a default behavior when you shut down. So I checked the event log.

The system is entering sleep.

Oh no.

I had been sort of expecting this outcome. If virtualization was off the table, then the only other thing I could see was "some kind of ACPI horseshit." It's the only part of the PC architecture left, really, that is capable of what one might call "nonlinear behavior."

But still... what the fuck? What do I do with this data? "When I switch OSes, Windows thinks it's going into standby." What... what? What?? Where the fuck do I go from there?

I spent several hours beating my head against this problem. I had taken several different runs at slurping stuff out of the install files and had come up with a few more bits and pieces, including a folder full of various Mystery EXEs called shit like "fwimport" and "ptbackup," and DLLs that mostly looked like they were drivers for the Hybrid variant.

Ultimately, I had to give up on the idea for a bit. I moved on to question #2: Where the hell are the files?

I know a few different ways to "hide" disk partitions, but this was Flummoxing me. The entire disk was allocated, so I couldn't figure out where another partition might be hiding. I figured the 4GB partition had to be the secret, but there wasn't anything in it.

Besides that, I attached some instruments to the installer and run it again to find out what files it was touching, and to my surprise, I discovered a sequence in which it runs "fwmount.exe", then suddenly starts accessing files on the F: drive.

I had no F: drive. And it was busy doing this for quite some time, so I had the opportunity to go to Explorer; no F! I opened Disk Management; no F! What!

A New Day Dawns Over The Valley Of Madness

I began to poke holes in the mystery when I decided to follow up on a lead that had been bugging me for some time.

All over the place, I was seeing references to "PSA." Hyperspace seemed to install from ".psa" files, and there were references to PSAs buried all over batch files and inside the string tables of the various EXEs. These seemed very important, so I started trying to figure out what they were.

At the same time, a friend I was discussing this with proposed that the inability to find Hyperspace's files could be an HPA situation. HPA is a kind of very-hidden partition, most well known for use on older Thinkpads, and it is in fact derived from... a Phoenix technology.

In fairly short order, we had determined that there was a connection. The "fw*" tools included with Hyperspace had the same names as tools that came on the Thinkpad HPA partitions. It turns out that "FW" is short for Phoenix FirstWare, a product they'd been selling since at least 2005, specifically for creating pre-boot recovery environments that can't be hurt by viruses, because they use partitions that the OS can't discover.

They accomplish this with beer.

More accurately, BEER and PARTIES, asinine backronyms for a technique that (inferring heavily) amounts to writing a secret, funnier MBR to the end of the disk. (That's probably why there's a mysterious 25MB gap at the end of the partition table per Windows, to leave room.)

I believe what makes BEER partitions valuable is that, while they are technically just "bytes on the disk," good luck getting anything short of a kernel driver (or root access under *nix) to read, let alone write to areas of a disk that aren't mentioned in the MBR. Under Windows, it's probably nearly impossible.

But once I understood what all this was doing, I was able to make some progress. Sure enough, "fwdir" revealed a bunch of secret partitions:

Five extra partitions that don't show up in any OS. I was also able to use "fwmount id=1", which said it mounted something to F: - but nothing would read it.

Like before, Windows would not acknowledge that anything was mounted at F:. It even let me remount a normal partition there, with no complaints. If I tried to change to F: from the command prompt, it said no such drive existed. But if I tried to "DIR F:", it said the filesystem wasn't readable. Aha!

Apparently it mounted the disk into some kind of langolier purgatory that only existed inside this cmd session. So, on a whim, I grabbed a copy of dd for win32 and did "dd if=\\.\F: of=e:\dump.img", and it got something!

Specifically: the same fucking partition with nothing in it other than grub.

That's exactly what I got when I tried dding partition 6 normally from linux, so this whole exercise was pointless. Thus, I got a bigger gun: I went back, did a normal dd of the whole partition, then shoved it into UFS Explorer and told it to carve out any lost filesystems.

Would you look at that! Filesystems!

It turns out that FirstWare / HPA doesn't really obscure the contents of the partitions in any way. All it is is a weird MBR. If you can suss out the partition boundaries from the raw bytes on the disk, they still have normal file tables and can be treated as regular drives.

So at this point, convinced that there was nothing unusual about the data format, I put my nose to the grindstone and taught myself how to find an ext3/ext4 signature in a hex editor.* I found a whole bunch of Magic Numbers, then loaded up a linux livedisk and tried mounting a loopback device at each offset, until I hit paydirt.

* The trick is to look for "53 EF 01". Technically the magic number is just 53 EF but that's not unique enough, you'll find it everywhere. I found that 53 EF 01 was very likely to be an actual partition header, as long as I didn't see a bunch of junk around it; the beginning of a partition will be fairly sparse and un-noisy, so if you see solid gibberish, you're probably inside a file.

Once you think you have the right location, get the offset of the "53" byte in decimal; subtract 1080 from it; then do mount -o offset=4344684544 /dev/source_drive /mnt/mount_point. if it works, you're done.

I now had read-write access to the filesystem, which I proceeded to jailbreak. But we'll come back to that.

Wrongs Darker Than Death Or Night

First, what was the answer to the OS swap question? Well, I had my suspicions as soon as I saw the event log entry, but in the process of running down the hidden partition, I started searching for "phoenix PSA", and came up with... the patent for Hyperspace! It's terrifying.

In a nutshell - and assuming I have interpreted it correctly - Hyperspace works by performing a fucking David Copperfield Statue of Liberty trick on the ACPI subsystem.

Normally, putting a computer to sleep is sorta like this:

When you return from sleep, it's basically the reverse:

None of this has anything to do with virtualization or changing OSes. The entire point is that the process is supposed to be nearly transparent, as if it never happened. The OS is allowed to cooperate, to make it go smoothly, but this is only supposed to help you decrease power utilization gracefully.

Hyperspace perverts this mechanism.

When you install it, your bootloader is replaced with a copy of grub, the common FOSS bootloader. As far as I can tell, this serves two purposes:

  1. It lets you choose either Hyperspace or Windows on startup, or it reads a flag, set from within either OS, that tells it to choose one of those without asking

  2. It injects some stuff into the BIOS memory area, then continues booting normally.

That Stuff is called the OSM, or "OS Steering Module." It's... actually also grub, apparently, just heavily modified.

It doesn't do much while the machine is running. It's just sitting in memory, inert. But when you try to go to sleep... shit gets interesting.

The patent puts the correct amount of mustard on this:

At this point the behavior of OSM diverges greatly from the similar (up to now) action involved in entering ACPI State S3.

For, having taken many of the same actions that precede entering ACPI State S3, the OSM proceeds instead to create and save a further restorable hardware context and set of wake vectors and in preparation for loading a second OS as described below.

Enormous props to the patent author for using those intensely load-bearing phrases to describe this whole rugpull of a process. The bold text is important because this thing is not doing what it is supposed to be doing. Oh no. Not by a long shot.

It's like this: Windows gets itself all ready for sleep, tucks itself in, then tells the BIOS "too eepy! time for bed, S3~" but instead of the S3 reaching the BIOS, OSM intercepts that call. The OS has halted itself, and is waiting to be told that it's time to wake up, but OSM instead proceeds to alter the universe around it.

See, the other disgusting thing that OSM did when the machine was first booted was to go into the e820 table, where the BIOS defines what memory is available to the system, and declare ~512MB of it as nonexistent (or "Address Range Reserved.") That means that when Windows begins booting, if the machine has 2GB of memory, it only sees 1.5GB, as if the other 512 wasn't even installed.

When OSM does its swaparoo, it alters that table. It unmarks the 512MB area as reserved, then reserves all the other memory that Windows was using, before adjusting the "wake vectors." Those are saved values that tell the CPU where to pick up when it comes back from sleep, and normally they'd point to a spot inside the Windows kernel - within it's 1.5GB of RAM. Instead, they now point to... the mysterious 512MB area.

If Hyperspace had been launched at boot, and then the user switched to Windows, then the 512MB area will contain a copy of it. Because when OSM initially booted Hyperspace, it prepped the e820 tables the other way around, so Hyperspace could see that 512MB area and nothing else. It therefore booted into just that section of memory, unaware of the 1.5GB that Windows had, and when the user switched to Windows, it did this exact same process in reverse.

(If Hyperspace isn't booted, of course, then OSM instead loads it from the hidden partition, hence the psaloader command, a custom grub driver that understands how to read BEER disks.)

Once both OSes have been booted, then Hyperspace will be in its 512MB closet, and Windows will be running in the remaining 1.5GB, and any time that either OS tries to go to S3 state, the OSM will move all the memory pointers around, then reverse the sleep process, causing the machine to instantly "wake" - from a sleep it never actually entered. Windows thinks it's taking a nap, but it wakes up as someone else, leaves the house, and commits murders it has no memory of.

In other words: Phoenix figured out how to turn the BIOS into a hypervisor. It's almost, almost beautiful, if it wasn't both incredibly fragile and completely uncalled-for. But christ. Christ almighty, what a hack. One could come to tears over it.

Wait Wait, It Gets Worse

So then, what about that last bit: moving files between the OSes.

Okay, in a laboratory setting, this isn't that cursed an idea. If two processes modify the same file, for instance, it can work fine as long as they don't

A) Write to the same section of the file at the same time. If one writes, and then the other does, that can be okay, but if both are doing it, then data from both processes will get interleaved, so neither process gets a clean write.

B) Assume they know what is in the file without checking. If a process writes to a file, then does so again while assuming it has not been modified by anything else, then it could end up corrupting the file by not being aware of new data boundaries.

It is possible for software that is old, simplistic, or designed carefully to get away with this. You absolutely cannot do it with random software from many developers, running under different OSes.

Now, hang on - this almost seems moot. Windows thinks it's asleep, so all file writes are suspended, right? Indeed, per another patent regarding Hyperspace, it adds a driver to Windows that is used to force all cached writes to be committed to disk when an OS swap is initiated. So is there really a problem?

Well, you can read the patent to get the whole story - I frankly don't fully understand it, but I think the problem is that, while disk caches have been flushed, and Windows isn't actively writing anything, there can still be apps that are in the middle of modifying a file. Consider a database editor: you put Windows to sleep, then wake it up. The editor still has the file open, and probably has a copy of it in memory that it assumes matches what's on disk. If that file was modified while the system was asleep... oof.

I don't quite get how that's solved by this, though. Initially I had assumed that this was high-power tomfoolery, something like... "Hyperspace's storage is inside an image file, and when you switch to Windows the helper app reads that file and copies any new contents into your real filesystem." And the reality is... sort of like that... but much... worse...

When you launch Hyperspace, it just straight up mounts all your disks with NTFS-3G. They're not read-only, but they also... aren't read-write. As I understand it, any writes to your Windows disks are virtual. The app thinks the writes happened, the files look updated, but the bytes have not been changed on the disk.

Instead, your changes are written to a journal file. When you return to Windows, that journal file is read via a UnionFS filter which merges one of the hidden PSA partitions into the Hyperspace utilities folder, or some shit like that. As Windows is coming out of standby, a driver installed by Phoenix reads that journal file, then replays all the changes made while you were in Hyperspace.

I don't get this. It doesn't solve the fundamental problem of two programs accessing the same file. Just because you delayed the write until the main OS was running again doesn't seem to solve anything.

It's also really dangerous! Like I said, there is NO WAY to do this "correctly." This is a monstrous act. In fact, in 2009, accessing NTFS read-write from within Linux was often just straight-up Considered Harmful. It took a long time for the NTFS modules to mature, and I think at the time if you'd told anyone you were doing this they might have slugged you.

The whole thing is just a house of cards. It's all so fragile. I'm pretty certain that Hyperspace refuses to install on anything other than specific builds of XP, Vista and 7, and I wouldn't blame it. Changes to NTFS, or the specific order of operations in the standby/resume process, or any of a thousand other things could result in massive data loss. All for very little - was this truly necessary? I mean, yes, I'm all for moonshots in service of usability, but... could this really have been worth the risk, versus just offering the user a small shared partition for data exchange, or telling them to just use the SD card reader?

Jailbreak by Thin Lizzy

A lot of effort was put into Hyperspace - just like Splashtop and Latitude-ON - to jail the user. You can only launch the bundled apps. They cannot access the direct Linux filesystem. Hyperspace is more willing to let you access the quiesced Windows filesystem than its own, and that makes sense. While it is, after all, just an ordinary partition, the whole goddamn thing looks very fragile. I imagine it would be trivial for a user to break Hyperspace, very likely causing massive data loss when something scribbles onto the mounted NTFS disks.

I put in several hours of effort to find an exploit with no success. There is no terminal emulator, of course, the VTY keyboard shortcut is disabled, and they've locked down all the GUI apps. The file pickers will not show you anything outside of the My Documents funhouse. You can open Nautilus, but any attempt to execute a binary from a mounted USB drive results in nothing happening. You can reach Open With, but attempts to refer to /usr/bin/whatever also result in nothing happening. I imagine they modified the apps and added a bunch of "return false" crowbars to achieve this.

I did try looking for open telnet/ssh ports, and in fact, nmap turned up something fascinating: an open port 6000. Seriously, it doesn't just look that way, it's genuinely wide open - I launched an X client on another machine and watched the server on the netbook actually respond to it. xhost didn't allow it in, but the fact that the socket is actually open means you absolutely could have found network level exploits to pwn these things back in 2009, no question.

I am not "a hacker", however, so I didn't run that down. Instead, I just used my newfound ability to mount the partitions - read-write with no harm, as it turned out - to modify the startup scripts.

I don't think this system actually has the vty mechanism installed, so trying to prevent X from starting would probably just brick it, and normally as soon as X starts, it launches the launcher which takes over the whole desktop. Interrupting that leaves you at a blank, useless wallpaper. But there did appear to be a lot of normal utilities on here, just unreachable due to the limited UI.

One that was missing, comically, was xterm. I'm guessing that was specifically intended to mitigate local exploits, because I did find a commented out reference to it in the X startup script.

I poked around in the filesystem for a bit until I found a yum repo, then dug into that, trying to ID the distro they'd forked to produce Hyperspace. This wasn't that easy because, as it turns out, Phoenix actually ran their own repos! They're long gone now, but they actually had an infrastructure set up for updates, meaning that if they'd stuck around, this system wouldn't necessarily have experienced the fate of so many "appliance" Linuces: leashed to a kernel that was old when it was released, full of unpatched exploits, and becoming useless as its SSL root certs expire.

I did find a couple references to Fedora in there however, so I looked up the version of Firefox included - 3.5.1 - and matched that up with Fedora 11. I downloaded an rpm of xterm, copied it over to /usr/bin/, adjusted the Xinitrc script, and:

Obviously, it is no great feat to get a shell on a linux that you have raw disk access to. But, for me, this was quite an accomplishment, given that I had to learn

...in maybe six hours. It's New To Me.

Having gotten this far, of course, the OS rapidly devolved into Linux and became uninteresting, except insofar as I learned some things about exactly how they'd jailed it.

It turns out that the UI isn't as custom as you might think. The UI is largely GNOME based, and the machine has a nearly complete complement of GNOME desktop components, although I couldn't get gnome-session to put anything on the screen. It actually has a normal window manager, which I had noticed early on in the process, both because I found "compiz" in the strings in one of the files, and also because when you open programs, they tend to "fold" into view.

That's right: Hyperspace put a compositing window manager with effects enabled on an Atom netbook that probably got a Windows Experience Score of 1.2. I can't swear it won't run Aero, but they didn't even try, and that was probably the right call.

The only reason that the UI doesn't seem to have normal windows is because they used a FOSS tool called devilspie (??) which seems to be made for exactly this kind of purpose. You write scripts that tell it how to treat each window on the screen, based on its name (with partial matching!) or class, and you can have it force a window to be fullscreen, to be undraggable, to always be a fixed size, to always be centered on the screen, etc.

The calculator app, for instance, is a normal galculator, but forced to be centered and undraggable. It's... unsettling. You can tell that the window border is normal chrome, you just can't interact with it.

Once you interrupt the launch process though, such as by starting xterm instead, all that goes away.

Compiz actually runs quite smoothly! The screen res is only 1024x600, so that's not all that much of an achievement, and it's certainly very cramped, but I'm fairly impressed tbh.

At this point there's kinda nothing else to tell. It's Linux, just like every other Linux. The only other thing you might find interesting is the mount table, because... holy shit, what a mess. Phew.

In Conclusion, What The Hell

The most astonishing thing about Phoenix Hyperspace is... that they made it. That they bothered. I hate to be a debbie downer, but what were they thinking?

Even in 2009, it had to have been obvious that computers were about to overcome their speed issues. For christ's sake, Android was around, and it sucked but it demonstrated that you could suck much better performance out of a truly miserable piece of hardware. Windows just needed to get more efficient, and RAM prices needed to come down a bit, and SSDs needed to be come more affordable - and all those things happened! really fast!

It's not surprising that nobody cared about any of these technologies by the beginning of 2010. What's surprising is that at least five companies all talked themselves into believing that this was going to put them on top, somehow.

The wildest thing about all of this is that it doesn't work. Hyperspace isn't "instant-on," it's maybe a few seconds faster than Windows 7 to boot, and almost certainly the cost of developing and adding it far outweighs the 1GB of additional RAM, which probably would have made Windows much spritelier even on these pallid Atoms.

Sure, the instant-on Linuxes booted faster... usually. Actually, out of all the machines I have with variations on this theme, one of them takes about 25 seconds to boot Splashtop, no better than Windows, and the other takes over a minute. That's the Dell Z600 however, which is another story entirely.

An incredible amount of R&D and ingenuity was thrown at this problem, to no real benefit. I am absolutely dumbfounded as to how any of this happened. It defies logic. I can't comprehend who thought any of this was a good idea. It's amazing that it happened, incredible to look at, but ultimately useless, and I wish the people who worked on this had been allowed to do something more productive and fulfilling.

Other articles in this series:

List of Articles