A fist full of VMware analytics

So the next goal for the product is VMworld Europe. I have been busy coding away to extend the analytics to cover file-system scanning to determine duplicate blocks. Its proving a bunch of interesting challenges, how to scale, how to determine duplicates across file-system boundaries and all sorts of other issues.

Final testing in progress, and those odd problems about regression bugs... but here's a first screenshot!

Oracle runs on an Intel Mac ... really!

So darn Oracle, when the hell will there be a native port to Intel Mac? I don't know, they all must be slacking around doing cool Linux and Xen stuff instead.

So I had 10gR1 running on my PowerBook G4 and PowerMac G5 and all has worked very well for some time. Somebody at some point in time thought it would be cool to support Mac and PowerPC (PPC). I get my first Intel Mac and there is no port to Intel or even any sign of it... what should I do?

Then I remembered Apple had Rosetta to emulate all those PowerPC instructions for non "universal" applications. So I first tried to install the Oracle software from scratch. No dice. Despite being Java it puked at a JNI dynamic library. Next I simply copied the existing install from my PowerBook G4 and after hacking my init.ora around (I had lost the original) and re-creating the orapw file... I have it running!

VMware Fusion fun... not!

So thank god to Wikipedia and Google. I had been trying to run Oracle under a Linux O/S under a Virtualized Server (yes, I'm that trendy). But could not get Oracle installed. Each time it tried to create a file or directory I got

$ cd /mnt/hgfs/oel4u5ora11g

$ touch 1.1

touch: cannot touch '1.1' : Permission denied

So I had set up a Share from the VMware preferences so that the guest O/S could not a directory on my local disk (through the Preferences). I had then gone through the directory and performed a

$ chmod -R a+rwx *

But was still getting the permission denied problem. In a simple solution on Wikipedia. Simply turn the Network from NAT to Bridged... and now I have Oracle all installed...

Blob, Binary and other Java hardships

So, what's the difference between

byte[] fred = {-122, -97, -120, 111};

System.out.println(fred.toString());
System.out.println(new String(fred));

Well, you would same they should print the same right. Sadly wrong. So this took me a day of debugging to find out why my Fact table was producing the wrong results from the Data Warehouse queries. So I tried

• Stuffing the data into a BLOB, well its binary after all. Broke all the joins over this column
• Converting to Hex but got different results for the same byte[] value
• Converting to a Binary String (i.e. 1's and 0's), but seemed horribly inefficient
• Converting to a String, like Hex got different results for the same byte[] value and was unprintable through the mysql prompt

So I ended up keeping the value as a byte[] in the Java code and then stuffing the value in a Binary column... easy when you know how. So I just need to reconcile the values I am now getting.

What's up?

So what's happened since the launch? Well all the normal things, keeping the site running, dealing with bugs and feedback and thinking about something new to add...

I think it has struck me and just about everybody else on the planet that Server Virtualization is going to be a real big thing in 2008. I was sitting on a cross-country flight thinking (in coach class again) and I was struck how the algorithms that I have used and applied to Databases are just as applicable to Storage and File-Systems especially if you can determine the cost savings of deploying compression and de-duplication on the storage. This is a very obvious thing to do in Server Consolidation, just how many copies of Win2k3 or XP do you need to store when you are simply changing the IP and hostname for each copy... so it got me thinking and the complier out again.

Launch Day

Launch at Oracle Open World

So a 24 hour period to be forgotten. I had this troubling bug, that I could not reproduce. Sometimes it would come up and sometimes it would not. It was Saturday night and I knew that I would be traveling all day Sunday. So I staged what I had and all I needed to do was to trip the DNS switch and we would be live.

I started the journey relaxing, reading this months Wired magazine. Then it came to me a possible cause of events that could reproduce the bug. I fired up my laptop and low and behold I could reproduce it. I knew that I had to be super careful, you know a last minute change. I spent the rest of the flight looking as ways of fixing it without actually changing any code.

I got to San Francisco with a plan but I had many, many things to do that night. Roughly they were

•Interview the street marketing team
•Pick up all the stuff (T-Shirts, flyers, business cards) for the give aways
•Fix the bug

I’ll talk about the marketing events in the next blog entry. So, I fixed the bug easily but then I was faced with a dilemma. So I stage it from my Laptop or do I push from the build server? Neither seem any easy choice. Was my laptop 100% setup with all the config information to push the live image, could I make the changes remotely and push on the build server?

After a couple of attempts (aka failures) I pushed the image from my laptop. It took 15 minutes or so and at last I thought I could take it easy. But I was wrong...

So what was the issue? By 1am (which by body was really saying 4am) I could not find a good reason why the code was failing to run when I ran it from my laptop on the hosted site at Amazon. I figured sleep would help.

4am I wake up, can’t sleep any longer. I try running it from a remote machine and it appears to work. Ok, then its an issue with my laptop. I add a few extra debug statement and finally narrow it down to an extra Firefox plug-in on my Laptop. It was failing when I upload a new archive. The archive got there Ok and was processed Ok, but an error was returned to the web client. Why was this different? Well is the only form in the Application that was doing an HTTP PUT, since the rest of the Application is Ajax then there were no other PUTs and GETs. The culprit? The Snap shots plug-in was mucking with the return of the HTTP GET.... grrrrrrrr.

So I thought that was the end. Then came my nightmare from DNS street. I switch the DNS entries as I had practiced. Not the best way to do this, but the easiest way... so I thought. As you know if takes time for DNS to propagate, but its pretty quick 5-10 minutes. So I switched at 8.45am. By 9am my browser on my Laptop was still pointing to the old static web page. By 9.15am it still was.... panic!

I tried my build server, and thank god it was pointing to the correct new hosted page. Figured it was a DNS propagation delay. I continue to check all day to find that my Laptop was still pointing at the old page. Holly shit I thought. Then I connected to a VPN end-point that configured DNS and vola! I could now see the hosted web page. The problem? Using the Hotel’s ISP, there was no auto-config of DNS so my laptop was using the cached entry pointing to the old IP. Once the laptop connected to a DNS server, then it got the new IP and resolved correctly.

I think I aged 10 years in those 24 hours...

Finishing Line?

Are we there yet?

So the finish line, or is it the start? I’m too tired to answer but happy to get to Oracle Open World and let Joe Public answer the question. So where’s the scores on the doors? Well the good news is this is what did get done! Some amazing work was done by Dave T to put this through its paces!

It feels impressive, but really does not fell like all the late nights, lattes and other nocturnal behavior. It came down to the following

•7116 lines of Java code the the UI
•9354 lines of Java code for the Server
•2382 lines of documentation



The UI look peachy, hear seen with the EC2 instances up and running... and I did get a lot done.



But alas, there is also the wall of shame, just those things that I just was never going to get to... I especially liked the “Zoro all todo’s” a callback to older coding days gone by!



Like Apollo 13, here’s my final, final check list to remember to do before launch on Monday morning... don’t forget, rocket goes bang etc.