youbeQ - Moving from Google Earth to Cesium

This is a guest post by André Santos on his experience porting the awesome youbeQ app from the Google Earth Plugin to Cesium. - Patrick

youbeQ first appeared at the end of 2011, we wanted to build a 3D platform where people could explore the Earth, talk and share content along the way, and the Google Earth Plugin seemed the right fit for our needs. It provided the Earth full of 3D buildings and a good representation of the terrain, but it also had a lot of limitations and performance inconsistencies that gave me a lot of headaches, but I kept on working with it trying to build something awesome.

youbeQ kept on evolving and changing and is now a driving and flight simulator, where you can explore the Earth with different vehicles.

example — Express Plane taking off from the Madrid Airport

Why Change?

The year of 2013 came, bringing the hope of a bright new future with the showcase of the new Google Maps with its awesome 3D mode in WebGL, but it also marked the beginning of the end for the NPAPI in Google Chrome, which the Google Earth Plugin uses.

Since our users use mostly Google Chrome this presented a big big problem and we needed an alternative that could give the same or similar functionality as the GE Plugin. Unfortunately there was nothing available at the time.

From time to time we would go for a look around the web to see how the virtual globes were doing, and I checked Cesium a couple of times but it lacked the ability to add and manipulate 3D models, something crucial for youbeQ. But one time, I chanced upon the cool post about 3D models, and after losing any hope of the new 3D layer in Google Maps being available any time soon I decided (a month ago) it was time to give Cesium a try. This marked the beginning of the challenge of changing youbeQ from the Google Earth Plugin to Cesium.

Changing to Cesium

The first thing I had to get used to, was working with Cartesian values instead of degrees (latitude, longitude), thankfully there is always a fromDegrees function to help you. Also, manipulating matrices for setting the position, orientation and scale of the different elements was something that took awhile to get used to. Another small detail was the order of the [latitude, longitude] when passing it into functions in Cesium, here it is [longitude, latitude].

So in order to migrate youbeQ, I started by thinking of all the different parts I had and needed to find a way to implement in Cesium:

Load 3D models and manipulate them in real-time
Manipulate the Camera in real-time
Create GroundOverlays and manipulate them in real-time
Create Placemarks and manipulate them in real-time
ScreenOverlays

Thanks to Sandcastle and the huge help of different members from the Cesium team, I was able to find solutions to almost every part. Their team was really fast in answering questions on the forum, which allowed me to reach a working version in a couple of weeks. youbeQ is running stable using Cesium since the beginning of this month (October 2014). Finally youbeQ is plugin free, is faster and the overall user experience improved.

How I did it

So let's go over each of those parts, and hopefully these tips/solutions can help people coming from the GE world get a better feel of how to do it in Cesium.

1. Models

This proved to be a big challenge that I faced during this migration.

In GE, loading a 3D model is pretty straightforward and the same applies to Cesium. But if you have multiple 3D models, where some are positioned with another as a reference but with custom behaviors, for example the flaps of a plane, this requires some math to get things in the right place, which isn't as straightforward in either GE or Cesium.

So let's say you have the main body and other bodies attached to it, I was doing some fancy quaternion construction given the Euler angles of the main body, multiplying that with the quarternions for each axis of the components, finding the resulting Euler angles and setting their position and orientation. This worked pretty well in GE, but then trying to accomplish the same in Cesium proved to be difficult (you can check the forum post to see a lot of code snippets of the mess I was in).

But then Dan Bagnell came to the rescue, and pointed me to having my model as a hierarchy of meshes, which takes care of positioning the components correctly and you only have to worry about the orientation. He also showed me that you can access each node of the model by its name and change its matrix, this allows you to change the node's position, orientation and even its scale:

var node = model.getNode('wheel_front_right');

var scale = new Cesium.Cartesian3(1.0, 1.0, 1.0);

var rotateQuat = Cesium.Quaternion.fromAxisAngle(Cesium.Cartesian3.UNIT_Z, heading);
var turnQuat = Cesium.Quaternion.fromAxisAngle(Cesium.Cartesian3.UNIT_Y, tilt);
var rollQuat = Cesium.Quaternion.fromAxisAngle(Cesium.Cartesian3.UNIT_X, roll);
var nodeQuat = Cesium.Quaternion.multiply(rollQuat, rotateQuat, new Cesium.Quaternion());
Cesium.Quaternion.multiply(nodeQuat, turnQuat, nodeQuat);

var translationArray = model.gltf.nodes[node.id].translation;
var translation = new Cesium.Cartesian3(translationArray[0], translationArray[1], translationArray[2]);
node.matrix = Cesium.Matrix4.fromTranslationQuaternionRotationScale(translation, nodeQuat, scale);

UPDATE: Starting with Cesium 1.6, the above code can be simplified to:

var node = model.getNode('wheel_front_right');

var translationArray = model.gltf.nodes[node.id].translation;
var translation = new Cesium.Cartesian3(translationArray[0], translationArray[1], translationArray[2]);

node.matrix = Transforms.headingPitchRollToFixedFrame(translation, heading, tilt, roll);

2. Camera

This was a really important part of the youbeQ experience, and with which I spent many hours banging my head against the wall, but with the help of Dan Bagnell, I finally got it to work as I wanted. In GE we can control the camera by setting its position [latitude, longitude, altitude] and orientation [heading, tilt, roll]:

var camera = ge.getView().copyAsCamera(ge.ALTITUDE_RELATIVE_TO_GROUND);
camera.set(lat, lng, alt, ge.ALTITUDE_ABSOLUTE, heading, tilt, roll);

ge.getView().setAbstractView(camera);

In Cesium, it is easy to place it where you want, but it requires some tricks to get it to look at the place you want. First you have to get the Cartesian of your [longitude, latitude, altitude], find the transform for the camera given that position, find the direction we want the camera to look at, apply the heading and tilt in world coordinates than do the twist (roll), simple enough. Here is some code:

var position = Cesium.Cartesian3.fromDegrees(lng, lat, cameraAlt, ellipsoid);
var transform = Cesium.Transforms.eastNorthUpToFixedFrame(position);

camera.transform = transform;

var yDir = Cesium.Matrix4.multiplyByPointAsVector(bodyModel.modelMatrix, Cesium.Cartesian3.UNIT_X, new Cesium.Cartesian3());
Cesium.Matrix4.multiplyByPointAsVector(camera.inverseTransform, yDir, yDir);

Cesium.Cartesian3.negate(yDir, yDir);
Cesium.Cartesian3.normalize(yDir, yDir); // yDir's magnitude might not be exactly 1 after the rotation

camera.lookAt(
  yDir,
  Cesium.Cartesian3.ZERO,
  Cesium.Cartesian3.UNIT_Z
);

var transform = camera.transform;
camera.setTransform(Cesium.Matrix4.IDENTITY);
camera.heading = -cameraHeading;
//camera.tilt = (Math.PI/2)-cameraTilt; // this clamped the value [0, PI/2]

var angle = ((Math.PI/2)-cameraTilt) - camera.tilt;
camera.look(camera.right, angle); // this is a workaround for the camera.tilt = X;
camera.setTransform(transform);
camera.twistLeft(cameraRoll);

Once again I have to thank Dan Bagnell, without his help I would still be driving a vehicle either looking up at the stars or looking everywhere but the vehicle.

UPDATE: Starting with Cesium 1.6, setting the heading, pitch, and roll in the above code can be simplified to:

camera.setView({
    heading : cameraHeading,
    pitch : cameraTilt,
    roll : cameraRoll
});

3. GroundOverlays

This is something pretty simple to accomplish in GE, and allows me to place "shadows" beneath the vehicles in youbeQ. You can set the image of the overlay, its location and rotation through a LatLonBox:

var overlay = ge.createGroundOverlay("");
overlay.setIcon(ge.createIcon(""));
overlay.getIcon().setHref(url);

var latLngBox = ge.createLatLonBox("");
latLngBox.setBox(north, south, east, west, rotation);
overlay.setLatLonBox(latLngBox);

ge.getFeatures().appendChild(overlay);

This proved to be a difficult element to recreate, at the moment, in Cesium. Thanks to the help of Mark Erikson on the forum, I found out that I could use the SingleTileImageryProvider to load an image that drapes over the terrain, but you can't change its position after loading, which prevents me from using this solution for my use case, but if you don't need to change its position after loading it should give pretty much the same result as a GroundOverlay:

var tile = new Cesium.SingleTileImageryProvider({
  url: url,
  rectangle: Cesium.Rectangle.fromDegrees(west, south, east, north)
});

tileLayer = scene.imageryLayers.addImageryProvider(tile);

According to Patrick Cozzi, the RectanglePrimitive will support draping over terrain, but this a feature that is not yet being developed.

4. Placemarks

In GE a placemark may contain a name and an icon, you can change its style, and easily set its position on the globe by setting its geometry. To change its position you can just update its geometry coordinates:

var placemark = ge.createPlacemark("");
placemark.setName(name);

var icon = ge.createIcon('');
icon.setHref(url);

var style = ge.createStyle(''); // create a new style
style.getIconStyle().setIcon(icon); // apply the icon to the style
placemark.setStyleSelector(style); // apply the style to the placemark

// Set the placemark's location.
var point = ge.createPoint('');
point.setLatitude(lat);
point.setLongitude(lng);

placemark.setGeometry(point);

ge.getFeatures().appendChild(placemark);

To accomplish the same in Cesium, I had to use Labels and Billboards. The label allows us to write text on the globe while the billboard allows us to set images. You have to create a LabelCollection and/or a BillboardCollection and add it to the scene. Then you can add your label and/or billboard elements to the corresponding collection:

var billboards = scene.primitives.add(new Cesium.BillboardCollection());
billboards.add({
  image : url,
  position : new Cesium.Cartesian3(0.0, 0.0, 0.0),
  horizontalOrigin: Cesium.HorizontalOrigin.CENTER
});

var labels = scene.primitives.add(new Cesium.LabelCollection());
labels.add({
  text: 'example text',
  position: new Cesium.Cartesian3(0, 0, 5),
  horizontalOrigin: Cesium.HorizontalOrigin.CENTER,
  font : '18px sans-serif'
});

In order to update its position I change the position attribute of each element in the collection according to the [longitude, latitude, altitude].

for(var i = 0, n = billboards.length; i < n; i++) {
  var x = billboards.get(i);
  x.position = Cesium.Cartesian3.fromDegrees(lng, lat, alt);
}

for(i = 0, n = labels.length; i < n; i++) {
  var x = labels.get(i);
  x.position = Cesium.Cartesian3.fromDegrees(lng, lat, alt);
}

5. ScreenOverlays

Since we are dealing with an HTML canvas element we can overlay anything on top of it, something that was not possible with the GE Plugin (except for Mac OSX), so ScreenOverlays became HTML elements.

So that is it for the big parts required for the transition. Almost everything was possible to implement, and the remaining elements will be possible to replicate in the future, so I believe I made the right choice in switching to Cesium.

Conclusion

Cesium is a great platform, it provides a lot of elements that allow you to build any kind of application, it has better performance than GE, but there is a big element missing that people coming from the GE world love, the 3D buildings layer, which is something that affects greatly youbeQ.

There will eventually be some kind of 3D buildings layer in Cesium but until then we, in youbeQ, will probably have to add our own models in some key locations or else the world seems too deserted.

Apart from that, we are now focused on improving the experience and realism of the vehicles, we are also thinking of re-introducing a global chat, allowing for direct audio and video calls between travelers and also group calls. But we will see what makes more sense to introduce into youbeQ.

PS: This is something that I use a lot in GE: finding the altitude of the terrain at a given position. For anyone interested in knowing how to find that in Cesium:

var position = Cesium.Cartesian3.fromDegrees(longitude, latitude, 0, ellipsoid);
var altitude = scene.globe.getHeight(Cesium.Ellipsoid.WGS84.cartesianToCartographic(position));

Beware that altitude can be undefined.