Cross Domain data channel with HTML5 Canvas

Standard Ajax is restricted to single origin policy so JSONP is the de-facto way for exchanging data with cross-domain sources and it works pretty well. Alternative, though bit hacky way, is to use HTML5 Canvas as cross domain work-around using pseudo Images as “covert channel”.

Basic idea is simple, javascript in client requests image file from 3rd party site where server encodes a data to the Image, client can use cookies and url parameters to identify itself as desired. Then client renders image, and decodes the data from the image pixels.

Backend

Backend needs to be able to construct images with custom pixel level data, in this example we use Node.js and canvas module that is server side HTML5 Canvas implementation based on Cairo graphics library.

This function accepts any object and returns canvas object that contains the objects JSON presentation encoded in image pixels.

function encodeDataToImage( data ) {

	// Convert data to binary buffer while being utf-8
	var s = encodeURIComponent( JSON.stringify(data) );
	var buffer = new Buffer(s, 'utf8');
	var pixelc = (buffer.length / 3) + (buffer.length % 3 ? 1 : 0)

	// Encode data as PNG image
	var Canvas = require('canvas');
	var canvas = new Canvas(pixelc, 1)
	var ctx = canvas.getContext('2d');
	var imgdata = ctx.getImageData(0, 0, pixelc, 1);

	for (var i=0, k=0; i < pixelc * 4; i += 4 ) {
		imgdata.data[i + 3] = 0xFF; // set alpha to full opaque
		for (var j=0; j < 3 && k < buffer.length; k++, j++ ) {
			imgdata.data[i + j] = buffer[k];
		}
	}
	// set "image" data
	ctx.putImageData(imgdata, 0, 0);
	return canvas;
}

Define xd request handler that builds and sends the data coded image to the client. (Example in Express.js).

someapp.get('/xd', function(req, res ) {
    // do here something with query or cookies, like resolve uid and set
    // data.
    // Example data
    var data = { a: 1, en: 'owl', fi: 'pöllö', es: 'búho', uid: req.query.uid }

    var canvas = encodeDataToImage( data );
    var img = canvas.toBuffer();
    res.contentType('png');
    res.header('Content-Length', img.length);
    res.send( img );
});

Browser
At browser side load the image and decode it back to object

function queryXD( query, callback ) {

	var img = new Image();
	img.src = 'http://some.site.example.com/xd?' + query;
	img.addEventListener('load', function() {

		// Image loaded, create temporary canvas
		var canvas = document.createElement('canvas');
		var ctx = canvas.getContext('2d');

		// draw image on canvas
		canvas.width = img.width;
		canvas.height = img.height;
		ctx.drawImage( img, 0, 0 );

		// collect bytes from image pixels
		var bytes = [];
		var imgdata = ctx.getImageData(0, 0, img.width, img.height);
		for (var i=0; i < img.width * 4; i++ ) {
			if ( i && (i + 1) % 4 == 0) {
				i++;
			}
			var b = imgdata.data[i];
			if (!b) {
				break;
			}
			bytes.push( b );
		}

		// convert bytes to string and parse JSON
		var s = decodeURIComponent( String.fromCharCode.apply(null, bytes) );
		var data = JSON.parse(s);

		callback(false, data);
	}, false);

        // image failed to load
	img.addEventListener('error', function(err) {
		callback(err);
	}, false);
}

And now its simple to do cross domain data exchange like

queryXD('uid=2134', function(err, data) {
   alert(data.en + ' is ' + data.fi + ' in Finnish and ' + data.es + ' in Spanish');
});

Caveats
Proxies and browsers like to cache the images, so use every time unique dummy parameter to force fetch.

HTML5 Canvas Layout and Mobile Devices

Common problem with Canvas and mobile devices is how to get the canvas to fill the browser window properly. This can be tricky and require lots of tweaking and testing with different devices to get it exactly right.

Even if you get the size correctly defined, the rotation is another hurdle and the layout could break after orientation change or two.

I wrote here example of simple layout page, that should work both on desktops and mobile devices Android (>2.2) and iPhone/iPad. It should appear as following layout in all browsers shown here in the iPhone screenshots and not break on resize or orientation change.

The layout works also after rotation.

Page defines a canvas (green), that occupies most of the screen and under that a fixed height div (yellow) containing ‘Some Text Here’. On every resize the code draws black rectangle that is -10 pixels short from each canvas border and writes number of orientation changes and the resize events for debugging purposes. The document background is defined blue to reveal possible unwanted overflows.

How it works?

DOM/CSS

First the meta elements tell to mobile devices how to handle the page. No scaling and width is fixed to device width.

    <meta name = "viewport" content = "user-scalable=no,
          initial-scale=1.0, maximum-scale=1.0, width=device-width" />
    <meta name="apple-mobile-web-app-capable" content="yes" />

The document is wrapped in single div (“container”) that contains the canvas and the fixed height div.

      <body>
	<div id="container">
	  <canvas id="canvas">
		HTML5 Canvas not supported.
	  </canvas>
	  <div id="fix">Some Text Here</div>
	</div>
       ...

Container is forced to fill the browser window by CSS rule that defines overflow as auto and width/height 100%.

	  body,html
	  {
		  height: 100%;
		  margin: 0;
		  padding: 0;
		  color: black;
	  }
	  #container
	  {
		  width: 100%;
		  height: 100%;
		  overflow: auto;
	  }

Canvas element is inside the container and has no initial height and width. It is defined as display block in CSS to avoid unwanted padding or margins. Canvas default display is inline, that is something you almost never want.

	  #container canvas {
		  vertical-align: top;
		  display: block;
		  overflow: auto;
	  }

Finally div (“fix”) is defined with fixed height

	  #fix {
		  background: yellow;
		  height: 20px;
	  }

This is not enough though, and some JS handling is required for resize and the orientation change.

Javascript

The JS listens both timeout and orientation change events and installs a timeout function that gets cancelled if browser sends several events rapidly.

    var resizeTimeout;
    $(window).resize(function() {
       clearTimeout(resizeTimeout);
       resizeTimeout = setTimeout(resizeCanvas, 100);
    });

    var otimeout;
    window.onorientationchange = function() {
       clearTimeout(otimeout);
       otimeout = setTimeout(orientationChange, 50);
    }

Orientation change listener does nothing important, it just updates the counter for debugging purposes.

The resizeCanvas is more involved. When browser is iPhone it first increases the container height 60 pixels higher than the browser window height. This makes possible to scroll the window down and hide the iPhone Safari address bar.

    if (ios) {
      // increase height to get rid off ios address bar
      $("#container").height($(window).height() + 60)
    }
    ...
    setTimeout(function() { window.scrollTo(0, 1);  }, 100);

Then it gets the container width and height, that are height and width of the browser window.

    var width = $("#container").width();
    var height = $("#container").height();

And finally forces the canvas size and width to the required. The height is subtracted by 20 to leave room for the fixed height div.

    cheight = height - 20; // subtract the fix div height
    cwidth = width;

    // set canvas width and height
    $("#canvas").attr('width', cwidth);
    $("#canvas").attr('height', cheight);

There could be better way to do this, but at least this seems to be pretty robust and works in all major desktop and mobile browsers.