<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
	>

<channel>
	<title>Brave New Method</title>
	<atom:link href="http://bravenewmethod.wordpress.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://bravenewmethod.wordpress.com</link>
	<description>Technology tryouts</description>
	<lastBuildDate>Fri, 17 Feb 2012 11:23:47 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
<cloud domain='bravenewmethod.wordpress.com' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
<image>
		<url>http://s2.wp.com/i/buttonw-com.png</url>
		<title>Brave New Method</title>
		<link>http://bravenewmethod.wordpress.com</link>
	</image>
	<atom:link rel="search" type="application/opensearchdescription+xml" href="http://bravenewmethod.wordpress.com/osd.xml" title="Brave New Method" />
	<atom:link rel='hub' href='http://bravenewmethod.wordpress.com/?pushpress=hub'/>
		<item>
		<title>HTML5 Canvas Layout and Mobile Devices</title>
		<link>http://bravenewmethod.wordpress.com/2011/08/28/html5-canvas-layout-and-mobile-devices/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/08/28/html5-canvas-layout-and-mobile-devices/#comments</comments>
		<pubDate>Sun, 28 Aug 2011 19:46:11 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[html5]]></category>
		<category><![CDATA[Javascript]]></category>
		<category><![CDATA[canvas]]></category>
		<category><![CDATA[mobile]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=150</guid>
		<description><![CDATA[Common problem with Canvas and mobile devices is how to get the canvas to fill the browser window properly. This can be tricky and require lots of tweaking and testing with different devices to get it exactly right. Even if you get the size correctly defined, the rotation is another hurdle and the layout could [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=150&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Common problem with Canvas and mobile devices is how to get the canvas to fill the browser window properly. This can be tricky and require lots of tweaking and testing with different devices to get it exactly right.</p>
<p>Even if you get the size correctly defined, the rotation is another hurdle and the layout could break after orientation change or two.</p>
<p>I wrote here<a href="http://gamesmademe.com/examples/layout.html"> example of simple layout page</a>, that should work both on desktops and mobile devices Android (&gt;2.2) and iPhone/iPad. It should appear as following layout in all browsers shown here in the iPhone screenshots and not break on resize or orientation change.</p>
<p><img class="alignnone" src="http://gamesmademe.com/examples/layout_p.png" alt="Potrait Layout" width="320" height="480" /></p>
<p>The layout works also after rotation.</p>
<p><img class="alignnone" src="http://gamesmademe.com/examples/layout_l.png" alt="Landscape Layout" width="480" height="320" /></p>
<p>Page defines a canvas (green), that occupies most of the screen and under that a fixed height div (yellow) containing &#8216;Some Text Here&#8217;. On every resize the code draws black rectangle that is -10 pixels short from each canvas border and writes number of orientation changes and the resize events for debugging purposes. The document background is defined blue to reveal possible unwanted overflows.</p>
<p>How it works?</p>
<p><strong>DOM/CSS</strong></p>
<p>First the meta elements tell to mobile devices how to handle the page. No scaling and width is fixed to device width.</p>
<pre>    &lt;meta name = "viewport" content = "user-scalable=no,
          initial-scale=1.0, maximum-scale=1.0, width=device-width" /&gt;
    &lt;meta name="apple-mobile-web-app-capable" content="yes" /&gt;</pre>
<p>The document is wrapped in single div (&#8220;container&#8221;) that contains the canvas and the fixed height div.</p>
<pre>      &lt;body&gt;
	&lt;div id="container"&gt;
	  &lt;canvas id="canvas"&gt;
		HTML5 Canvas not supported.
	  &lt;/canvas&gt;
	  &lt;div id="fix"&gt;Some Text Here&lt;/div&gt;
	&lt;/div&gt;
       ...</pre>
<p>Container is forced to fill the browser window by CSS rule that defines overflow as auto and width/height 100%.</p>
<pre>	  body,html
	  {
		  height: 100%;
		  margin: 0;
		  padding: 0;
		  color: black;
	  }
	  #container
	  {
		  width: 100%;
		  height: 100%;
		  overflow: auto;
	  }</pre>
<p>Canvas element is inside the container and has no initial height and width. It is defined as display block in CSS to avoid unwanted padding or margins. Canvas default display is inline, that is something you almost never want.</p>
<pre>	  #container canvas {
		  vertical-align: top;
		  display: block;
		  overflow: auto;
	  }</pre>
<p>Finally div (&#8220;fix&#8221;) is defined with fixed height</p>
<pre>	  #fix {
		  background: yellow;
		  height: 20px;
	  }</pre>
<p>This is not enough though, and some JS handling is required for resize and the orientation change.</p>
<p><strong>Javascript</strong></p>
<p>The JS listens both timeout and orientation change events and installs a timeout function that gets cancelled if browser sends several events rapidly.</p>
<pre>    var resizeTimeout;
    $(window).resize(function() {
       clearTimeout(resizeTimeout);
       resizeTimeout = setTimeout(resizeCanvas, 100);
    });

    var otimeout;
    window.onorientationchange = function() {
       clearTimeout(otimeout);
       otimeout = setTimeout(orientationChange, 50);
    }</pre>
<p>Orientation change listener does nothing important, it just updates the counter for debugging purposes.</p>
<p>The resizeCanvas is more involved. When browser is iPhone it first increases the container height 60 pixels higher than the browser window height. This makes possible to scroll the window down and hide the iPhone Safari address bar.</p>
<pre>    if (ios) {
      // increase height to get rid off ios address bar
      $("#container").height($(window).height() + 60)
    }
    ...
    setTimeout(function() { window.scrollTo(0, 1);  }, 100);</pre>
<p>Then it gets the container width and height, that are height and width of the browser window.</p>
<pre>    var width = $("#container").width();
    var height = $("#container").height();</pre>
<p>And finally forces the canvas size and width to the required. The height is subtracted by 20 to leave room for the fixed height div.</p>
<pre>    cheight = height - 20; // subtract the fix div height
    cwidth = width;

    // set canvas width and height
    $("#canvas").attr('width', cwidth);
    $("#canvas").attr('height', cheight);</pre>
<p>There could be better way to do this, but at least this seems to be pretty robust and works in all major desktop and mobile browsers.</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/150/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/150/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/150/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=150&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/08/28/html5-canvas-layout-and-mobile-devices/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>

		<media:content url="http://gamesmademe.com/examples/layout_p.png" medium="image">
			<media:title type="html">Potrait Layout</media:title>
		</media:content>

		<media:content url="http://gamesmademe.com/examples/layout_l.png" medium="image">
			<media:title type="html">Landscape Layout</media:title>
		</media:content>
	</item>
		<item>
		<title>Keeping CouchDB design docs up to date with Node.js</title>
		<link>http://bravenewmethod.wordpress.com/2011/06/03/keeping-couchdb-design-docs-up-to-date-with-node-js/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/06/03/keeping-couchdb-design-docs-up-to-date-with-node-js/#comments</comments>
		<pubDate>Fri, 03 Jun 2011 15:10:27 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Javascript]]></category>
		<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[couchdb]]></category>
		<category><![CDATA[nodejs]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=141</guid>
		<description><![CDATA[CouchDB views are defined typically as Javascript snippets and are part of special documents called design documents. I noticed that keeping these design documents up to date during development is pretty cumbersome and error prone. So I devised simply way to keep them updated using Node.js and Cradle couchdb driver. Idea is to define the [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=141&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>CouchDB views are defined typically as Javascript snippets and are part of special documents called design documents. I noticed that keeping these design documents up to date during development is pretty cumbersome and error prone. So I devised simply way to keep them updated using <a title="Node.js" href="http://nodejs.org/">Node.js</a> and <a title="Cradle" href="https://github.com/cloudhead/cradle">Cradle</a> couchdb driver.</p>
<p>Idea is to define the views in as variables in runnable js script and run that with Node each time it&#8217;s changed.</p>
<p>Here is the code. Copy it to e.g. cdb-views.js.</p>
<pre>var cradle = require('cradle');

cradle.setup({ host: 'localhost',
               port: 5984,
               options: { cache:true, raw: false }});

var cclient = new (cradle.Connection)

function _createdb(dbname) {
    var db = cclient.database(dbname);
    db.exists(function(err, exists) {
        if (!exists) {
            db.create()
        }
    });
    return db;
}
var DB_SOMETHING = _createdb('somedb')

function cradle_error(err, res) {
    if (err) console.log(err)
}

function update_views(db, docpath, code) {
    function save_doc() {
        db.save(docpath, code, cradle_error);
        return true;
    }
    // compare function definitions in document and in code
    function compare_def(docdef, codedef) {
        var i = 0;
        if (!codedef &amp;&amp; !docdef) {
            return false;
        }
        if ((!docdef &amp;&amp; codedef) || (!codedef &amp;&amp; docdef)) {
            console.log('new definitions - updating "' + docpath +'"')
            return true;
        }        
        for (var u in docdef) {
            i++;
            if (!codedef[u] || docdef[u] != codedef[u].toString()) {
                console.log('definition of "' + u + '" changed - updating "' + docpath +'"')
                return true;
            }
        }
        // check that both doc and code have same number of functions
        for (var u in codedef) {
            i--;
            if (i &lt; 0) {
                console.log('new definitions - updating "' + docpath +'"')
                return true;
            }
        }
        return false;
    }
    db.get(docpath, function(err, doc) {
        if (!doc) {
            console.log('no design doc found updating "' + docpath +'"')
            return save_doc();
        }
        if (compare_def(doc.updates, code.updates) || compare_def(doc.views, code.views)) {
            return save_doc();
        }
        console.log('"' + docpath +'" up to date')            
    });
}

var EXAMPLE1_DDOC = {
    language: 'javascript',
    views: {
        active: {
            map: function (doc) {
                if (doc.lastsession) {
                    emit(parseInt(doc.lastsession / 1000), 1)
                }
            },
            reduce: function(keys, counts, rereduce) {
                return sum(counts)
            }
        },
        users: function(doc) {
            if (doc.created) {
                emit(parseInt(doc.created / 1000), 1)
            }
        }
    }    
}

var EXAMPLE2_DDOC = {
    language: 'javascript',
    views: {
        myview: function(doc) {
            if (doc.param1 &amp;&amp; doc.param2) {
                emit([doc.param1, doc.param2], null)
            }
        }
    }
}

update_views(DB_SOMETHING, '_design/example1', EXAMPLE1_DDOC)
update_views(DB_SOMETHING, '_design/example2', EXAMPLE2_DDOC)</pre>
<p>The code is pretty simple.</p>
<ol>
<li>First it loads the Cradle couchdb driver and creates needed databases if they do not already exist. In this example only single database &#8216;somedb&#8217; is created.</li>
<li>The update_views is responsible of keeping the design docs up to date. It loads the design doc from defined DB and compares it to the code defined in the design doc in this file. If it has changed (or missing) it will be recreated.</li>
<li>The example design docs (EXAMPLE1_DDOC and EXAMPLE2_DDOC) are simple design doc definitions as Javascript object. You&#8217;re familiar with CouchDB so this is self explanatory.</li>
<li>Lastly the code just calls the update_views to update the design documents.</li>
</ol>
<p>Now it&#8217;s possible to maintain the views in this Javascript file, the Node will make sure that the syntax is always valid.</p>
<p>Example output:</p>
<p>Views are up to date.</p>
<pre>
$ node cdb-views.js
"_design/example1" up to date
"_design/example2" up to date
</pre>
<p>Definition of view example2/myview has changed</p>
<pre>
$ node cdb-views.js
"_design/example1" up to date
definition of "myview" changed - updating "_design/example2"
</pre>
<p>Design doc example2 can not be found and is created.</p>
<pre>
$ node cdb-views.js
"_design/example1" up to date
no design doc found updating "_design/example2"
</pre>
<p>&nbsp;</p>
<p>&nbsp;</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/141/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/141/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/141/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=141&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/06/03/keeping-couchdb-design-docs-up-to-date-with-node-js/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Callbacks from Threaded Node.js C++ Extension</title>
		<link>http://bravenewmethod.wordpress.com/2011/03/30/callbacks-from-threaded-node-js-c-extension/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/03/30/callbacks-from-threaded-node-js-c-extension/#comments</comments>
		<pubDate>Wed, 30 Mar 2011 18:57:23 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Javascript]]></category>
		<category><![CDATA[addon]]></category>
		<category><![CDATA[javascript]]></category>
		<category><![CDATA[nodejs]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=134</guid>
		<description><![CDATA[Writing threaded Node.js extension requires some care. All Javascript in Node.js is executed in single main thread, so you can not simply call the V8 engine directly from your background thread. That would cause segfault. Recommended way to do this is to spawn new thread on background and use the libev events to wake up [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=134&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Writing threaded Node.js extension requires some care. All Javascript in Node.js is executed in single main thread, so you can not simply call the V8 engine directly from your background thread. That would cause segfault. Recommended way to do this is to spawn new thread on background and use the libev events to wake up the main thread to execute the Javascript callbacks.</p>
<p>Node.js framework has lots of ready stuff for implementing extensions,  but there is no simple example how to implement this kind extension so here it is.</p>
<h2>Add-on Source</h2>
<p>Save this source to texample.cc</p>
<pre>#include &lt;queue&gt;

// node headers
#include &lt;v8.h&gt;
#include &lt;node.h&gt;
#include &lt;ev.h&gt;
#include &lt;pthread.h&gt;
#include &lt;unistd.h&gt;
#include &lt;string.h&gt;

using namespace node;
using namespace v8;

// handles required for callback messages
static pthread_t texample_thread;
static ev_async eio_texample_notifier;
Persistent&lt;String&gt; callback_symbol;
Persistent&lt;Object&gt; module_handle;

// message queue
std::queue&lt;int&gt; cb_msg_queue = std::queue&lt;int&gt;();
pthread_mutex_t queue_mutex = PTHREAD_MUTEX_INITIALIZER;

// The background thread
static void* TheThread(void *)
{
    int i = 0;
    while(true) {
         // fire event every 5 seconds
        sleep(5);
       pthread_mutex_lock(&amp;queue_mutex);
       cb_msg_queue.push(i);
       pthread_mutex_unlock(&amp;queue_mutex);
       i++;
      // wake up callback
      ev_async_send(EV_DEFAULT_UC_ &amp;eio_texample_notifier);
    }
    return NULL;
}

// callback that runs the javascript in main thread
static void Callback(EV_P_ ev_async *watcher, int revents)
{
    HandleScope scope;

    assert(watcher == &amp;eio_texample_notifier);
    assert(revents == EV_ASYNC);

    // locate callback from the module context if defined by script
    // texample = require('texample')
    // texample.callback = function( ... ) { ..
    Local&lt;Value&gt; callback_v = module_handle-&gt;Get(callback_symbol);
    if (!callback_v-&gt;IsFunction()) {
         // callback not defined, ignore
         return;
    }
    Local&lt;Function&gt; callback = Local&lt;Function&gt;::Cast(callback_v);

    // dequeue callback message
    pthread_mutex_lock(&amp;queue_mutex);
    int number = cb_msg_queue.front();
    cb_msg_queue.pop();
    pthread_mutex_unlock(&amp;queue_mutex);

    TryCatch try_catch;

    // prepare arguments for the callback
    Local&lt;Value&gt; argv[1];
    argv[0] = Local&lt;Value&gt;::New(Integer::New(number));

    // call the callback and handle possible exception
    callback-&gt;Call(module_handle, 1, argv);

    if (try_catch.HasCaught()) {
        FatalException(try_catch);
    }
}

// Start the background thread
Handle&lt;Value&gt; Start(const Arguments &amp;args)
{
    HandleScope scope;

    // start background thread and event handler for callback
    ev_async_init(&amp;eio_texample_notifier, Callback);
    //ev_set_priority(&amp;eio_texample_notifier, EV_MAXPRI);
    ev_async_start(EV_DEFAULT_UC_ &amp;eio_texample_notifier);
    ev_unref(EV_DEFAULT_UC);
    pthread_create(&amp;texample_thread, NULL, TheThread, 0);

    return True();
}

void Initialize(Handle&lt;Object&gt; target)
{
    HandleScope scope;

    NODE_SET_METHOD(target, "start", Start);

    callback_symbol = NODE_PSYMBOL("callback");
    // store handle for callback context
    module_handle = Persistent&lt;Object&gt;::New(target);
}

extern "C" {
static void Init(Handle&lt;Object&gt; target)
{
    Initialize(target);
}

    NODE_MODULE(texample, Init);
}
</pre>
<p>Function walkthrough</p>
<ul>
<li>The Init function gets called when you <em>require(&#8216;texample&#8217;)</em> the native module.</li>
<li>Init function defines module function <em>start </em>that will be called by javascript. It also stores the module handle for locating and calling the script defined callback on right context.</li>
<li>Start function initializes the libev event notifier and starts the background thread TheThread</li>
<li>Thread TheThread simply loops, sleeps and puts incrementing integers to our queue and wakes up the main thread</li>
<li>Callback function gets waken up the libev and it locates the javascript function <em>callback</em> and calls it</li>
</ul>
<h2>Building</h2>
<p>Copy this to the &#8216;wscript&#8217;  file.</p>
<pre>def set_options(opt):
  opt.tool_options("compiler_cxx")

def configure(conf):
  conf.check_tool("compiler_cxx")
  conf.check_tool("node_addon")

def build(bld):
  obj = bld.new_task_gen("cxx", "shlib", "node_addon")
  obj.cxxflags = ["-g", "-D_FILE_OFFSET_BITS=64",
                  "-D_LARGEFILE_SOURCE", "-Wall"]
  obj.target = "texample"
  obj.source = "texample.cc"</pre>
<p>Compile the code with node-waf</p>
<pre>$ node-waf configure
$ node-waf build</pre>
<h2>Running</h2>
<p>Start node shell</p>
<pre>$ node</pre>
<p>Load the native add on module</p>
<pre>&gt; texample = require('./build/default/texample');</pre>
<p>Define the callback function that the module will call</p>
<pre>&gt; texample.callback = function(i) {
... console.log('Bang: ' + i);
... }
&gt;</pre>
<p>Call start to kick of the background thread</p>
<pre>&gt; texample.start();
true
&gt;</pre>
<p>Wait for 5 seconds, you&#8217;ll start seeing your callback getting triggered every 5 seconds.</p>
<pre>&gt; Bang: 0
Bang: 1
&gt; Bang: 2</pre>
<p>Have fun!</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/134/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/134/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/134/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=134&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/03/30/callbacks-from-threaded-node-js-c-extension/feed/</wfw:commentRss>
		<slash:comments>3</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Embedding V8 Javascript Engine and Go</title>
		<link>http://bravenewmethod.wordpress.com/2011/03/30/embedding-v8-javascript-engine-and-go/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/03/30/embedding-v8-javascript-engine-and-go/#comments</comments>
		<pubDate>Wed, 30 Mar 2011 08:00:29 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[golang]]></category>
		<category><![CDATA[javascript]]></category>
		<category><![CDATA[v8]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=123</guid>
		<description><![CDATA[This is two common examples merged together; how to run V8 as embedded and how to call C modules from Go language. I&#8217;m using Ubuntu 10.04 x64 with standard gcc toolchain. Step 1. Compile v8 Get v8 source and build v8 as shared library. Use this command line and copy libv8.so to to your project [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=123&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>This is two common examples merged together; how to run V8 as embedded and how to<br />
call C modules from Go language. I&#8217;m using Ubuntu 10.04 x64 with standard gcc toolchain.</p>
<h2>Step 1. Compile v8</h2>
<p><a href="http://code.google.com/apis/v8/build.html">Get v8 source</a> and build v8 as shared library.<br />
Use this command line and copy libv8.so to to your project directory:</p>
<pre>$ scons mode=release library=shared snapshot=on arch=x64
$ cp libv8.so ~/v8example</pre>
<h2>Step 2. C Wrapper for V8</h2>
<p>Write C++ function that accepts javascript source code as argument and compiles and runs it in v8.</p>
<p>Header file:</p>
<pre>#ifndef _V8WRAPPER_H
#define _V8WRAPPER_H

#ifdef __cplusplus
extern "C" {
#endif
    // compiles and executes javascript and returns the script return value as string
    char * runv8(const char *jssrc);

#ifdef __cplusplus
}
#endif

#endif // _V8WRAPPER_H</pre>
<p>Source file, this is slightly modified version from official v8 <a href="http://code.google.com/apis/v8/embed.html">C++ embedders guide</a>.</p>
<pre>#include &lt;v8.h&gt;
#include &lt;string.h&gt;

#include "v8wrapper.h"

using namespace v8;

char * runv8(const char *jssrc)
{
    // Create a stack-allocated handle scope.
    HandleScope handle_scope;

    // Create a new context.
    Persistent&lt;Context&gt; context = Context::New();

    // Enter the created context for compiling and
    // running the script.
    Context::Scope context_scope(context);

    // Create a string containing the JavaScript source code.
    Handle&lt;String&gt; source = String::New(jssrc);

    // Compile the source code.
    Handle&lt;Script&gt; script = Script::Compile(source);

    // Run the script
    Handle&lt;Value&gt; result = script-&gt;Run();

    // Dispose the persistent context.
    context.Dispose();

    // return result as string, must be deallocated in cgo wrapper
    String::AsciiValue ascii(result);
    return strdup(*ascii);
}</pre>
<p>Makefile.wrapper</p>
<pre>V8_INC=/home/user/builds/v8/include

CC=g++
CFLAGS=-c -fPIC -I$(V8_INC)
SOURCES=v8wrapper.cc
OBJECTS=$(SOURCES:.cc=.o)
TARGET=libv8wrapper.so

all: $(TARGET)

.cc.o:
    $(CC) $(CFLAGS) $&lt; -o $@

$(TARGET): $(OBJECTS)
    ld -G -o $@ $(OBJECTS)</pre>
<p>Compile to get the shared library</p>
<pre>$ make -f Makefile.wrapper</pre>
<p>You should end up with file <em>libv8wrapper.so</em></p>
<h2>Step 3. CGO Wrapper for Go</h2>
<p>Now define a CGO wrapper source file that exposes the v8 to the Go language.</p>
<p>Go source file for the CGO compiler. Note that the comments are functional and contain instructions to cgo compiler. The libv8.so and just compiled libv8wrapper.so are assumed to be in current working directory for linking.</p>
<pre>// #cgo LDFLAGS: -L. -lv8wrapper -lv8  -lstdc++ -pthread
// #include &lt;stdlib.h&gt;
// #include "v8wrapper.h"
import "C"
import "unsafe"

func RunV8(script string) string {

  // convert Go string to nul terminated C-string
  cstr := C.CString(script)
  defer C.free(unsafe.Pointer(cstr))

  // run script and convert returned C-string to Go string
  rcstr := C.runv8(cstr)
  defer C.free(unsafe.Pointer(rcstr))

  return C.GoString(rcstr)
}</pre>
<p>CGO Makefile. Note here that you need to have GOROOT defined. The OS and Architecture are defined here too.</p>
<pre>include $(GOROOT)/src/Make.inc

GOOS=linux
GOARCH=amd64

TARG=v8runner
CGOFILES=\
    v8runner.go\

include $(GOROOT)/src/Make.pkg</pre>
<p>Compile to Go package <em>v8runner</em> and install it</p>
<pre>$ make -f Makefile.cgo
$ make -f Makefile.cgo install</pre>
<p>Install copies the package file to the $GOROOT/pkg/linux_amd64/v8runner.a where it can be imported by Go compiler and linker.</p>
<h2>Step 4. The GO program</h2>
<p>Now you&#8217;re finally ready to make plain Go program that runs v8.</p>
<pre>package main

import "v8runner"
import "fmt"

func main() {
    r: = v8runner.RunV8("'Hello Go World'")
    fmt.Println(r)
}</pre>
<p>Makefile.hello</p>
<pre>include $(GOROOT)/src/Make.inc

TARG=hello
GOFILES=hello.go</pre>
<p>Compile</p>
<pre>$ make -f Makefile.hello</pre>
<p>Set <em>LD_LIBRARY_PATH</em> to current directory, assuming you have libv8.so and libv8wrapper.so there.</p>
<pre>$ export LD_LIBRARY_PATH=.</pre>
<p>Run the program</p>
<pre>$ ./hello
Hello Go World</pre>
<p>To recap the steps</p>
<ol>
<li>Shared C++ library that exposes C-function to run javascript : libv8wrapper.so</li>
<li>CGO compiled wrapper that passes arguments between Go and C world and calls the C functions: v8runner</li>
<li>Go program that imports the package and uses it normally.</li>
</ol>
<p>This hack has some  caveats.</p>
<ul>
<li>There is currently no way to link everything statically, as the CGO does not support it. You need to use shared libraries.</li>
<li>I&#8217;m not aware of any easy way to call back Go from the CGO wrapped C++. You need wrappers over wrappers as demonstrated by this post: <a href="http://groups.google.com/group/golang-nuts/msg/c98b4c63ba739240">http://groups.google.com/group/golang-nuts/msg/c98b4c63ba739240</a>. Matroska ftw.</li>
<li>Only one thread at a time can use v8 instance. You need to use Isolates (See v8 source for more information) how to support multiple instances. Still only one thread at a time can use specific instance</li>
</ul>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/123/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/123/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/123/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=123&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/03/30/embedding-v8-javascript-engine-and-go/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Developing on Google App Engine for Production</title>
		<link>http://bravenewmethod.wordpress.com/2011/03/23/developing-on-google-app-engine-for-production/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/03/23/developing-on-google-app-engine-for-production/#comments</comments>
		<pubDate>Wed, 23 Mar 2011 10:02:29 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[app engine]]></category>
		<category><![CDATA[production]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=111</guid>
		<description><![CDATA[If you&#8217;re considering App Engine as platform for your next big thing, here is potpourri of observations that you might find worth reading. This is not tutorial, and basic App Engine hands-on experience is required. Stuff here is written from experiences in Python environment, for Java mileage may vary. There is also lots of functionality [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=111&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>If you&#8217;re considering App Engine as platform for your next big thing, here is potpourri of observations that you might find worth reading. This is not tutorial, and basic App Engine hands-on experience is required. Stuff here is written from experiences in Python environment, for Java mileage may vary. There is also lots of functionality that is not covered here because I didn&#8217;t personally use them or they are otherwise well documented.</p>
<h2>Queries and Indexes</h2>
<p>Applications can use basically only two queries: Get data entity by key or get data entities by range.  In ranged query key can be property value, or composite of property values. Anything that needs more than one filter property and specific order will need composite index definition.</p>
<p>For key ordered queries App Engine supports self-merge for table, but in real life it doesn&#8217;t work always very far as when number of entities grow you may eventually hit the error <em>&#8220;NeedIndexError: The built-in indices   are not efficient enough for this   query and your data. Please add a   composite index for this query.&#8221;</em>. This means that some values are too sparse to filter data efficiently. e.g. you have 30 000 entities and one of the properties you&#8217;re querying is boolean flag that is either True or False to every entity.</p>
<p>App engine uses composite keys for  building indexes for  queries that need specific order. Be careful when combining more than one list property values in composite index,  App Engine will build permutations of all key values and even with modest lists you end up with hundreds of index entries.</p>
<p>For example define model with two list properties and timestamp</p>
<pre>class MyModel(db.Model):
   created = db.DateTimeProperty()
   list1 = db.StringListProperty()
   list2 = db.ListProperty(int)</pre>
<p>Define composite index for the model</p>
<pre>- kind: MyModel
  properties:
  - name: list1
  - name: list2
  - name: created
    direction: desc</pre>
<p>Put entity</p>
<pre>m = MyModel()
m.list1 = ["cat", "dog", "pig", "angry", "bird"]
m.list2 = [1, 2, 3]
m.put()</pre>
<p>This would create following reverse and custom index entries</p>
<ul>
<li> 5 for each item in list1</li>
<li> 3 for each item in list2</li>
<li> 1 for created</li>
<li> 5 * 3 = 15 entries for permutations (cat, 1, created), (cat, 2, created), (cat, 3, created), (dog, 1, created), (dog, 2, created), &#8230;</li>
</ul>
<p>Total 15 + 1 + 3 + 5 = 24 entries. This is not much in the example, but if grows exponentially when number of list entries and indexes increases. 3 lists in index each having 10 values would mean 10^3 = 1000 index entries.</p>
<p>Maximum number of index entries is 5000 per entity, and this is shared with implicit reverse property index and explicit custom indexes. For example if you have listproperty that you use in custom index, it can have at maximum ~2500 values because the implicit reverse index will take 2500 and the custom index rest 2500 totalling 5000.</p>
<p>Remember to set <em>indexed=false</em> in property definition if you don&#8217;t need to query against property, this saves both space and CPU.</p>
<p>Query latencies are pretty ok,  ~100ms for few dozen entities and you can use IN-operator to  make parallel queries. Just make sure that your &#8216;IN&#8217; queries  do not  return lots of overlapping results as that can hurt performance. Direct  get by key latencies are very good. (~20ms). Naturally latency increases linearly if  your objects are very large, especially with long listproperties.</p>
<p>Text search is in App Engine<a href="http://code.google.com/appengine/docs/roadmap.html"> roadmap and under development</a>. Meanwhile  you can make simple startswith queries against single property or list of strings. Queries are identical in both cases.<br />
Single property startswith query</p>
<pre>class User(db.Model):
   name = db.StringProperty()

users = User.all().filter('name &gt;=', query).filter('name &lt;', query + '\ufffd').fetch()</pre>
<p>Listproperty startswith query</p>
<pre>class SomeModel(db.Model):
   keywords = db.StringListProperty()

models = SomeModel.all().filter('keywords &gt;=', query).filter('keywords &lt;', query + '\ufffd').fetch()</pre>
<p>Note that in latter the sort order may not what you wish for as you must  sort always first by first inequality filter property, in this example  keywords. Just keep in mind the index growth when you add more properties to the query.</p>
<p>Soft memory limit is &lt; 200MB that is reached easily if you&#8217;ve large entities, don&#8217;t rely that you can do lots of in-memory sorting. Especially Python memory overhead is pretty big. As rule of thumb you can manipulate ~15000 properties per call. (e.g. 1000 entities each having 15 properties). Each element in listproperty is counted as property.</p>
<p>You&#8217;ll see often DeadLineExceedError in the logs, nothing you can do to  these except to use highly replicated datastore. Just note that it has  much higher CPU cost. Curiously frequency of these errors seem pretty constant and independent of the load. Maybe App Engine gives more priority to more popular apps.<a href="http://code.google.com/appengine/docs/python/datastore/queries.html#Queries_and_Indexes"></a></p>
<h2>Quotas</h2>
<p>Depends lot of your application, but at least in my experience the CPU is limiting factor for most of the use cases. This is mainly because you need to do most of the work when inserting new objects instead of when querying them, so even rarely used queries will cost you in every single insert. Queries needs indexes and storing entities with indexes cost API CPU time. Both your own application execution and the API (DB, etc..) execution time is counted in your quota. Be sure to measure and estimate your costs. Putting entities with very large listproperties that use custom indexes can easily cost 25-60seconds of CPU time per entity.</p>
<p>In case combined CPU time (app + api) grows large enough (&gt; ~1000ms) App Engine will warn you in logs that the call uses high amount of CPU and may run over it&#8217;s quota. Curiously it makes this same warning even when you have billing enabled but  it wont&#8217; restrict your app in that case however.</p>
<h2>Scalability is Latency</h2>
<p>App Engine scalability rules are complex but what mostly matters is your average latency. If you&#8217;ve only slow request handler (latency &gt; 500ms) app engine will limit your scalability. It&#8217;s not bad to have few slow ones, but make sure that the average is somewhere around ~250ms or less. In worst case App Engine refuses to start new instances and queues new requests to already serving instances thus growing the user perceived request latency. You can observe this from App Engine dashboard log entries showing &#8216;pending_ms&#8217; times.</p>
<p>Note that cpu time is not same thing as latency, for example these two pieces of code have roughly same CPU cost, but latter has only 1/3 of latency</p>
<p>Slow put</p>
<pre> e1.put()
 e2.put()
 e3.put()</pre>
<p>Fast put</p>
<pre>db.put([e1, e2, e3])</pre>
<p>Slow get</p>
<pre> e1 = db.get(key1)
 e2 = db.get(key2)
 e3 = db.get(key3)</pre>
<p>Fast get</p>
<pre>ents = db.get([key1, key2, key3])</pre>
<h2>Parental fetch, aka relation index, aka. parent reference</h2>
<p>App Engine DB API does not support partial fetch that can be issue if you have very large listproperties in the entities. It&#8217;s possible to achieve something similar by using parent keys. For example if you&#8217;ve large number of elements in listproperty, you can make key only query againts that property and fetch only the keys. Then get keys parent value and fetch entity you need.</p>
<pre>class Message(db.Model):
  text = db.StringProperty()

class MsgIdx(db.Model)
  recipients = db.ListProperty(db.Key)

msg = Message(text="Hello World")
msg.put()

idx = MsgIndex(key_name='idx', parent=msg)
idx.recipients.append(user1.key())
idx.recipients.append(user2.key())
 ...
idx.put()</pre>
<p>Query messages where userX is in recipient list, first get keys</p>
<pre>keys = MsgIndex.all(keys_only=True).filter('recipients', userX).fetch(100)</pre>
<p>query actual message objects</p>
<pre>msg = db.get([k.parent() for k in keys])</pre>
<p>In this way you avoid serializing the potentially large recipient list completely.</p>
<p>See <a href="http://www.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html">Brett Slatkin&#8217;s presentation</a> for more details.</p>
<h2>Transactions</h2>
<p>App Engine DB supports  transactions but it&#8217;s not possible to  implement global consistency, because transaction can only operate  objects in single entity group. For example if you have entity A and B that  have no parent keys, you can not operate them both in single transaction. <a href="http://code.google.com/appengine/docs/python/datastore/entities.html#Entity_Groups_and_Ancestor_Paths">Entity group is all entities</a> with same parent root key, entity  without parent key is its own group.</p>
<p>Word of warning, when you use transactions all entities with same parent  key are locked for transaction (entity group), in general there should  not be more than 1-3 updates per second for single entity group or  you&#8217;ll get lots of transaction collisions retries that will eat your CPU  and increase latency. Collision retries are logged as warnings in App Engine console.</p>
<h2>Pre-fetch Referenceproperties</h2>
<p>Prefetch referenceproperties before accessing them in sequence.<br />
Bad, will trigger separate DB query for user property each time</p>
<pre>class Foo(db.Model):
  user = db.ReferenceProperty(User)

foos = Foo.all().fetch(100)
for f in foo:
  print f.user.name</pre>
<p>Good, See <a href="http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine">prefetch_reprop function implementation here</a>.</p>
<pre>foos = Foo.all().fetch(100)
prefetch_refprop(foos,  Foo.user)
for f in foo:
  print f.user.name</pre>
<p>This will decrease latency and API CPU time significantly</p>
<h2>Debugging and Profiling</h2>
<p>Standard  Python debugger does not work in App Engine development server, but you  can use following wrapper and start dev_appserver.py from command line  to get into debugger.</p>
<pre>def appe_set_trace():
  import pdb, sys
  debugger = pdb.Pdb(stdin=sys.__stdin__,
  stdout=sys.__stdout__)
  debugger.set_trace(sys._getframe().f_back)</pre>
<p>API profiling. Define appengine_config.py in your app and define access stats handler in app.yaml.</p>
<pre>def webapp_add_wsgi_middleware(app):
  from google.appengine.ext.appstats import recording
  app = recording.appstats_wsgi_middleware(app)
  return app

- url: /stats.*
  script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py
  login: admin</pre>
<p>CPU profiling. Define profiling wrapper that dumps the CPU times to the log.</p>
<pre>def real_main():
  # Run the WSGI CGI handler with that application.
  util.run_wsgi_app(application)

def profile_main():
  # This is the main function for profiling
  # We've renamed our main() above to real_main()
  import cProfile, pstats
  prof = cProfile.Profile()
  prof = prof.runctx("real_main()", globals(), locals())
  stream = StringIO.StringIO()
  stats = pstats.Stats(prof, stream=stream)
  stats.sort_stats("cumulative")  # time or cumulative
  stats.print_stats(80)  # 80 = how many to print
  # The rest is optional.
  # stats.print_callees()
  # stats.print_callers()
  logging.info('Profile data:\n%s', stream.getvalue());

if __name__ == '__main__':
  main = profile_main
  #main = real_main
  main()</pre>
<h2>Task TransientErrors</h2>
<p>Task add fails often with transienterror, just retry it once more and you should get rarely failed task adds.</p>
<pre>try:
   taskqueue.add(...
except taskqueue.TransientError:
   taskqueue.add(..  # retry once more</pre>
<h2>Misc</h2>
<p>Other things.</p>
<ul>
<li>Static files are not served from application environment, your application can not access them programmatically.</li>
<li>Urlfetch service has maximum of 10 sec timeout and can do maximum 10 parallel queries per instance. Queries fail occasionally with application error that is usually caused by server timeout. Queries are done from pretty random source ip&#8217;s that are shared by all other  engine apps. You can not override header.</li>
<li>Naked domains are not supported (like example.com)</li>
<li>Memcache  lifetime can very short, mere minutes but if your application is  popular App Engine might give more priority. Use multi get  and set when ever possible.</li>
</ul>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/111/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/111/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/111/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=111&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/03/23/developing-on-google-app-engine-for-production/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Interpreting Go Socket Errors</title>
		<link>http://bravenewmethod.wordpress.com/2011/03/17/interpreting-go-socket-errors/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/03/17/interpreting-go-socket-errors/#comments</comments>
		<pubDate>Thu, 17 Mar 2011 12:49:54 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Experimental]]></category>
		<category><![CDATA[go]]></category>
		<category><![CDATA[golang]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=104</guid>
		<description><![CDATA[Go sockets returns error variables when something goes wrong, and the different error codes are documented here in the Go documentation. However I was not able to find coherent example that would show how the error variable is supposed to be used. Canonical way seems to be just checking it against nill and dump it [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=104&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Go sockets returns error variables when something goes wrong, and the different error codes are documented here in the <a href="http://golang.org/pkg/os/#Error">Go documentation</a>. However I was not able to find coherent example that would show how the error variable is supposed to be used. Canonical way seems to be just checking it against nill and dump it out in case it&#8217;s something else, like this:</p>
<pre>n, err := conn.Read(buffer[:])
if err != nill {
    fmt.Printf("%v\n", err)
}</pre>
<p>Real applications (especially system applications) need to branch based on error to recover properly, so just error description is not enough. I made here example what it&#8217;s possible to deduct from the error variable.</p>
<pre>conn, err := net.Dial("tcp", "", "example.com:80")
n, err := conn.Read(buffer[:])

if err != nil {

    // print error string e.g.
    // "read tcp example.com:80: resource temporarily unavailable"
    fmt.Printf("reader %v\n", err)

    // print type of the error, e.g. "*net.OpError"
    fmt.Printf("%T", err)

    if err == os.EINVAL {
      // socket is not valid or already closed
      fmt.Println("EINVAL");
    }
    if err == os.EOF {
      // remote peer closed socket
      fmt.Println("EOF");
    }

    // matching rest of the codes needs typecasting, errno is
    // wrapped on OpError
    if e, ok := err.(*net.OpError); ok {
       // print wrapped error string e.g.
       // "os.Errno : resource temporarily unavailable"
       fmt.Printf("%T : %v\n", e.Error, e.Error)
       if e.Timeout() {
         // is this timeout error?
         fmt.Println("TIMEOUT")
       }
       if e.Temporary() {
         // is this temporary error?  True on timeout,
         // socket interrupts or when buffer is full
         fmt.Println("TEMPORARY")
       }

      // specific granular error codes in case we're interested
     switch e.Error {
        case os.EAGAIN:
           // timeout
           fmt.Println("EAGAIN")
       case os.EPIPE:
          // broken pipe (e.g. on connection reset)
          fmt.Println("EPIPE")
       default:
          // just write raw errno code, can be platform specific
          // (see syscall for definitions)
          fmt.Printf("%d\n", int64(e.Error.(os.Errno)))
     }
 }</pre>
<p>For example in case read times out, the code would print following</p>
<pre>read tcp 192.0.32.10:80: resource temporarily unavailable
*net.OpError
os.Errno : resource temporarily unavailable
TIMEOUT
TEMPORARY
EAGAIN</pre>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/104/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/104/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/104/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=104&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/03/17/interpreting-go-socket-errors/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Apple Push Notifications with Go language</title>
		<link>http://bravenewmethod.wordpress.com/2011/02/25/apple-push-notifications-with-go-language/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/02/25/apple-push-notifications-with-go-language/#comments</comments>
		<pubDate>Fri, 25 Feb 2011 10:06:44 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Experimental]]></category>
		<category><![CDATA[apn]]></category>
		<category><![CDATA[go]]></category>
		<category><![CDATA[golang]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=91</guid>
		<description><![CDATA[I started to familiarize myself to the Go language, and decided to do the usual try out, i.e. sending Apple Push Notifications. It&#8217;s my personal usability benchmark for new programming environments. So far in the series Push notifications with Node.js Push notifications with Erlang Step 1. Prerequisites Get and build Go. Example here was done [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=91&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>I started to familiarize myself to the <a href="http://golang.org/">Go language</a>, and decided to do the usual try out, i.e. sending Apple Push Notifications. It&#8217;s my personal usability benchmark for new programming environments. So far in the series</p>
<ul>
<li><a title="Apple Push Notifications with Node.js" href="http://bravenewmethod.wordpress.com/2010/12/09/apple-push-notifications-with-node-js/">Push notifications with Node.js</a></li>
<li><a title="Apple Push Notifications with Erlang" href="http://bravenewmethod.wordpress.com/2011/02/16/apple-push-notifications-with-erlang/">Push notifications with Erlang</a></li>
</ul>
<h2>Step 1. Prerequisites</h2>
<p>Get and build Go. Example here was done on Ubuntu 10.04 LTS x64 with Go installed based on instructions here at <a href="http://golang.org/doc/install.html">Go getting started guide</a>.</p>
<ul>
<li><a title="Apple Push Notifications with Node.js" href="http://bravenewmethod.wordpress.com/2010/12/09/apple-push-notifications-with-node-js/">Read introduction to Apple Push here</a> and get application and private key sandbox certificates as .pem files.</li>
<li>And of course you need to have 32 byte push token from your iOS application.</li>
</ul>
<h2>Step 2. The Code.</h2>
<p>The code here is complete, copy it to file apn.go.</p>
<p>Make sure you change the certificate files (cert.pem and key-noenc.pem) to point to your own certificate files. Also, replace the push token with your own push token, it&#8217;s written as hexadecimal string in this example for clarity.</p>
<pre>package main

import (
   "crypto/tls"
   "fmt"
   "net"
   "json"
   "os"
   "time"
   "bytes"
   "encoding/hex"
   "encoding/binary"
)

func main() {

   // load certificates and setup config
   cert, err := tls.LoadX509KeyPair("cert.pem", "key-noenc.pem")
   if err != nil {
       fmt.Printf("error: %s\n", err.String())
       os.Exit(1)
   }
   conf := &amp;tls.Config {
        Certificates: []tls.Certificate{cert},
   }

   // connect to the APNS and wrap socket to tls client
   conn, err := net.Dial("tcp", "", "gateway.sandbox.push.apple.com:2195")
   if err != nil {
      fmt.Printf("tcp error: %s\n", err.String())
      os.Exit(1)
   }
   tlsconn := tls.Client(conn, conf)

   // Force handshake to verify successful authorization.
   // Handshake is handled otherwise automatically on first
   // Read/Write attempt
   err = tlsconn.Handshake()
   if err != nil {
      fmt.Printf("tls error: %s\n", err.String())
      os.Exit(1)
   }
   // informational debugging stuff
   state := tlsconn.ConnectionState()
   fmt.Printf("conn state %v\n", state)

   // prepare binary payload from JSON structure
   payload := make(map[string]interface{})
   payload["aps"] = map[string]string{"alert": "Hello Push"}
   bpayload, err := json.Marshal(payload)

   // decode hexadecimal push device token to binary byte array
   btoken, _ := hex.DecodeString("6b4628de9317c80edd1c791640b58fdfc46d21d0d2d1351687239c44d8e30ab1") 

   // build the actual pdu
   buffer := bytes.NewBuffer([]byte{})
   // command
   binary.Write(buffer, binary.BigEndian, uint8(1))

   // transaction id, optional
   binary.Write(buffer, binary.BigEndian, uint32(1))

   // expiration time, 1 hour
   binary.Write(buffer, binary.BigEndian, uint32(time.Seconds() + 60*60))

   // push device token
   binary.Write(buffer, binary.BigEndian, uint16(len(btoken)))
   binary.Write(buffer, binary.BigEndian, btoken)

   // push payload
   binary.Write(buffer, binary.BigEndian, uint16(len(bpayload)))
   binary.Write(buffer, binary.BigEndian, bpayload)
   pdu := buffer.Bytes()

   // write pdu
   _, err = tlsconn.Write(pdu)
   if err != nil {
      fmt.Printf("write error: %s\n", err.String())
      os.Exit(1)
   }

   // wait for 5 seconds error pdu from the socket
   tlsconn.SetReadTimeout(5*1E9)

   readb := [6]byte{}
   n, err := tlsconn.Read(readb[:])
   if n &gt; 0 {
     fmt.Printf("received: %s\n", hex.EncodeToString(readb[:n]))
   }

   tlsconn.Close()
}
﻿</pre>
<h2>Step 3. Compile and Run</h2>
<p>Simple</p>
<pre>$ 6g apn.go
$ 6l apn.6
$ ./6.out
conn state {true 47}
$
</pre>
<p>If everything went fine, the program exits within few seconds and  you&#8217;ll see your push notification appear on your iPhone.</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/91/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/91/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/91/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=91&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/02/25/apple-push-notifications-with-go-language/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Socket binary data stream parsing in Erlang</title>
		<link>http://bravenewmethod.wordpress.com/2011/02/22/socket-binary-data-stream-parsing-in-erlang/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/02/22/socket-binary-data-stream-parsing-in-erlang/#comments</comments>
		<pubDate>Tue, 22 Feb 2011 11:20:06 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Experimental]]></category>
		<category><![CDATA[erlang]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=83</guid>
		<description><![CDATA[Erlang code has two different ways of reading from the socket, active and passive. In passive mode, your code calls recv to the socket to receive bytes. In active mode you install controlling process to the socket and receive data as Erlang messages. Binary data parsing is more complicated on the latter, as application code [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=83&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Erlang code has two different ways of reading from the socket, active and passive. In passive mode, your code calls <a href="http://www.erlang.org/doc/man/gen_tcp.html#recv-2">recv</a> to the socket to receive bytes. In active mode you install controlling process to the socket and receive data as Erlang messages.</p>
<p>Binary data parsing is more complicated on the latter, as application code doesn&#8217;t have any control over size of the packets (or flow control for that matter) that it receives. It could be 1 byte, few kilobytes or whatever. Naive pattern matching fails if you don&#8217;t get exactly the amount of bytes you want.</p>
<p>Lets assume simple binary protocol, where you need to read packets that are varying in length and formatted as following. Timestamp is defined in first 4 bytes, next 2 bytes define length of payload followed by the payload itself.</p>
<pre>|    Timestamp   |  Len  |    Payload      |
|  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | ... |</pre>
<p>Incidentally, this is the data format used by Apple Push notification feedback service.</p>
<p>First lets define a function that accepts binary Data and process Pid where it sends parsed result. This tail recursive function simply matches as many packets from data as possible, and returns with remaining unparsed data.</p>
<pre>match_data(Data, Parent) -&gt;
   case Data of
      &lt;&lt;Timestamp:32/big, Size:16/big, PushToken:Size/binary-unit:8, Rest/binary&gt;&gt; -&gt;
         Parent ! {Timestamp, PushToken}, % notify parent
         %% parse rest of the data
        match_data(Rest, Parent);
      Rest -&gt;
         %% no match
        Rest
   end.</pre>
<p>Then function that actually receives the data from the socket. It receives arbitrary pieces of data, concatenates it to existing unparsed data and calls the match_data handler to make actual packet matching. Then it loops again with unparsed portion of data.</p>
<pre>loop(Bin, Parent) -&gt;
   receive
      {_, _Sock, Data} -&gt;
         loop(match_data(erlang:list_to_binary([Bin, Data]), Parent), Parent);
      {ssl_closed, _Sock} -&gt; ok;
      {_event, _Event} -&gt;
         error_logger:error_msg("Unexpected", [_event, _Event])
   end.</pre>
<p>Install the loop as controlling process, with initially empty &#8220;seed&#8221; data.</p>
<pre>Pid = self(),
ssl:setopts(Sock, [{active, true}, {mode, binary}]),
ssl:controlling_process(Sock, spawn(fun() -&gt; loop(&lt;&lt;&gt;&gt;, Pid) end)).</pre>
<p>This way data is parsed correctly, no matter what size of chunks data is returned from the socket.</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/83/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/83/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/83/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=83&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/02/22/socket-binary-data-stream-parsing-in-erlang/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Node.js TLS client example</title>
		<link>http://bravenewmethod.wordpress.com/2011/02/21/node-js-tls-client-example/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/02/21/node-js-tls-client-example/#comments</comments>
		<pubDate>Mon, 21 Feb 2011 10:21:44 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Javascript]]></category>
		<category><![CDATA[nodejs]]></category>
		<category><![CDATA[ssl]]></category>
		<category><![CDATA[tls]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=77</guid>
		<description><![CDATA[Couldn&#8217;t find good end to end example of Node.js new raw SSL client API, so here is one. This snippet just connects to the https://encrypted.google.com and fetches the front page. tls = require('tls'); // callback for when secure connection established function connected(stream) { if (stream) { // socket connected stream.write("GET / HTTP/1.0\n\rHost: encrypted.google.com:443\n\r\n\r");   } else [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=77&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Couldn&#8217;t find good end to end example of Node.js new raw SSL client API, so here is one. This snippet just connects to the https://encrypted.google.com and fetches the front page.</p>
<pre>tls = require('tls');

// callback for when secure connection established
function connected(stream) {
    if (stream) {
       // socket connected
      stream.write("GET / HTTP/1.0\n\rHost: encrypted.google.com:443\n\r\n\r");  
    } else {
      console.log("Connection failed");
    }
}

// needed to keep socket variable in scope
var dummy = this;

// try to connect to the server
dummy.socket = tls.connect(443, 'encrypted.google.com', function() {
   // callback called only after successful socket connection
   dummy.connected = true;
   if (dummy.socket.authorized) {
      // authorization successful
      dummy.socket.setEncoding('utf-8');
      connected(dummy.socket);
   } else {
      // authorization failed
     console.log(dummy.socket.authorizationError);
     connected(null);
   }
});

dummy.socket.addListener('data', function(data) {
   // received data
   console.log(data);
});

dummy.socket.addListener('error', function(error) {
   if (!dummy.connected) {
     // socket was not connected, notify callback
     connected(null);
   }
   console.log("FAIL");
   console.log(error);
});

dummy.socket.addListener('close', function() {
 // do something
});</pre>
<p>If you want to use client certificate authentication, define the options and give that as additional parameter to the tls.connect call.</p>
<pre>var keyPem = fs.readFileSync("key-noenc.pem", encoding='ascii');
var certPem = fs.readFileSync("cert.pem", encoding='ascii');
var options = {key:keyPem, cert:certPem };

...
dummy.socket = tls.connect(443, 'some.example.com', options, function() {
....</pre>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/77/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/77/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/77/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=77&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/02/21/node-js-tls-client-example/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
		<item>
		<title>Apple Push Notifications with Erlang</title>
		<link>http://bravenewmethod.wordpress.com/2011/02/16/apple-push-notifications-with-erlang/</link>
		<comments>http://bravenewmethod.wordpress.com/2011/02/16/apple-push-notifications-with-erlang/#comments</comments>
		<pubDate>Wed, 16 Feb 2011 13:36:38 +0000</pubDate>
		<dc:creator>tikonen</dc:creator>
				<category><![CDATA[Experimental]]></category>
		<category><![CDATA[apns]]></category>
		<category><![CDATA[erlang]]></category>

		<guid isPermaLink="false">http://bravenewmethod.wordpress.com/?p=65</guid>
		<description><![CDATA[Continuing from the Node.js based example I wrote earlier, here is example how to do the same with Erlang. You can check more details from the previous post, but as reminder the Apple Push Notification interface is simple binary based protocol that you use over SSL authenticated socket. 1. Prerequisites I assume you have erlang [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=65&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p>Continuing from the <a title="Apple Push Notifications with Node.js" href="http://bravenewmethod.wordpress.com/2010/12/09/apple-push-notifications-with-node-js/">Node.js based example</a> I wrote earlier, here is example how to do the same with Erlang. You can check more details from the previous post, but as reminder the Apple Push Notification interface is simple binary based protocol that you use over SSL authenticated socket.</p>
<h2>1. Prerequisites</h2>
<p>I assume you have erlang installed, the version I&#8217;m using here is Erlang R13B03 (erts-5.7.4).</p>
<p>Check instructions here at <a title="Apple Push Notifications with Node.js" href="../2010/12/09/apple-push-notifications-with-node-js/">Node.js based example</a> how to get the push certificates as .pem files.</p>
<p>Install <a href="https://github.com/mochi/mochiweb">mochiweb</a> package in your erlang environment.</p>
<p>Check that you&#8217;ve all set.</p>
<pre>$ ERL_LIBS=. erl
Erlang R13B03 (erts-5.7.4)  [64-bit] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
2&gt; mochijson:encode("cat").
"\"cat\""
3&gt; application:start(ssl).
ok</pre>
<p>Later releases of Erlang may require you to start &#8216;crypto&#8217; and &#8216;public_key&#8217; applications before starting ssl.</p>
<h2>2. Sending Push Notification</h2>
<p>First code to convert hexadecimal strings to binary format. This is mainly for readability for the example. I don&#8217;t remember where I snacked that code, but it seems to be found from several sites around the Intertubes.</p>
<pre>-module(hex).
-export([bin_to_hexstr/1,hexstr_to_bin/1]).

bin_to_hexstr(Bin) -&gt;
   lists:flatten([io_lib:format("~2.16.0B", [X]) ||
                  X &lt;- binary_to_list(Bin)]).

hexstr_to_bin(S) -&gt;
   hexstr_to_bin(S, []).
hexstr_to_bin([], Acc) -&gt;
   list_to_binary(lists:reverse(Acc));
hexstr_to_bin([X,Y|T], Acc) -&gt;
   {ok, [V], []} = io_lib:fread("~16u", [X,Y]),
   hexstr_to_bin(T, [V | Acc]).</pre>
<p>Then the code to actually connect to APN and send the PDU</p>
<pre>-module(ssltest).
-export([sendpush/0]).
-import(hex).

sendpush() -&gt;
  Address = "gateway.sandbox.push.apple.com",
  Port = 2195,
  Cert = "cert.pem",
  Key = "key-noenc.pem",  

  %Options = [{cacertfile, CaCert}, {certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Options = [{certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Timeout = 1000,
  {ok, Socket} = ssl:connect(Address, Port, Options, Timeout),
</pre>
<p>Open SSL socket to the APN server with application certificate and private key.</p>
<pre>  Payload = mochijson:encode({struct, [{"aps", {struct, [{"alert", "This is Message"}]}}]}),
  BPayload = erlang:list_to_binary(Payload),
  PayloadLen = erlang:byte_size(BPayload),</pre>
<p>Convert JSON payload to binary</p>
<pre>  Token = "7518b1c2c7686d3b5dcac8232313d5d0047cf0dc0ed5d753c017ffb64ad25b60",
  BToken = hex:hexstr_to_bin(Token),
  BTokenLength = erlang:byte_size(BToken),</pre>
<p>Convert token from hexadecimal string to binary</p>
<pre>  SomeID= 1,
  {MSeconds,Seconds,_} = erlang:now(),
  Expiry = MSeconds * 1000000 + Seconds + 3600*1,
</pre>
<p>Transaction id (can be always 0) and 1 hour  expiration time</p>
<pre>  Packet = &lt;&lt;1:8, SomeID:32/big, Expiry:32/big, BTokenLength:16/big, BToken/binary, PayloadLen:16/big, BPayload/binary&gt;&gt;,</pre>
<p>Construct the binary packet.</p>
<pre>  ssl:send(Socket, Packet),
  ssl:close(Socket).
</pre>
<p>Send the PDU and close the socket</p>
<h2>3. Listening for Errors</h2>
<p>In case something went wrong, Apple will send you back single error packet for the first error and closes the socket. You need to read that one error code. The packet that triggered error is identified by the ID you set when sending it.</p>
<p>See table 5-1 at <a href="http://developer.apple.com/library/ios/#documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/CommunicatingWIthAPS/CommunicatingWIthAPS.html#//apple_ref/doc/uid/TP40008194-CH101-SW1">Apple documentation</a> to interpret error codes.</p>
<p>Example error listener</p>
<pre>recv(Parent) -&gt;
   receive
       {ssl, Sock, &lt;&lt;Command, Status, SomeID:32/big&gt;&gt;} -&gt;
           error_logger:error_msg("Received",
                                  [Command, Status, SomeID]),
           ssl:close(Sock),
           Parent ! {error, SomeID}; % notify parent
      {ssl_closed, _Sock} -&gt; ok  %
   end.</pre>
<p>And remember to spawn process and set it as the controlling process after creating the socket</p>
<pre>  Pid = self(),
  ssl:controlling_process(Sock, spawn(fun() -&gt; recv(Pid) end)),</pre>
<p>Note that you need to implement also poller application to read feedback info from Apple Feedback server. This is very similar to the receiver above as it only needs to connect and wait for packets from Apple server until it closes the socket. See <a href="http://developer.apple.com/library/ios/#documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/CommunicatingWIthAPS/CommunicatingWIthAPS.html#//apple_ref/doc/uid/TP40008194-CH101-SW1">Apple documentation</a> for more in depth explanation.</p>
<p>&nbsp;</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/bravenewmethod.wordpress.com/65/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/bravenewmethod.wordpress.com/65/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/bravenewmethod.wordpress.com/65/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=bravenewmethod.wordpress.com&amp;blog=18135076&amp;post=65&amp;subd=bravenewmethod&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://bravenewmethod.wordpress.com/2011/02/16/apple-push-notifications-with-erlang/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/40ccd6f3c734870720f616b887d36c05?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">tikonen</media:title>
		</media:content>
	</item>
	</channel>
</rss>
