Mongoose version 3 almost here and comes packed with new features, bugfixes, and performance improvements. This series of posts will cover the goodies and any important API changes you should be aware of.

versioning

This post will cover document versioning. To explain what document versioning is and why it is important, we first need to cover the state of things today in the 2.x branch.

Let’s say we have a blog post schema that contains an array of sub-documents, say comments:

1 2 3 4 5 6 7 
var commentSchema = new Schema({
    body: String
  , user: Schema.ObjectId
  , created: { type: Date, default: Date.now }
});
var postSchema = new Schema({ comments: [commentSchema] });
var Post = mongoose.model('Post', postSchema);

Now, suppose you get busy and write up a really great blog post, one you’re very proud of, and post it on hacker news. Surprise! Because of your amazing writing skills and content your post starts attracting a large number of views and comments.

What does our code look like to add a comment? It’s simple enough, we’ll just retreive this post and push a new comment object to it’s comments array:

1 2 3 4 5 
Post.findById(postId, function (err, post) {
  // handle errors ..
  post.comments.push({ body: someText, user: userId });
  post.save(callback);
})

I skipped all the form submission steps but you can see how we’re saving or comment. So far so good.

Now suppose a commenter realizes they posted something completely stupid and wish to fix their mistake:

1 2 3 4 5 6 
Post.findById(postId, function (err, post) {
  // handle errors ..
  var comment = post.comments.id(commentId);
  comment.body = updatedText;
  post.save(callback);
});

To see how this could be problematic we need to take a closer look at the underlying operation used to update the comment. When post.save() is executed, an update is issued to MongoDB that looks like the following:

posts.update({ _id: postId } , { $set: { 'comments.3.body': updatedText }})

Notice comments.3.body, this is called positional notation. This tells MongoDB to set the body of the comment in the comments array at index position 3 to the updated text.

Can you think of anything wrong with this? If another commenter on this blog post decides to delete their comment between the time this document was retrieved and the time the update sent, comments.3.body might specify the wrong sub-document since the position of our comment in the array may have changed. More generally, if any operation on our comments array causes the index of any comment to change we can get into hot water.

To mitigate this issue, Mongoose v3 now adds a schema-configurable version key to each document. This value is atomically incremented whenever a modification to an array potentially changes any array’s elements position. This value is also sent along in the where clause for any updates that require the use of positional notation. If our where clause still matches the document, it ensures that no other operations have changed our array elements position and it is ok to use use positional syntax.

So back to our previous example. Our update command in v3 now looks like the following:

posts.update({ _id: postId, __v: verionNumber } , { $set: { 'comments.3.body': updatedText }})

The version number is included in the where clause. If no document is found due to the version no longer matching or the document was removed from the collection, an error is returned to the callback that you can handle in your application:

post.save(function (err) { console.log(err); // Error: No matching document found. });

increment

In version 3, documents now have an increment() method which manually forces incrementation of the document version. This is also used internally whenever an operation on an array potentially alters array element position. These operations are:

$pull $pullAll $pop $set of an entire array

changing the version key

The version key is customizable by passing the versionKey option to the Schema constructor:

new Schema({ .. }, { versionKey: 'myVersionKey' });

Or by setting the option directly:

schema.set('versionKey', 'myVersionKey');

disabling

If you don’t want to use versioning in your schema you can disable it by passing false for the versionKey option.

schema.set('versionKey', false);

Try it today!

Go ahead and try out version 3 in your projects today and report any bugs you find or just give your feedback. The code is on github and available on npm under mongoose@3.0.0alpha1.

2

View comments

  1. Here's the new location: http://aaronheckmann.tumblr.com/

     

     

    0

    Add a comment

  2. If you’re using a mac, its easy to simulate slow networks for any mongod member using ipfw:

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 
    # first set up a pipe named "2" with bandwidth
    # limited to 16kbit/s and latency of 100ms
    sudo ipfw pipe 2 config bw 16Kbit/s delay 100ms

    # next add rule #1 using the pipe we set up, configured to
    # act only on local tcp traffic going to port 27019
    sudo ipfw add 1 pipe 2 tcp from me to me 27019 out

    # now you should be observing slow ping times for
    # mongod on 27019

    # let's disable rule #1
    sudo ipfw delete 1

    # back to normal!

    # our pipe still exists so if we want to enable it again
    sudo ipfw add 1 pipe 2 tcp from me to me 27019 out

    Obviously this isn’t just useful for mongodb, its helpful for any type of server testing etc. Too bad ipfw is marked as deprecated.

    0

    Add a comment

  3. Mongoose version 3 almost here and comes packed with new features, bugfixes, and performance improvements. This series of posts will cover the goodies and any important API changes you should be aware of.

    versioning

    This post will cover document versioning. To explain what document versioning is and why it is important, we first need to cover the state of things today in the 2.x branch.

    Let’s say we have a blog post schema that contains an array of sub-documents, say comments:

    1 2 3 4 5 6 7 
    var commentSchema = new Schema({
        body: String
      , user: Schema.ObjectId
      , created: { type: Date, default: Date.now }
    });
    var postSchema = new Schema({ comments: [commentSchema] });
    var Post = mongoose.model('Post', postSchema);

    Now, suppose you get busy and write up a really great blog post, one you’re very proud of, and post it on hacker news. Surprise! Because of your amazing writing skills and content your post starts attracting a large number of views and comments.

    What does our code look like to add a comment? It’s simple enough, we’ll just retreive this post and push a new comment object to it’s comments array:

    1 2 3 4 5 
    Post.findById(postId, function (err, post) {
      // handle errors ..
      post.comments.push({ body: someText, user: userId });
      post.save(callback);
    })

    I skipped all the form submission steps but you can see how we’re saving or comment. So far so good.

    Now suppose a commenter realizes they posted something completely stupid and wish to fix their mistake:

    1 2 3 4 5 6 
    Post.findById(postId, function (err, post) {
      // handle errors ..
      var comment = post.comments.id(commentId);
      comment.body = updatedText;
      post.save(callback);
    });

    To see how this could be problematic we need to take a closer look at the underlying operation used to update the comment. When post.save() is executed, an update is issued to MongoDB that looks like the following:

    posts.update({ _id: postId } , { $set: { 'comments.3.body': updatedText }})

    Notice comments.3.body, this is called positional notation. This tells MongoDB to set the body of the comment in the comments array at index position 3 to the updated text.

    Can you think of anything wrong with this? If another commenter on this blog post decides to delete their comment between the time this document was retrieved and the time the update sent, comments.3.body might specify the wrong sub-document since the position of our comment in the array may have changed. More generally, if any operation on our comments array causes the index of any comment to change we can get into hot water.

    To mitigate this issue, Mongoose v3 now adds a schema-configurable version key to each document. This value is atomically incremented whenever a modification to an array potentially changes any array’s elements position. This value is also sent along in the where clause for any updates that require the use of positional notation. If our where clause still matches the document, it ensures that no other operations have changed our array elements position and it is ok to use use positional syntax.

    So back to our previous example. Our update command in v3 now looks like the following:

    posts.update({ _id: postId, __v: verionNumber } , { $set: { 'comments.3.body': updatedText }})

    The version number is included in the where clause. If no document is found due to the version no longer matching or the document was removed from the collection, an error is returned to the callback that you can handle in your application:

    post.save(function (err) { console.log(err); // Error: No matching document found. });

    increment

    In version 3, documents now have an increment() method which manually forces incrementation of the document version. This is also used internally whenever an operation on an array potentially alters array element position. These operations are:

    $pull $pullAll $pop $set of an entire array

    changing the version key

    The version key is customizable by passing the versionKey option to the Schema constructor:

    new Schema({ .. }, { versionKey: 'myVersionKey' });

    Or by setting the option directly:

    schema.set('versionKey', 'myVersionKey');

    disabling

    If you don’t want to use versioning in your schema you can disable it by passing false for the versionKey option.

    schema.set('versionKey', false);

    Try it today!

    Go ahead and try out version 3 in your projects today and report any bugs you find or just give your feedback. The code is on github and available on npm under mongoose@3.0.0alpha1.

    2

    View comments

  4. JS RegExp ignoreCase
    0

    Add a comment

  5. To quote the wise Isaac Schlueter:

    Some programs are community centers, and you want to entice people to
    come inside.

    Some programs are factories, and you want to make sure the workers
    don't hurt themselves.

    Some programs are shops, and you want people to visit, but only a few
    go in the back.

    And then some programs are art, and while you might hope that some
    people find them beautiful or perhaps even useful, and contributions
    may be accepted and feedback always welcome, the real purpose is just
    to hang it in your house and let it bring you joy.

    There is not just one kind of thing in this world.

    --i
    0

    Add a comment

  6. I was just reading Juriy Zaytsev's excellent post about the delete operator and got to this statement:


    "Even if function is calling itself recursively, a new execution context is being entered with every invocation."

    This surprises me. I figured that javascript engines were able to detect recursion and reuse the same execution context somehow. So is recursion in javascript even more expensive than I thought? I knew that calling a function on each iteration of a loop was slow (compared to straight up loops) but how does recursion compare? To find out I set up a quick experiment to benchmark three different options:


    • Recursion

    • A function called on each iteration of a loop (function loop)

    • A vanilla loop (no function calls)

    Each benchmark will run a maximum of 490 iterations per test with each test running 400 times per page load. The test page will load 50 times in each browser. A maximum of 490 iterations is used because Safari 3.2 begins throwing "too much recursion" exceptions at 500 and there are a few overhead calls for each test. Here are is the code I used:

    Recursion

    function () {
    var i = 0,
    rec = function () {
    i++;
    if ( i < 490 ) rec();
    }
    ;
    rec();
    }

    Function loop

    function () {
    var i = 0,
    fn = function () {
    i++;
    }
    ;

    for (; i < 490; ) fn();
    }

    Vanilla loop

    function () {
    var i = 0;
    for (; i < 490; ) {
    i++;
    }
    }

    What were the results? Here's the breakdown by browser type. Green background means it's the fastest loop method when functions are involved. Bold text means it's the fasted loop method overall.


    Browserrecursion avgfn loop avgloop avg
    Firefox 3.5.7 Mac OSX 10.5.892.440200.4001.240
    Safari 3.2 Mac OSX70.62039.9807.660
    Safari 4 Mac OSX 10.5.86.5004.4200.600
    Firefox 3.5.7 WinXP76.680179.9402.580
    Firefox 3.0.7 WinXP63.18043.0206.460
    Firefox 2 WinXP861.340775.60014.300
    Chrome 4.0.288.1 WinXP1.7201.7200.140
    Safari 4 WinXP4.7803.7801.540
    Internet Explorer8 WinXP137.88066.2603.800
    Internet Explorer7 WinXP418.980388.18018.380
    Internet Explorer6 WinXP512.260479.86016.640
    Opera 10.10 WinXP22.86023.4008.460
    Safari 3.2.3 WinXP93.02052.22012.900
    Internet Explorer8 Vista192.760124.7009.360
    Total2555.0202383.480104.060

    Function loops were faster overall when nothing was happening on each iteration. Let's test again with a little more going on; hopefully somewhat closer to a real world application. The code:

    Recursion

    function () {
    var i = 0,
    rec = function () {
    var c = i + "a" + (Math.random() / i);
    i++;
    if ( i < 490 ) rec();
    }
    ;
    rec();
    }

    Function loop

    function () {
    var i = 0,
    fn = function () {
    var c = i + "a" + (Math.random() / i);
    i++;
    }
    ;

    for (; i < 490; ) fn();
    }

    Vanilla loop

    function () {
    var i = 0;
    for (; i < 490; ) {
    var c = i + "a" + (Math.random() / i);
    i++;
    }
    }


    Browserrecursion avgfn loop avgloop avg
    Firefox 3.5.7 Mac OSX 10.5.81030.1201542.2001174.200
    Safari 3.2 Mac OSX854.480764.040705.580
    Safari 4 Mac OSX 10.5.8523.380519.420534.000
    Firefox 3.5.7 WinXP750.0201125.260712.780
    Firefox 3.0.7 WinXP776.860787.2601270.620
    Firefox 2 WinXP2458.4802726.2001506.620
    Chrome 4.0.288.1 WinXP436.360448.200451.060
    Safari 4 WinXP370.180367.380365.240
    Internet Explorer8 WinXP691.160583.160450.840
    Internet Explorer7 WinXP1753.1601711.1401153.140
    Internet Explorer6 WinXP1357.2401289.140841.480
    Opera 10.10 WinXP682.420656.040626.000
    Safari 3.2.3 WinXP897.320846.180752.440
    Internet Explorer8 Vista723.300618.580481.060
    Total13304.4813984.19911025.06

    Keep in mind these tests did not test looping speeds only, they were effectively testing each browsers Math.random(), string concatenation speeds, etc.


    So what does this tell us?


    The results aren't very clear in terms of execution times. Strict tests with nothing going on in each iteration tended to lean toward function loops being faster than recursion. Tests with more going on generally leaned toward recursion being faster in browsers that have tracing techniques enabled. Firefox optimized more for recursion than in other browsers. In Internet Explorer and Safari function loops were always fastest. Opera and Chrome were pretty much a wash.

    If we think of "fastest" not in terms of overall execution time but also in terms of how many actual users would benefit from either technique, then function looping is the winner since IE still has the most market share.


    Conclusion


    The take away on this is that it doesn't matter if you use recursion or function loops, if you really want to be fast, inline your code and refrain from any function execution at all.



    0

    Add a comment

  7. I recently started thinking about a plugin manager for jQuery, something that could integrate with a server side file rollup service like the YUI Configurator. I looked around a bit and didn't see anything quite like I wanted so I decided to roll my own.

    Surprises

    There were a few surprises along the way that I wanted to document, because I don't want to forget them and hopefully you'll find them helpful too.


    1. Detecting when stylesheets are loaded

    2. Detecting when downloaded javascript was parsed by the browser

    3. Fully resolving URIs cross-browser

    This post will talk about fully resolving URIs cross-browser.


    Overview

    If you've ever spent time reading URIs from anchors, scripts, or links, you probably know what this post is about. Reading these attributes cross-browser isn't straight-forward. For example, let's start with the following script tag:


    <script src="/myscript.js" id="myscript"></script>

    Now let's read the src attribute, simple right?


    var script = document.getElementById("myscript");
    alert(script.src);

    In Firefox, Safari, Chrome, and Opera we see http://absolute/path/to/myscript.js. But in Internet Explorer we see myscript.js. So how do we go about absolutely resolving URIs cross-browser?


    The Solution

    It turns out that if we first create an anchor element using .innerHTML and subsequently read it's href attribute, it will be absolutely resolved.


    var resolve = (function () {
    var div = document.createElement("div");
    return function (URI) {
    div.innerHTML = "<a href='" + URI + "'></a>";
    return div.firstChild.href;
    }
    })();

    Now let's adjust our original example to use our new resolve function and we'll be all set.


    var script = document.getElementById("myscript");
    alert( resolve(script.src) );

    Conclusion

    Hopefully this was somewhat helpful, and if not, thanks for reading anyway.

    If you're interested in using the jQuery plugin manager in your own project or are just curious, there is a demo available here and the source code is up on Github.

    1

    View comments

  8. I recently started thinking about a plugin manager for jQuery, something that could integrate with a server side file rollup service like the YUI Configurator. I looked around a bit and didn't see anything quite like I wanted so I decided to roll my own.

    Surprises

    There were a few surprises along the way that I wanted to document, because I don't want to forget them and hopefully you'll find them helpful too.


    1. Detecting when stylesheets are loaded

    2. Detecting when downloaded javascript was parsed by the browser

    3. Fully resolving URIs cross-browser

    This post will talk about detecting when downloaded javascript was parsed by the browser.

    Getting Started

    You may be wondering why I bother writing a post about detecting when downloaded javascript has been parsed by the browser. Well, the reason is that in my testing I found that Internet Explorer rarely threw exceptions when using globals that should have been available when my success callback fired. So I came up with a work around that continually tests whether all of the properties specified are available in the DOM.


    var _providesAvailable = function (provides) {
    var available = true,
    i = 0,
    len = provides.length,
    names,
    nLen,
    ns,
    j
    ;

    while ( available && i < len ) {
    names = provides[i].split('.');
    nLen = names.length;
    ns = window;
    j = 0;

    for (; j < nLen; j++) {
    if ( !(names[j] in ns) ) {
    available = false;
    break;
    }
    ns = ns[names[j]];
    }
    i++;
    }
    return available;
    };

    var arrayOfExpectedProps = [
    'foo.bar',
    'something.completely.different'
    ],
    cb = function () {
    if (!_providesAvailable(arrayOfExpectedProps)) {
    return setTimeout(cb, 30);
    }
    console.log("done");
    }
    ;

    cb();


    While not perfect (if I pass in anything other than an array to _providesAvailable it will fail) it's fine for what I'm doing. I've since switched to using jQuery's ajax method and haven't had any additional problems with this bug but I still left the _providesAvailable method enabled for now just to be safe.

    Plugin Registration

    With a way to now detect when global properties are in the DOM, we can specify which properties that must be available before our success callback is fired for any of our registered plugins:


    jQuery.use.add({
    name: 'jquery-myplugin',
    file: ['someFile.js', 'somestyles.css'],
    provides: 'jQuery.myPlugin.sweetness'
    });

    jQuery.use('jquery-myplugin', function ($) {
    // window.jQuery.myPlugin.sweetness is now available
    });

    In the above example we use the jQuery.use.add method to register our plugin. name determines which unique key you'll use to pull in all of the assets you specify, file determines which file[s] to request, and provides determines the global properties that need to be available in the DOM before firing your callback. For more information about what options are available, see the project on Github.

    Conclusion

    We've seen why we need a method to detect when dynamically created scripts are appended to the DOM and how to go about detecting when they're truly ready for use. Hopefully this was somewhat helpful, and if not, thanks for reading anyway.

    If you're interested in using the jQuery plugin manager in your own project or are just curious, there is a demo available here and the source code is up on Github. Have fun!

    What's Next

    In my next post I'll cover fully resolving URIs cross-browser.


    1

    View comments

  9. I recently started thinking about a plugin manager for jQuery, something that could integrate with a server side file rollup service like the YUI Configurator. I looked around a bit and didn't see anything quite like I wanted so I decided to roll my own.

    Surprises

    There were a few surprises along the way that I wanted to document, because I don't want to forget them and hopefully you'll find them helpful too.


    1. Detecting when stylesheets are loaded

    2. Detecting when downloaded javascript was parsed by the browser

    3. Fully resolving URIs cross-browser

    This post will talk about detecting when stylesheets are loaded. See a demo or fork the Gist on Github.

    Getting Started

    In order to get things moving we'll need a link tag pointed at some css:


    var css = document.createElement('link');
    css.type = 'text/css';
    css.rel = 'stylesheet';
    css.href = "http://groups.google.com/groups/style.css?ig=1&stock=1&av=4&hl=en&v=639";

    // I chose a different domain here because I want to
    // support loading stylesheets from any environment.

    Implementation

    Now on to the good stuff. Surprise! In Internet Explorer and Opera life is easy. You can detect when the stylesheet loads by attaching an event listener directly to the element and listening for it's load event. This even works cross-domain.


    jQuery(css).bind('load', function () {
    // this stylesheet loaded!
    });

    But this doesn't cut it for other browsers since they don't implement the load event on stylesheets. Really, they don't. So I took a trip down a path of possible workarounds.

    In Safari and Chrome we can detect when the stylesheet is loaded as soon as it's sheet.cssRules properties are available.


    (function sheetLoaded () {
    try {
    css.sheet.cssRules;

    } catch (e) {
    setTimeout(sheetLoaded, 50);
    return;

    }
    // if we got here the stylesheet is loaded
    })();

    So far so good. This even works on stylesheets loaded from other domains. But what about Firefox? Unfortunately, I could only get Firefox to cooperate with this technique when loading stylesheets from the same domain. If anyone knows a work around feel free to share it in the comments below or grab the Gist and fork away. For now, I've left detecting when stylesheets are loaded out of my plugin manager since I want to provide consistent behavior across all supported browsers.

    Update

    Julian Aubourg has been working on rewriting the jQuery ajax logic and discovered a solution to detecting when stylesheets load in Firefox.


    (function sheetLoaded () {
    try {
    css.sheet.cssRules;

    } catch (e) {

    if ( "NS_ERROR_DOM_SECURITY_ERR" != e.name ) {
    setTimeout(sheetLoaded, 50);
    return;

    }
    }

    // if we got here the stylesheet is loaded

    })();

    Here's how it works. Gecko (Firefox's layout engine) throws an NS_ERROR_DOM_INVALID_ACCESS_ERR exception if you try to access the cssRules property while the link element is loading. Once the stylesheet loads it either stops throwing exceptions (in the case of same domain requests) or throws a different NS_ERROR_DOM_SECURITY_ERR exception. By watching for the security exception we can safely determine when the stylesheet is loaded.

    I updated the Gist and plan on implementing his great technique soon.


    Tested Browsers


    • Safari 4 Mac

    • Firefox 3.5.7 Mac

    • Chrome 3.0.195.33 WinXP

    • Firefox 3.5.1 WinXP

    • Opera 9.64 WinXP

    • Opera 10.10 WinXP

    • IE 6-8 WinXP

    • Safari 4 Vista

    • Firefox 3.5.2 Vista

    • IE 7, 8 Vista

    What's Next

    In my next post I'll cover detecting when downloaded javascript is parsed by the browser.


    0

    Add a comment

  10. Maybe this is a bit too Big Brother but I think that places like libraries and movie theaters should broadcast a signal, that when detected by a mobile device, the device switches automatically to silent mode.

    This could work for airlines too. An airline could broadcast a signal that forces devices to either shut down or switch to an acceptable mode during critical times like take-off and landing.

    Too big brother? Maybe. But providing greater context-sensitivity to our mobile world has huge potential.
    0

    Add a comment

Loading