The fetch() method is called by the store the first time it encounters a new Query object, regardless of whether the query is local or remote.   It is also called by the store whenever you call refresh() on a record array related to a query.

 

The purpose of this method is to give the data source a chance to fetch any data related to the query.  For example, if you create a local query to find all Contact records, then your data source could be implemented to use this as a cue to retrieve contacts from the server as well.  

 

For local queries, all your data source needs to do is to potentially load data into the store.  It will not need to actually tell the store which records belong in the query results since they are computed locally anyway. However, your data source will need to tell the store that it has finished fetching data for the query by calling store.dataSourceDidFetchQuery().

 

For remote queries, the data source will need to tell the store which records belong in the query result set.  You do this by providing the store with a list of storeKeys that will be later converted into real records as they are needed by the application. One benefit of this model is that for a large data set, you can sometimes choose to load just a list of which records belong in the result set without loading the data itself—which can be much faster.

 

Sometimes, for very large data sets, even loading just the list of results—let alone the data—is simply too much to suck down over the internet.  In these cases, you can actually provide a promise to deliver storeKeys to the store instead of the list of storeKeys themselves.  You do this with a SparseArray, which is a special array-like object that will call your data source only when ranges in the sparse array are accessed by the application.  This technique is called incremental loading and it is covered in more detail in a section below.

 

The rest of this section contains sample implementations of the fetch() method depending on how you want to handle each query.

 

Fetching Local Queries

 

Local queries have their results computed automatically based on the records that are currently loaded into memory.  When your fetch() method is called, all you need to do is to make sure any records that you might want to appear in the query results are actually loaded into memory.

 

For example, let’s say you created a local query to find all contacts.  Your query code would look like this:

 

// in core.js

MyApp.CONTACTS_QUERY = SC.Query.local(MyApp.Contact);

 

// elsewhere in your app:

var contacts = MyApp.store.find(MyApp.CONTACTS_QUERY);

 

The first time find() is called with this query, the fetch() method on your data source will be called with the store and query as parameters.  All you need to do is make sure all the contacts are loaded from your server.  Here is how you might implement this in your application (this code uses SC.Request to handle the AJAX):

 

MyApp.DataSource = SC.DataSource.extend({

  ...

 

  fetch: function(store, query) {

    if (query === MyApp.CONTACTS_QUERY) {

      SC.Request.getUrl('/app/contacts?alt=json')

        .set('isJSON', YES)

        .notify(this, this._didFetchAllContacts, { query: query, store: store })

        .send();

    }

    // other handlers..

 

    return YES; // Not required, but good form.

  },

 

  _didFetchAllContacts: function(response, params) {

    var store = params.store;

    var query = params.query; 

 

    if (SC.$ok(response)) {

      // load the contacts into the store...

      store.loadRecords(MyApp.Contact, response.get('body'));

 

      // notify store that we handled the fetch

      store.dataSourceDidFetchQuery(query);

 

    // handle error case

    } else store.dataSourceDidErrorQuery(query, response);

  }

 

  //.. other data source methods

});

 

Notice that in the handler that manages the response from the server never actually provides the list of store keys related to the fetch back to the query.  Since the query is local, it will recompute automatically when you load the additional records into the store.

 

Fetching Remote Queries With Data

 

One of the more common reasons you might use a remote query is because the server will provide data to you in a particular order, which you want to maintain throughout your entire application.  

 

For example, assume you are fetching a list of top ten best selling books on your website.  The server request in this case will provide the list of books, in order of best sales.  Since you want to retain the actual order returned by the server, you could use a remote query to build this result.  Here is how you might define this query and request it in your application:

 

// in core.js:

Admin.TOP_SELLERS = SC.Query.remote(Admin.Book, { isTopSellers: YES });

 

// elsewhere in your app:

var topSellers = Admin.store.find(Admin.TOP_SELLERS);

 

This code will return a record array for the TOP_SELLERS query. The first time this record array is created, it will call fetch(), passing the store and the query.  Here is how you might implement fetch() for this query:

 

Admin.DataSource = SC.DataSource.extend({

  fetch: function(store, query) {

    if (query === Admin.TOP_SELLERS) {

      SC.Request.getUrl(‘/admin/top_sellers?alt=json’)

        .set(‘isJSON’, YES) 

        .notify(this, this._didFetchTopSellers, { store: store, query: query })

        .send();

    } // .. other cases

  },

 

  _didFetchTopSellers: function(response, params) {

    var store = params.store,

        query = params.query;

 

    if (SC.$ok(response)) {

      var storeKeys = store.loadRecords(Admin.Book, response.get('body'));

      store.loadQueryResults(query, storeKeys);

 

    // handle error 

    } else store.dataSourceDidErrorQuery(query);

  }

 

  // .. other methods

});

 

This code both loads the record data into the store and then takes the returned list of store keys and sets the to be the query results for the remote query.  The record array for the query will now update with the actual list of records in the order you provided them from the server.

 

Fetching Remote Queries Without Data

 

Sometimes you might use a remote query because you expect to receive a lot of data.  Rather than load all of this data up front, you might just load a list of the records that belong in the set then retrieve the actual records on demand.  

For example, let’s say you wanted to load a customer list.  You have a request on your server that will simply return a full list of the customer IDs on your system.  You can then fetch individual custom details on demand.  Here is how the query would be used in your application:

 

// in core.js:

Admin.CUSTOMER_LIST = SC.Query.remote(Admin.Customer);

 

// elsewhere in your app:

var customers = Admin.store.find(Admin.CUSTOMER_LIST);

 

This code will return a record array for the CUSTOMER_LIST query. The first time this record array is created, it will call fetch(), passing the store and the query.  Here is how you might implement fetch() for this query:

 

Admin.DataSource = SC.DataSource.extend({

  fetch: function(store, query) {

    if (query === Admin.CUSTOMER_LIST) {

      SC.Request.getUrl(‘/admin/customers?alt=json’)

        .set(‘isJSON’, YES) 

        .notify(this, this._didFetchCustomerList, { store: store, query: query })

        .send();

    } // .. other cases

  },

 

  _didFetchCustomerList: function(response, params) {

    var store = params.store,

        query = params.query;

 

    if (SC.ok(response)) {

 

      // map ids to store keys

      var storeKeys = response.get('body').map(function(id) {

        return Admin.Customer.storeKeyFor(id);

      }, this);

      store.loadQueryResults(query, storeKeys);

 

    // handle error 

    } else store.dataSourceDidErrorQuery(query);

  }

 

  // .. other methods

});

 

This will populate the record array for this query with the storeKeys you passed in.  Whenever you try to actually access one or more of these records, it will trigger a request to the store to retrieve the individual customer record.

 

[ TODO (skylar) : This behavior seems too "magical" - some more explanation on what is going on under the hood to cause the behavior is different would be helpful. For instance, what exactly tells the store that record data must be fetched for this id/storeKey later? Is it the simple difference that calling storeKeyFor() on the record class rather than allowing the store to create the storeKey itself notes the the full record must be fetched later? Is it the omission of actually calling loadRecords() or equivalent? What happens if I call loadRecords but only have an "id" in the data hashes? - will the full records still be fetched later as in the example above? ]

 

 

Fetching Remote Queries Using Incremental Loading

 

Sometimes your data set is so large you don’t even want to load the list of records up front.  In this case, the data source can actually provide the store not with a list of store keys but with a promise to provide a store keys when they are actually needed.  You can then implement additional methods to load the list of records as needed from the server.   This technique is called incremental loading.

 

Queries that use incremental loading look just like any other remote query:

 

// in core.js:

Admin.CUSTOMER_LIST = SC.Query.remote(Admin.Customer);

 

// elsewhere in your app:

var customers = Admin.store.find(Admin.CUSTOMER_LIST);

 

The code for fetch() is similar as well.  However since you just providing a promise to provide store keys instead of loading them directly, you don’t usually need a callback:

 

Admin.DataSource = SC.DataSource.extend({

  fetch: function(store, query) {

    if (query === Admin.CUSTOMER_LIST) {

      // setup sparse array with self as delegate

      store.loadQueryResults(query, SC.SparseArray.create({

        delegate: this

      }));

   } // .. other cases

  },

 

  // .. other methods

});

 

A sparse array is a special object that implements the SC.Array mixin.  However, whenever it is asked for key properties, such as the length or a range of objects at a given location, it will invoke related methods on its delegate instead.  The two methods you must implement are sparseArrayDidRequestLength() and sparseArrayDidRequestRange().  

 

sparseArrayDidRequestLength() should retrieve the total length of the array from the server and set it.  You can also predictively load some records as well.  sparseArrayDidRequestRange() should load a range of records from the server and set them.

 

The following example code shows how you might implement these two methods on a data source assuming your server has an API to return a range of records from a list and assuming the return JSON always contains a “count” property with the total number of records in the list.

 

Admin.DataSource = SC.DataSource.extend({

 

  fetch: function(store, query) {

    if (query === Admin.CUSTOMER_LIST) {

      // setup sparse array with self as delegate

      store.loadQueryResults(query, SC.SparseArray.create({

        delegate: this, store: store, query: query

      }));

   } // .. other cases

  },

 

  // since the length is always set whenever we load records, just load the

  // first 20 records anyway.  

  sparseArrayDidRequestLength: function(sparseArray) {

    return this.sparseArrayDidRequestRange(sparseArray, { start: 0, length: 50 });

  },

 

  sparseArrayDidRequestRange: function(sparseArray, range) {

    var url = “/admin/customers?alt=json&start=%@&length=%@”.fmt(range.start, range.length);

    SC.Request.getUrl(url).set(‘isJSON’, YES)

      .notify(this, this._didFetchCustomers, { 

         array: sparseArray, start: range.start, length: range.length

       }).send();

  }),

 

  _didFetchCustomers: function(response, params) {

    var sparseArray = params.array,

        start       = params.start,

        length      = params.length;

 

    if (SC.$ok(response)) {

      var count = response.count;

      var storeKeys = sparseArray.get(‘store’)

            .loadRecords(Admin.Customer, response.get('body').records);

      

      sparseArray.provideLength(count);  

      sparseArray.provideObjectsInRange({ start: start, length: length }, storeKeys);

      sparseArray.rangeRequestCompleted(start);

 

    }

  },

 

  // .. other methods

});

 

With this basic support implemented you should now be able to load data incrementally from the server.  If you use a CollectionView to display data - which uses incremental rendering to only display the data on screen - this implementation will actually load data incrementally as you scroll through the results.

 

Mixing and Matching

 

The above sections described the basic different ways you can handle fetch requests.  Of course, fetch will be called for a variety of different queries in your application.  Normally you will mix and match you implementation for each one using one of these techniques for each.

 

 

 

Moving On 

 

On to Retrieving Records » 

Back to DataStore Programming Guide Home »